Module parquet 

Parquet serialization for efficient mesh data transfer.

Uses columnar format (ara3d BOS-compatible) for dramatically better compression compared to JSON serialization. Typical compression ratios:

• JSON: ~30KB per mesh with ~500 vertices
• Parquet: ~2KB per mesh (15x smaller)

Structs

StreamingParquetCacheWriter

Incremental whole-model cache writer for the streaming endpoint: each batch’s columns are appended as one Parquet row group per table, so no MeshData has to be retained past the batch that produced it (previously the endpoint kept a FULL second copy of the model’s meshes just to re-serialize them at Complete). The mesh-table vertex_start/index_start columns carry GLOBAL offsets (whole-model), matching what the one-shot serialize_to_parquet emits for the cached fast-path replay.

Enums

ParquetError

Errors during Parquet serialization.

Functions

check_u32_len 🔒

Fail loud instead of silently truncating a wire-format u32 length prefix when a section exceeds 4 GiB.

frame_combined_sections 🔒

Assemble the three Parquet buffers directly into the OUTER combined framing the parse endpoints wrap the geometry blob in: [geo_len:u32][geo_bytes][data_model_len=0:u32], where geo_bytes is exactly frame_sections’s [mesh_len][mesh][vertex_len][vertex][index_len][index] layout. Endpoints that don’t attach a data model inline (the streamed cache fill) previously called frame_sections for the inner blob and then copied that whole blob a second time into an outer Vec to add the [geo_len]...[dm_len=0] wrapper. Writing both frames into one pre-sized allocation skips that second copy; the resulting bytes are identical to the old two-copy path.

frame_sections 🔒

Assemble the three Parquet buffers into the length-prefixed section layout shared by the whole-model serializer and the incremental cache writer. Section lengths are u32 on the wire; fail loud instead of truncating a section over 4 GiB into a silently corrupt blob.

serialize_batch_with_layout

Serialize one batch under the layout the client asked for, sharing nothing.

serialize_combined_for_layout 🔒

The whole-model body of POST /api/v1/parse/parquet under the layout the client asked for, already in the outer [geo_len][geo_bytes][data_model_len=0] framing. Only SharedShapes runs the shape-sharing planner; a default request gets the pre-#3888 bytes (#3888). Framed by frame_combined_sections, which guards the outer length (it can pass u32::MAX while every section passes) and writes both frames in one allocation.

serialize_with_plan 🔒

write_parquet_buffer 🔒

Write a RecordBatch to a Parquet buffer with LZ4 compression. Dictionary encoding is disabled for numeric columns (floats, integers) as they have high entropy and dictionary encoding provides no benefit while adding significant overhead.

write_sections 🔒

The mesh, vertex and index tables, each written as its own Parquet buffer.

writer_props 🔒

Writer properties shared by the one-shot and incremental writers: LZ4, and dictionary encoding disabled for numeric columns (high-entropy vertex data gains nothing from a dictionary while paying significant overhead).

Type Aliases

Sections 🔒

The mesh, vertex and index Parquet buffers, in wire order.