Module parquet_optimized 

Optimized Parquet serialization using ara3d BimOpenSchema format.

Key optimizations over basic Parquet: integer quantized vertices (10,000x multiplier = 0.1mm precision), mesh dedup via content hashing PLUS rotation-aware representation-identity dedup (#3575), byte colors (0-255 instead of float 0-1), optional normals, material dedup.

Typical additional compression: 3-5x over basic Parquet format.

Structs

MaterialKey 🔒

Material key for deduplication.

OptimizedStats

Statistics about the optimized serialization.

Constants

VERTEX_MULTIPLIER

Vertex multiplier for integer quantization. 10,000 = 0.1mm precision.

Functions

assemble_optimized_output 🔒

Header: [version:u8][flags:u8][instance_len:u32][mesh_len:u32][material_len:u32][vertex_len:u32][index_len:u32] Then: [instance_parquet][mesh_parquet][material_parquet][vertex_parquet][index_parquet]

check_optimized_section_lengths 🔒

Validate the five optimized-writer section lengths against the u32 wire limit. Takes plain lengths, not byte slices, so a >4 GiB test case is just the number u32::MAX as usize + 1 — no multi-gigabyte buffer needed.

color_to_byte 🔒

Convert float color (0-1) to byte (0-255).

quantize_position 🔒

Quantize a float position to integer (0.1mm precision).

serialize_to_parquet_optimized 🔒

Serialize mesh data to optimized Parquet format (ara3d BOS-compatible): instances table (entity → mesh/material index, origin, rotation), meshes table (unique geometries), materials table, vertices table (quantized integers), indices table. Dedup covers both bit-identical placements (content hash) and, as of issue #3575, one shape at DIFFERENT rotations (IfcMappedItem / shared IfcRepresentationMap reuse — furniture, pipe runs, repeated structural members).

serialize_to_parquet_optimized_with_stats

Serialize with stats.

write_parquet_buffer 🔒

Write a RecordBatch to a Parquet buffer with LZ4 compression. Dictionary encoding is disabled for numeric columns (floats, integers) as they have high entropy and dictionary encoding provides no benefit while adding significant overhead.