Module parquet_replay_batches 

Splits a cached combined-geometry Parquet blob back into the per-batch chunks a live parse would have streamed, so a cache hit on POST /api/v1/parse/parquet-stream can replay the same Start / Batch+ Progress / Complete event shape as a miss (issue #3895) instead of one oversized Batch.

StreamingParquetCacheWriter (parquet.rs) appends one Parquet row group per live stream batch to each of the three section tables (mesh / vertex / index) — an invariant that writer pins by disabling the row-count-driven row-group split. This module reads those boundaries back out: the vertex/index tables are already batch-local (no offset is baked into their columns), and the mesh table’s vertex_start / index_start columns are re-based from whole-model globals back to batch-local zero, so the re-encoded per-batch blob matches what the live per-batch serializer (serialize_to_parquet, base offsets 0/0) would have produced for that same batch.

A cross-batch client (stream_shapes=cross-batch, #5407) gets each batch as written instead, whole-stream offsets and all, with its bases stated beside it: split_into_stream_batches. A blob a cross-batch stream wrote cannot be re-based batch-locally at all, since its rows point into earlier row groups, so split_into_batches refuses it and a batch-local client is replayed the whole blob as one batch, which decodes correctly, only not progressively.

Structs

ReplayBatch

One reconstructed batch, ready to base64-encode into a Batch SSE event.

Section 🔒

One Parquet section of the cached blob, with its footer parsed ONCE. Re-opening a ParquetRecordBatchReaderBuilder per row group re-parses the whole footer each time, which is quadratic in the row-group count — hundreds of milliseconds on a model that streamed a few hundred batches.

Functions

column_sum 🔒

Sum a non-nullable UInt32 mesh column, dividing each value by per_row first. The division is PER ROW, not on the total, because build_mesh_tables counts index-table rows the same way — one row per indices.len() / 3 per mesh, so a mesh whose index count is not a multiple of 3 rounds down on its own, not against its neighbours.

encode_batch 🔒

furthest_range_end 🔒

The furthest start + count any row of rb names (0 for no rows). None when either column is missing or not UInt32.

localize_mesh_batch 🔒

Re-base a mesh-table row group’s vertex_start / index_start columns from whole-model globals back to batch-local zero, by subtracting the group’s own first-row values — the exact base offset StreamingParquetCacheWriter::append fed into build_mesh_tables for that batch. Every other column is untouched. Columns are resolved BY NAME, so a schema reorder can never silently re-base the wrong one.

rebase_to_zero 🔒

Subtract a UInt32 column’s first value from every value. checked_sub rather than -: on a foreign blob whose offsets are not non-decreasing this would panic in debug and, with the release profile’s overflow-checks = false, WRAP to a ~4-billion offset that decodes as valid geometry. None instead sends the caller to its fallback.

split_geometry_sections 🔒

Split the triple-framed [mesh_len][mesh][vertex_len][vertex][index_len] [index] geometry blob into its three sections. None on any short or mismatched-length framing, or trailing bytes past the third section.

split_into_batches

Split a cached geometry blob (the inner section framing cached_geometry_slice returns) back into per-original-batch chunks.

split_into_stream_batches

Split a cached geometry blob into its batches for a CROSS-BATCH client (#5407): each batch keeps its whole-stream offsets and states the bases its vertex and index tables start at, exactly as the live cross-batch stream does. Works on a blob either kind of stream wrote, because a batch-local stream’s cache rows are whole-stream too and simply never point backwards.

take_length_prefixed 🔒