Expand description
The per-batch progress checkpoints a live parquet-stream parse reported, cached so a later cache hit can replay the SAME numbers.
progress events are in units of geometry JOBS, not meshes: the pipeline
calls its batch callback with processed_jobs / total_jobs
(processing/src/processor/mod.rs), and a job can produce several meshes
or none. A replay that counted the meshes it emitted would therefore
report different units AND a different total from a miss on the same file,
over the same public event. Nothing in the geometry blob records the job
counts, so the live path writes them here and the replay reads them back.
Structsยง
- Stream
Progress ๐ - What a live parse reported, in job units.
- Stream
Progress ๐Recorder - Collects
StreamProgressfrom the live event stream.
Functionsยง
- cache_
stream_ ๐progress - Store the checkpoints beside the geometry. Best-effort: a failure here costs the next cache hit its job-unit progress, not its correctness.
- load_
stream_ ๐progress - Read back the checkpoints for a cache hit.
Nonefor an entry written before this sidecar existed, or one that no longer deserializes. - stream_
progress_ ๐cache_ key - Cache key for the checkpoint sidecar, keyed alongside the geometry and metadata entries for the same request.