replay_by_client_hash

Function replay_by_client_hash 

Source
pub(super) async fn replay_by_client_hash(
    state: &AppState,
    query: &ParseQuery,
    quality: TessellationQuality,
    stream_shapes: StreamShapes,
    sha256: &str,
) -> Result<Response, ApiError>
Expand description

Serve POST /api/v1/parse/parquet-stream from a client-supplied file hash, with no request body at all (issue #3901).

The upload used to be unavoidable on a hit: the cache key is the SHA-256 of the RECEIVED bytes, so extract_file had to finish before the cache could be consulted, and on a 40 MB model over a real connection that upload was the whole cost of the hit. A client that hashes locally can name the entry instead.

The hash SELECTS; it never asserts. Everything the replay needs has to be on disk under that key already, which is exactly what try_cached_replay checks (geometry body, metadata header, a data model at the current payload version, plus whatever it later adds). Delegating to it rather than re-listing those entries here is what makes “a hash-only hit replays the same events as an upload hit” true by construction instead of by two lists agreeing. A miss answers 404, the status GET /api/v1/cache/check/{hash} already uses for “upload it”, and the client uploads.

A MISS costs no admission slot. The probe answers it from two small reads (the metadata header and the data-model marker) before touching the gate, because the common miss is a file the server has never seen and charging a parse slot for a disk lookup would mean every cold-cache client wins admission twice, probe then upload, and could be shed on the probe rather than queueing for the upload that would have succeeded.

A HIT does take one, around the replay BUILD, dropped before the response is returned. That is what the body-carrying hit path does, and for the same reason: try_cached_replay reads the whole geometry blob into memory, base64-encodes every batch, and materializes the full event vector before a byte is sent, which is several times the model’s geometry resident at once. Leaving that window unbounded would make the cheapest request a client can send the cheapest way to exhaust the server. Draining the finished stream needs no slot.