Module in_flight

Module in_flight 

Source
Expand description

In-memory marker for “a data-model write is running right now for this cache key” (issue #5129).

The key is the data-model ENTRY key (data_model_cache_key), not the request key it is built from (#6034): the full and the rooted-only tables are separate entries, and a fill of one says nothing about the other.

get_data_model needs to tell a client “not yet, keep polling” apart from “not for this key, stop asking”: the first is 202, the second 404. A cache miss alone cannot make that distinction — the entry could be absent because nothing ever asked for it, which is exactly the case the issue’s 404 is for. This map is the missing signal: a key is present in it for the lifetime of every background task that could still write it (parse_parquet_stream’s data-model fill), and gone only once every such task for that key has ended, however each one ends.

Refcounted, not a plain set (PR #5134 review): two concurrent uploads of the same content each spawn their own fill and each call begin with the same cache key. A plain set’s remove on either Drop would clear the key while the other fill is still running, and a GET racing that window would see 404 for a fill that is, in fact, still in progress.

Structs§

InFlightGuard
RAII marker returned by InFlightKeys::begin. Decrements its key’s count on drop – on task completion, on early return (e.g. admission saturated), and on panic unwind alike – removing the key only once the count reaches zero, so one of two concurrent fills for the same key ending can never un-mark the other’s still-running fill.
InFlightKeys
The refcounted set of cache keys with a data-model write in flight. One instance lives on AppState, shared by every request. The count is how many live guards currently name that key, not a byte size or anything meaningful beyond “still positive”.