Feature availability is not event time: building an honest training snapshot

Ask what the model could know at the decision

A feature can have an old event timestamp and still be new information. Suppose a sensor observation describes 09:00, reaches the platform at 09:12, and becomes queryable after validation at 09:16. A prediction issued at 09:05 could not have used it. Joining on the observation time alone places future knowledge into the historical decision, even if every row in the training table looks chronologically sensible.

I would make that distinction part of the dataset contract before comparing models. This study uses a hypothetical equipment-monitoring task; it makes no claim about a deployed maintenance predictor. The question is how to assemble examples that reflect an information boundary. A model evaluation becomes much less persuasive if the offline pipeline knows more than the service it is meant to represent.

Keep the clocks separate

For each feature record, I would distinguish event time, source revision time, ingestion time, and availability time. Event time describes the observation. A source revision identifies a correction. Ingestion time records arrival at a particular platform boundary. Availability time is the earliest point at which the serving path could use that version under the defined validation and publication rules. The last field cannot always be reconstructed from the first three.

For example, an ingestion job may validate a batch and publish it atomically only after checking all records. Every record arrived earlier than the batch became visible. A second system might publish independently by entity. Both can be reasonable, but they imply different availability histories. I would record the publication boundary explicitly rather than filling an availability column with whatever timestamp happens to exist in the raw source.

Clock precision also matters. Two updates can share a timestamp, and clocks across services may disagree. A deterministic version or sequence within the relevant entity and feature domain is useful for ties. That sequence does not establish an order across unrelated sources. The training contract must state what is comparable and how ambiguous cases are handled, especially when source records are corrected after the original prediction.

An as-of join needs the right meaning of as-of

Feast's point-in-time join documentation describes retrieving historical feature values relative to entity timestamps and a feature-view TTL. That provides a useful vocabulary for historical retrieval. It should not be read as evidence that any event-time join automatically enforces the availability rule needed for this example. I would inspect the generated query and the versioned storage model rather than infer semantics from the operation's name.

Here the admissibility rule is available_at ≤ prediction_at, together with the requirement that the observation itself is not in the future. The illustrative query selects the most recent eligible observation and then its latest eligible revision. It assumes append-only revision history for one named feature. If that history was overwritten, SQL cannot recover the values that were actually visible at the earlier decision.

A separate freshness rule belongs in the predicate. The example uses a two-hour observation window solely to demonstrate the mechanism. That is a design parameter, not a recommended operational setting. When no eligible observation exists, the left join retains a missing feature. Quietly dropping those predictions would alter the evaluated population and hide the conditions under which the live model has the least information.

1-- Hypothetical PostgreSQL pattern for one feature.
2SELECT p.prediction_id, f.value, f.available_at, f.version
3FROM predictions AS p
4LEFT JOIN LATERAL (
5  SELECT value, available_at, version
6  FROM feature_history AS h
7  WHERE h.entity_id = p.entity_id
8    AND h.feature_name = 'temperature_mean'
9    AND h.event_at <= p.prediction_at
10    AND h.available_at <= p.prediction_at
11    AND h.event_at > p.prediction_at - INTERVAL '2 hours'
12  ORDER BY h.event_at DESC, h.available_at DESC, h.version DESC
13  LIMIT 1
14) AS f ON TRUE;

The target has its own availability problem

Features are only half of the temporal contract. A label can arrive after the event it describes, and different entities can have different observation windows. Suppose the target is a confirmed equipment fault within the next seven days. An example created yesterday does not yet have a complete seven-day outcome. Treating its missing label as a negative introduces a different kind of error from feature leakage, but the resulting validation score can be just as misleading.

I would define label maturation, exclusions, and the date through which outcomes were observable. A training cutoff then has at least two roles: it limits eligible feature information and determines which examples have sufficiently complete outcomes. For a prospective evaluation, those rules are frozen before scoring. A later correction to a label should create a new dataset revision, not silently change the result behind an existing report.

Splitting by time is necessary for many forecasting questions, but it does not by itself prevent repeated entities or overlapping windows from leaking information across the boundary. I would examine whether a machine, customer, document family, or duplicated observation appears on both sides. The appropriate split follows the intended generalization question. A claim about unseen equipment needs a different experiment from a claim about future observations on already known equipment.

Test the join with examples that should be excluded

Before training anything, I would construct a small fixture containing a late observation, a corrected record, an exact-boundary timestamp, a missing value, and two versions with the same observation time. Each expected result can be written down by hand. The point is to test the information rule independently of the model. A sophisticated predictor cannot tell me that the feature join selected a revision unavailable at prediction time.

One useful test starts with a fixed historical prediction, then appends a later-arriving correction. Rebuilding the original snapshot should leave that prediction's features unchanged when the correction was unavailable at the cutoff. A second test advances the cutoff beyond the correction's availability and expects the new revision. These paired tests make a subtle temporal property visible without inventing an accuracy benchmark.

I would inspect missingness by source delay, entity type, and time period before treating it as noise. A missing value can represent a normal operating condition, a failed pipeline, or a record rejected by validation. Those meanings may require different model inputs or serving behavior. An imputation strategy belongs inside the training split, and any indicator describing missingness must itself be available at prediction time.

Publish a dataset argument, not just a model score

The result of this work should be a versioned snapshot with a reproducible query, source versions, prediction boundaries, and label rules. It should state the number of eligible examples and why others were excluded. I would keep the small temporal fixtures alongside that build. They explain the contract to the next engineer more directly than a diagram of services can.

The model report can then compare baselines, calibration, and error slices on the same snapshot. If a new feature improves a metric, I want to know whether it contributes information available to the intended decision or merely benefits from a more permissive reconstruction. That question changes the order of the work: establish what could be known, preserve the examples, and only then ask what the model learned. The improvement must survive the data contract that production actually has.