feat: Add enabled/disabled toggle for feature views#6401
Conversation
f74306c to
4685564
Compare
| if hasattr(existing_proto.meta, "state") and hasattr(updated_fv, "state"): | ||
| state_val = getattr(updated_fv, "state") | ||
| if isinstance(state_val, FeatureViewState): | ||
| existing_proto.meta.state = state_val.to_proto() |
There was a problem hiding this comment.
Every feast apply resets all feature views back to CREATED, even if they were already AVAILABLE_ONLINE
There was a problem hiding this comment.
both feast apply paths now preserve the existing state from the registry instead of overwriting it with the default STATE_UNSPECIFIED. Works across all registry backends.
| print(f"Feature view '{name}' is already in state {new_state.name}.") | ||
| return | ||
|
|
||
| fv.state = new_state |
There was a problem hiding this comment.
I think this needs validation, It's better to add Valid transitions maps so that user is not able to bypass or skip some state
There was a problem hiding this comment.
Added _VALID_STATE_TRANSITIONS map and can_transition_to() validation in the set-state command. Invalid transitions are now rejected with a message showing allowed transitions.
| (start_date, end_date) | ||
| ) | ||
| existing_feature_view.last_updated_timestamp = _utc_now() | ||
| # Transition state to AVAILABLE_ONLINE after materialization. |
There was a problem hiding this comment.
We are trying to set AVAILABLE_ONLINE state at multiple places, either we should set this state in feature_store.py or registry.py. Since _materialize_single_feature_view sets state to AVAILABLE_ONLINE and then calls registry.apply_materialization which also sets it to AVAILABLE_ONLINE, this is redundant.
There was a problem hiding this comment.
Removed the duplicate from feature_store.py.
| end_date = utils.make_tzaware(end_date) or _utc_now() | ||
|
|
||
| # Transition state to MATERIALIZING before starting. | ||
| previous_state = getattr(feature_view, "state", None) |
There was a problem hiding this comment.
Similar to cli, can_transition_to() check can be enforced here as well
There was a problem hiding this comment.
Added can_transition_to() check in both materialization paths, guarded with state != STATE_UNSPECIFIED for backward compatibility.
| fv = store.registry.get_any_feature_view(name, store.project) | ||
| except FeastObjectNotFoundException as e: | ||
| print(e) | ||
| exit(1) |
There was a problem hiding this comment.
Use sys.exit(1) or ctx.exit(1) instead
Signed-off-by: RutujaPathade <[email protected]>
Signed-off-by: ntkathole <[email protected]> Signed-off-by: RutujaPathade <[email protected]>
Signed-off-by: RutujaPathade <[email protected]>
Signed-off-by: RutujaPathade <[email protected]>
…+ join nodes (feast-dev#6395) * fix: Apply field mapping to join keys in local compute engine nodes When a batch source defines a `field_mapping` that renames an entity join key (e.g. `USERID` -> `user_id`), the source-read node renames the columns on the pulled Arrow table to their mapped names. Downstream `LocalDedupNode` and `LocalJoinNode` then look up the *pre-mapping* names from `column_info.join_keys`, which raises `KeyError: Index(['USERID'])` during materialization (or returns an empty join). Add a `join_keys_columns` property on `ColumnInfo` that mirrors the existing `timestamp_column` / `created_timestamp_column` properties — returning join keys translated through `field_mapping` — and use it from the dedup and join nodes. Fixes feast-dev#5942. Signed-off-by: 1fanwang <[email protected]> * test: also cover LocalJoinNode field_mapping case Signed-off-by: 1fanwang <[email protected]> --------- Signed-off-by: 1fanwang <[email protected]> Signed-off-by: RutujaPathade <[email protected]>
…east-dev#6354) Signed-off-by: ntkathole <[email protected]> Signed-off-by: RutujaPathade <[email protected]>
…e examples Signed-off-by: jvincent-mongodb <[email protected]> Signed-off-by: RutujaPathade <[email protected]>
The requested_features parameter was accepted by online_read and online_read_async but never used -- DynamoDB always fetched all features stored in the values map regardless. Add a ProjectionExpression to BatchGetItem requests when requested_features is provided, reducing data transfer, latency, and read costs. Fixes feast-dev#6058 Signed-off-by: Jonathan Wrede <[email protected]> Signed-off-by: RutujaPathade <[email protected]>
The return dict contains both str and Dict[str, str] values, so the return type must be Dict[str, Any] not Dict[str, str]. Signed-off-by: Jonathan Wrede <[email protected]> Signed-off-by: RutujaPathade <[email protected]>
c050656 to
86e81b8
Compare
…te_batch When pushing features with array/list types (e.g. STRING_LIST) to BigQuery via offline_write_batch, the data arrives as empty arrays because BigQuery's parquet loader does not infer list structure by default. Set parquet_options.enable_list_inference = True on the LoadJobConfig so array columns are written correctly. Fixes feast-dev#5845 Signed-off-by: Jonathan Wrede <[email protected]> Signed-off-by: RutujaPathade <[email protected]>
…ev#6381) * fix(trino): Clean up temporary entity tables after retrieval TrinoOfflineStore.get_historical_features() creates a temporary table for the entity DataFrame but never drops it, leaking tables indefinitely. Apply the same context manager pattern used by BigQuery, Redshift, and Athena offline stores: wrap the query in a generator that issues DROP TABLE IF EXISTS in a finally block. Fixes feast-dev#6306 Signed-off-by: Jonathan Wrede <[email protected]> * fix: sort imports for ruff compliance Signed-off-by: Jonathan Wrede <[email protected]> * fix: decouple temp table cleanup from query access Avoid dropping the temporary entity table on to_sql() calls. Previously, every method used a context manager that dropped the table on exit, so calling to_sql() before to_df() would destroy the table and cause subsequent queries to fail. Now the query is stored as a plain string and cleanup is handled by a dedicated _drop_temp_table() method called only after query execution (to_df, to_trino). A __del__ fallback ensures cleanup if execution methods are never called. The _cleaned_up flag makes the drop idempotent. Signed-off-by: Jonathan Wrede <[email protected]> --------- Signed-off-by: Jonathan Wrede <[email protected]> Signed-off-by: RutujaPathade <[email protected]>
…6362) * feat(bigquery): Support DATE-type event timestamp columns When the event_timestamp column in BigQuery is a DATE type, the generated SQL wraps comparison values in TIMESTAMP(), causing a type mismatch error. This adds a timestamp_field_type parameter to BigQuerySource that, when set to "DATE", generates DATE() comparisons instead. Closes feast-dev#2530 (part 2) Signed-off-by: Jonathan Wrede <[email protected]> * fix(bigquery): Use protobuf 4.25.x compatible generated code The proto files were regenerated with protobuf 6.31.1 / grpcio-tools 1.80.0, which imports runtime_version -- a module that does not exist in protobuf 4.25.x used by the project. Revert generated code to 4.25.1 format while keeping the new timestamp_field_type field. Signed-off-by: Jonathan Wrede <[email protected]> * fix(bigquery): Add Literal type annotation for cast_style Mypy infers str from the ternary expression; annotate with the exact Literal union so the call to get_timestamp_filter_sql passes type checking. Signed-off-by: Jonathan Wrede <[email protected]> * fix: Make timestamp_field_type default to None in FeatureViewQueryContext Callers that do not use DATE-typed timestamp fields (e.g. Spark offline store tests) should not be forced to pass timestamp_field_type. Adding a default keeps the new field backward-compatible. Signed-off-by: Jonathan Wrede <[email protected]> * fix: Keep timestamp_field_type required in FeatureViewQueryContext A default value on timestamp_field_type breaks the SparkFeatureViewQueryContext subclass because its non-default fields (min_date_partition, max_date_partition) would follow a field with a default. Instead, keep it required and update the Spark test to pass it. Signed-off-by: Jonathan Wrede <[email protected]> * fix: regenerate protos matching upstream mypy-protobuf style Reset all non-DataSource generated files to match master. Only DataSource_pb2.py and DataSource_pb2.pyi contain our timestamp_field_type additions (field 28). The .pyi stub is hand-edited to match the existing import style used on master. Signed-off-by: Jonathan Wrede <[email protected]> --------- Signed-off-by: Jonathan Wrede <[email protected]> Signed-off-by: RutujaPathade <[email protected]>
Signed-off-by: ntkathole <[email protected]> Signed-off-by: RutujaPathade <[email protected]>
Mount the existing REST registry routers under /registry on the feature server so that fastapi_mcp automatically exposes registry introspection (list/get for entities, feature views, data sources, feature services, permissions, projects, saved datasets, lineage, search) as MCP tools. The RegistryServer is created in-process from store.registry — no external registry server is required. Auth is enforced via inject_user_details on every mounted router. Made-with: Cursor Signed-off-by: Chaitany patel <[email protected]> Made-with: Cursor Signed-off-by: RutujaPathade <[email protected]>
…elds Signed-off-by: RutujaPathade <[email protected]>
…chine to be opt-in Signed-off-by: RutujaPathade <[email protected]>
…egistry Signed-off-by: RutujaPathade <[email protected]>
Signed-off-by: RutujaPathade <[email protected]>
Signed-off-by: RutujaPathade <[email protected]>
Signed-off-by: RutujaPathade <[email protected]>
Co-authored-by: Nikhil Kathole <[email protected]> Signed-off-by: RutujaPathade <[email protected]>
Signed-off-by: RutujaPathade <[email protected]>
Signed-off-by: RutujaPathade <[email protected]>
Signed-off-by: RutujaPathade <[email protected]>
86e81b8 to
a4c85dc
Compare
…nabled-disabled-v2 Signed-off-by: RutujaPathade <[email protected]>
What this PR does / why we need it
Adds a lifecycle state machine and enabled/disabled toggle for feature views:
CREATED → GENERATED → MATERIALIZING → AVAILABLE_ONLINEtracks feature views from registration through materializationENABLED / DISABLEDtoggle controls serving and materializationstate == AVAILABLE_ONLINE AND enabled == TrueWhich issue(s) this PR fixes
Closes #6331
Checks
git commit -s)Testing Strategy