Skip to content

TrinoOfflineStore does not cleanup temporary tables #6306

@mattiamatrix

Description

@mattiamatrix

Expected Behavior

Temporary entity tables created during get_historical_features() should be cleaned up after the retrieval job completes, consistent with how BigQuery, Redshift, and Athena handle it.

Current Behavior

Each call to get_historical_features() with a pandas DataFrame entity_df creates a permanent table ({catalog}.{dataset}.feast_entity_df_<uuid>) that is never dropped, leaking tables in the Trino catalog indefinitely.

Steps to reproduce

  1. This is a known issue in TrinoOfflineStore https://github.com/feast-dev/feast/blob/master/sdk/python/feast/infra/offline_stores/contrib/trino_offline_store/trino.py#L486

Specifications

  • Version: 0.50+
  • Platform: Any
  • Subsystem: TrinoOfflineStore

Possible Solution

Refactor TrinoRetrievalJob to accept a context manager (query_generator) instead of a plain query string, following the pattern used by BigQuery/Redshift/Athena, and add a DROP TABLE IF EXISTS in the finally block.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions