Skip to content

Feature Stores

zenml.feature_stores special

A feature store enables an offline and online serving of feature data.

Feature stores allow data teams to serve data via an offline store and an online low-latency store where data is kept in sync between the two. It also offers a centralized registry where features (and feature schemas) are stored for use within a team or wider organization.

As a data scientist working on training your model, your requirements for how you access your batch / 'offline' data will almost certainly be different from how you access that data as part of a real-time or online inference setting. Feast solves the problem of developing train-serve skew where those two sources of data diverge from each other.

base_feature_store

The base class for feature stores.

BaseFeatureStore (StackComponent, ABC)

Base class for all ZenML feature stores.

Source code in zenml/feature_stores/base_feature_store.py
class BaseFeatureStore(StackComponent, ABC):
    """Base class for all ZenML feature stores."""

    @property
    def config(self) -> BaseFeatureStoreConfig:
        """Returns the `BaseFeatureStoreConfig` config.

        Returns:
            The configuration.
        """
        return cast(BaseFeatureStoreConfig, self._config)

    @abstractmethod
    def get_historical_features(
        self,
        entity_df: Union[pd.DataFrame, str],
        features: List[str],
        full_feature_names: bool = False,
    ) -> pd.DataFrame:
        """Returns the historical features for training or batch scoring.

        Args:
            entity_df: The entity DataFrame or entity name.
            features: The features to retrieve.
            full_feature_names: Whether to return the full feature names.

        Returns:
            The historical features as a Pandas DataFrame.
        """

    @abstractmethod
    def get_online_features(
        self,
        entity_rows: List[Dict[str, Any]],
        features: List[str],
        full_feature_names: bool = False,
    ) -> Dict[str, Any]:
        """Returns the latest online feature data.

        Args:
            entity_rows: The entity rows to retrieve.
            features: The features to retrieve.
            full_feature_names: Whether to return the full feature names.

        Returns:
            The latest online feature data as a dictionary.
        """
config: BaseFeatureStoreConfig property readonly

Returns the BaseFeatureStoreConfig config.

Returns:

Type Description
BaseFeatureStoreConfig

The configuration.

get_historical_features(self, entity_df, features, full_feature_names=False)

Returns the historical features for training or batch scoring.

Parameters:

Name Type Description Default
entity_df Union[pandas.core.frame.DataFrame, str]

The entity DataFrame or entity name.

required
features List[str]

The features to retrieve.

required
full_feature_names bool

Whether to return the full feature names.

False

Returns:

Type Description
DataFrame

The historical features as a Pandas DataFrame.

Source code in zenml/feature_stores/base_feature_store.py
@abstractmethod
def get_historical_features(
    self,
    entity_df: Union[pd.DataFrame, str],
    features: List[str],
    full_feature_names: bool = False,
) -> pd.DataFrame:
    """Returns the historical features for training or batch scoring.

    Args:
        entity_df: The entity DataFrame or entity name.
        features: The features to retrieve.
        full_feature_names: Whether to return the full feature names.

    Returns:
        The historical features as a Pandas DataFrame.
    """
get_online_features(self, entity_rows, features, full_feature_names=False)

Returns the latest online feature data.

Parameters:

Name Type Description Default
entity_rows List[Dict[str, Any]]

The entity rows to retrieve.

required
features List[str]

The features to retrieve.

required
full_feature_names bool

Whether to return the full feature names.

False

Returns:

Type Description
Dict[str, Any]

The latest online feature data as a dictionary.

Source code in zenml/feature_stores/base_feature_store.py
@abstractmethod
def get_online_features(
    self,
    entity_rows: List[Dict[str, Any]],
    features: List[str],
    full_feature_names: bool = False,
) -> Dict[str, Any]:
    """Returns the latest online feature data.

    Args:
        entity_rows: The entity rows to retrieve.
        features: The features to retrieve.
        full_feature_names: Whether to return the full feature names.

    Returns:
        The latest online feature data as a dictionary.
    """

BaseFeatureStoreConfig (StackComponentConfig) pydantic-model

Base config for feature stores.

Source code in zenml/feature_stores/base_feature_store.py
class BaseFeatureStoreConfig(StackComponentConfig):
    """Base config for feature stores."""

BaseFeatureStoreFlavor (Flavor)

Base class for all ZenML feature store flavors.

Source code in zenml/feature_stores/base_feature_store.py
class BaseFeatureStoreFlavor(Flavor):
    """Base class for all ZenML feature store flavors."""

    @property
    def type(self) -> StackComponentType:
        """Returns the flavor type.

        Returns:
            The flavor type.
        """
        return StackComponentType.FEATURE_STORE

    @property
    def config_class(self) -> Type[BaseFeatureStoreConfig]:
        """Config class for this flavor.

        Returns:
            The config class.
        """
        return BaseFeatureStoreConfig

    @property
    @abstractmethod
    def implementation_class(self) -> Type[BaseFeatureStore]:
        """Implementation class.

        Returns:
            The implementation class.
        """
        return BaseFeatureStore
config_class: Type[zenml.feature_stores.base_feature_store.BaseFeatureStoreConfig] property readonly

Config class for this flavor.

Returns:

Type Description
Type[zenml.feature_stores.base_feature_store.BaseFeatureStoreConfig]

The config class.

implementation_class: Type[zenml.feature_stores.base_feature_store.BaseFeatureStore] property readonly

Implementation class.

Returns:

Type Description
Type[zenml.feature_stores.base_feature_store.BaseFeatureStore]

The implementation class.

type: StackComponentType property readonly

Returns the flavor type.

Returns:

Type Description
StackComponentType

The flavor type.