feat: Implemented Tiling Support for Time-Windowed Aggregations by ntkathole · Pull Request #5724 · feast-dev/feast

ntkathole · 2025-11-16T11:17:39Z

What this PR does / why we need it:

This PR implements tiling for computing time-windowed aggregations efficiently by pre-aggregating data into small, time-bucketed units (tiles) that can be reused across multiple queries. Instead of scanning all raw events for each window, it:

Pre-aggregates data into small time buckets (tiles)
Computes windowed aggregations by subtracting tiles: window_at_T = tile_at_T - tile_at_(T-window_size)
Stores intermediate representations (IRs) for holistic aggregations (avg, std, var) to enable correct merging
Integrates tiling into Spark and Ray compute engines for StreamFeatureView

Sawtooth Window Tiling

Tile Generation: Events bucketed into hop-sized intervals (e.g., 1-minute tiles)
Cumulative Aggregation: Each tile contains cumulative sum/count from start
Window Computation: Windowed aggregation = tile_at_T - tile_at_(T-window_size)
IR Handling: For holistic aggs, IRs subtracted separately, then final value recomputed

Enable Tiling in StreamFeatureView

sales_features = StreamFeatureView(
    name="sales_features",
    entities=[customer],
    source=kafka_source,
    aggregations=[
        Aggregation(
            column="amount",
            function="avg",
            time_window=timedelta(minutes=5),
        ),
        Aggregation(
            column="amount",
            function="sum",
            time_window=timedelta(minutes=5),
        ),
    ],
    timestamp_field="event_timestamp",
    enable_tiling=True,                      # Enable tiling optimization
    tiling_hop_size=timedelta(minutes=1),    # Generate tiles every 1 minute
)

Architecture

┌─────────────────────────────────────────────────────────────┐
│  StreamFeatureView (enable_tiling=True)                     │
└────────────────────────┬────────────────────────────────────┘
                         │
         ┌───────────────┴───────────────┐
         │                               │
    ┌────▼────┐                    ┌────▼────┐
    │  Spark  │                    │   Ray   │
    │  Node   │                    │  Node   │
    └────┬────┘                    └────┬────┘
         │                               │
         │  Convert to pandas            │
         ▼                               ▼
┌─────────────────────────────────────────────────────────────┐
│  Tiling Logic.                                                        │
│  ┌──────────────────────────────────────────────────────┐   │
│  │  orchestrator.apply_sawtooth_window_tiling()         │   │
│  │  - Group by entity + hop interval                    │   │
│  │  - Compute cumulative aggregations (tiles)           │   │
│  │  - Output: Cumulative tiles at each hop              │   │
│  └──────────────────────┬───────────────────────────────┘   │
│                         │                                    │
│  ┌──────────────────────▼───────────────────────────────┐   │
│  │  tile_subtraction.convert_cumulative_to_windowed()   │   │
│  │  - Subtract tiles: window = tile_T - tile_(T-W)      │   │
│  │  - Recompute holistic aggs from IRs                  │   │
│  │  - Output: Windowed aggregations                     │   │
│  └──────────────────────┬───────────────────────────────┘   │
│                         │                                    │
│  ┌──────────────────────▼───────────────────────────────┐   │
│  │  tile_subtraction.deduplicate_keep_latest()          │   │
│  │  - Keep latest window per entity                     │   │
│  └──────────────────────────────────────────────────────┘   │
└─────────────────────────┬───────────────────────────────────┘
                          │  Convert back to engine format
                          ▼
                  ┌───────────────┐
                  │ Online Store  │
                  │               │
                  └───────────────┘

Copilot

Pull Request Overview

This PR implements tiling support for efficient time-windowed aggregations in Feast's streaming feature views. The implementation uses a sawtooth window tiling algorithm with intermediate representations (IRs) to enable correct merging of holistic aggregations (avg, std, var) while providing performance benefits for streaming scenarios.

Key Changes:

Adds core tiling logic with IR-based aggregation in new infra/tiling/ module
Extends StreamFeatureView with enable_tiling and tiling_hop_size configuration options
Integrates tiling into Spark and Ray compute engines with pandas-based processing
Updates protobuf definitions and documentation

Reviewed Changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 23 comments.

Show a summary per file

File	Description
`sdk/python/feast/infra/tiling/base.py`	Defines IR metadata structures for algebraic and holistic aggregations
`sdk/python/feast/infra/tiling/orchestrator.py`	Implements cumulative tile generation using sawtooth window algorithm
`sdk/python/feast/infra/tiling/tile_subtraction.py`	Converts cumulative tiles to windowed aggregations via tile subtraction
`sdk/python/feast/infra/tiling/__init__.py`	Exports tiling module public API
`sdk/python/feast/stream_feature_view.py`	Adds tiling configuration to StreamFeatureView class
`protos/feast/core/StreamFeatureView.proto`	Adds protobuf fields for tiling configuration
`sdk/python/feast/protos/feast/core/StreamFeatureView_pb2.py`	Generated protobuf Python code for tiling fields
`sdk/python/feast/protos/feast/core/StreamFeatureView_pb2.pyi`	Generated protobuf type stubs for tiling fields
`sdk/python/feast/infra/compute_engines/spark/nodes.py`	Implements tiling execution path for Spark engine
`sdk/python/feast/infra/compute_engines/spark/feature_builder.py`	Passes tiling config to Spark aggregation nodes
`sdk/python/feast/infra/compute_engines/ray/nodes.py`	Implements tiling execution path for Ray engine
`sdk/python/feast/infra/compute_engines/ray/feature_builder.py`	Passes tiling config to Ray aggregation nodes
`sdk/python/feast/utils.py`	Extracts input columns from aggregations for feature views
`sdk/python/tests/unit/infra/compute_engines/spark/test_nodes.py`	Updates test to pass required spark_session parameter
`docs/getting-started/concepts/tiling.md`	Comprehensive documentation on tiling concepts and usage
`docs/getting-started/concepts/stream-feature-view.md`	Adds reference to tiling documentation
`docs/getting-started/concepts/README.md`	Adds tiling.md to concepts index

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

sdk/python/feast/infra/compute_engines/ray/nodes.py

sdk/python/feast/infra/tiling/orchestrator.py

sdk/python/feast/infra/tiling/tile_subtraction.py

sdk/python/feast/stream_feature_view.py

sdk/python/feast/infra/tiling/tile_subtraction.py

sdk/python/feast/infra/compute_engines/ray/nodes.py

sdk/python/feast/infra/tiling/orchestrator.py

sdk/python/feast/stream_feature_view.py

HaoXuAI · 2025-11-20T06:18:14Z

sdk/python/feast/aggregation/tiling/__init__.py

@@ -0,0 +1,27 @@
+"""
+Tiling for efficient time-windowed aggregations.


Really cool. Is there a way to merge with the Aggregation interface? Or something like
aggregation/

Aggregation -- Base aggregation interface

tiling/

yea, this make sense. Changed now to:

feast/ - aggregation/ - __init__.py (Aggregation class) - tiling/

Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>

franciscojavierarceo

Happy Thanksgiving! 🚀🚀🚀

…t-dev#5724)

…t-dev#5724) Signed-off-by: Jacob Weinhold <29459386+jfw-ppi@users.noreply.github.com>

# [0.58.0](v0.57.0...v0.58.0) (2025-12-16) ### Bug Fixes * Add java proto ([#5719](#5719)) ([fc3ea20](fc3ea20)) * Add possibility to force full features names for materialize ops ([#5728](#5728)) ([55c9c36](55c9c36)) * Fixed file registry cache sync ([09505d4](09505d4)) * Handle hyphon in sqlite project name ([#5575](#5575)) ([#5749](#5749)) ([b8346ff](b8346ff)) * Pinned substrait to fix protobuf issue ([d0ef4da](d0ef4da)) * Set TLS certificate annotation only on gRPC service ([#5715](#5715)) ([75d13db](75d13db)) * SQLite online store deletes tables from other projects in shared registry scenarios ([#5766](#5766)) ([fabce76](fabce76)) * Validate not existing entity join keys for preventing panic ([0b93559](0b93559)) ### Features * Add annotations for pod templates ([534e647](534e647)) * Add Pytorch template ([#5780](#5780)) ([6afd353](6afd353)) * Add support for extra options for stream source ([#5618](#5618)) ([18956c2](18956c2)) * Added matched_tag field search api results with fuzzy search capabilities ([#5769](#5769)) ([4a9ffae](4a9ffae)) * Added support for enabling metrics in Feast Operator ([#5317](#5317)) ([#5748](#5748)) ([a8498c2](a8498c2)) * Configure CacheTTLSecondscache,CacheMode for file-based registry in Feast Operator([#5708](#5708)) ([#5744](#5744)) ([f25f83b](f25f83b)) * Implemented Tiling Support for Time-Windowed Aggregations ([#5724](#5724)) ([7a99166](7a99166)) * Offline Store historical features retrieval based on datetime range for spark ([#5720](#5720)) ([27ec8ec](27ec8ec)) * Offline Store historical features retrieval based on datetime range in dask ([#5717](#5717)) ([a16582a](a16582a)) * Production ready feast operator with v1 apiversion ([#5771](#5771)) ([49359c6](49359c6)) * Support for Map value data type ([#5768](#5768)) ([#5772](#5772)) ([b99a8a9](b99a8a9))

# [0.58.0](feast-dev/feast@v0.57.0...v0.58.0) (2025-12-16) ### Bug Fixes * Add java proto ([feast-dev#5719](feast-dev#5719)) ([fc3ea20](feast-dev@fc3ea20)) * Add possibility to force full features names for materialize ops ([feast-dev#5728](feast-dev#5728)) ([55c9c36](feast-dev@55c9c36)) * Fixed file registry cache sync ([09505d4](feast-dev@09505d4)) * Handle hyphon in sqlite project name ([feast-dev#5575](feast-dev#5575)) ([feast-dev#5749](feast-dev#5749)) ([b8346ff](feast-dev@b8346ff)) * Pinned substrait to fix protobuf issue ([d0ef4da](feast-dev@d0ef4da)) * Set TLS certificate annotation only on gRPC service ([feast-dev#5715](feast-dev#5715)) ([75d13db](feast-dev@75d13db)) * SQLite online store deletes tables from other projects in shared registry scenarios ([feast-dev#5766](feast-dev#5766)) ([fabce76](feast-dev@fabce76)) * Validate not existing entity join keys for preventing panic ([0b93559](feast-dev@0b93559)) ### Features * Add annotations for pod templates ([534e647](feast-dev@534e647)) * Add Pytorch template ([feast-dev#5780](feast-dev#5780)) ([6afd353](feast-dev@6afd353)) * Add support for extra options for stream source ([feast-dev#5618](feast-dev#5618)) ([18956c2](feast-dev@18956c2)) * Added matched_tag field search api results with fuzzy search capabilities ([feast-dev#5769](feast-dev#5769)) ([4a9ffae](feast-dev@4a9ffae)) * Added support for enabling metrics in Feast Operator ([feast-dev#5317](feast-dev#5317)) ([feast-dev#5748](feast-dev#5748)) ([a8498c2](feast-dev@a8498c2)) * Configure CacheTTLSecondscache,CacheMode for file-based registry in Feast Operator([feast-dev#5708](feast-dev#5708)) ([feast-dev#5744](feast-dev#5744)) ([f25f83b](feast-dev@f25f83b)) * Implemented Tiling Support for Time-Windowed Aggregations ([feast-dev#5724](feast-dev#5724)) ([7a99166](feast-dev@7a99166)) * Offline Store historical features retrieval based on datetime range for spark ([feast-dev#5720](feast-dev#5720)) ([27ec8ec](feast-dev@27ec8ec)) * Offline Store historical features retrieval based on datetime range in dask ([feast-dev#5717](feast-dev#5717)) ([a16582a](feast-dev@a16582a)) * Production ready feast operator with v1 apiversion ([feast-dev#5771](feast-dev#5771)) ([49359c6](feast-dev@49359c6)) * Support for Map value data type ([feast-dev#5768](feast-dev#5768)) ([feast-dev#5772](feast-dev#5772)) ([b99a8a9](feast-dev@b99a8a9)) Signed-off-by: Anthonette Adanyin <106275232+antznette1@users.noreply.github.com>

# [0.58.0](v0.57.0...v0.58.0) (2025-12-16) ### Bug Fixes * Add java proto ([#5719](#5719)) ([fc3ea20](fc3ea20)) * Add possibility to force full features names for materialize ops ([#5728](#5728)) ([55c9c36](55c9c36)) * Fixed file registry cache sync ([09505d4](09505d4)) * Handle hyphon in sqlite project name ([#5575](#5575)) ([#5749](#5749)) ([b8346ff](b8346ff)) * Pinned substrait to fix protobuf issue ([d0ef4da](d0ef4da)) * Set TLS certificate annotation only on gRPC service ([#5715](#5715)) ([75d13db](75d13db)) * SQLite online store deletes tables from other projects in shared registry scenarios ([#5766](#5766)) ([fabce76](fabce76)) * Validate not existing entity join keys for preventing panic ([0b93559](0b93559)) ### Features * Add annotations for pod templates ([534e647](534e647)) * Add Pytorch template ([#5780](#5780)) ([6afd353](6afd353)) * Add support for extra options for stream source ([#5618](#5618)) ([18956c2](18956c2)) * Added matched_tag field search api results with fuzzy search capabilities ([#5769](#5769)) ([4a9ffae](4a9ffae)) * Added support for enabling metrics in Feast Operator ([#5317](#5317)) ([#5748](#5748)) ([a8498c2](a8498c2)) * Configure CacheTTLSecondscache,CacheMode for file-based registry in Feast Operator([#5708](#5708)) ([#5744](#5744)) ([f25f83b](f25f83b)) * Implemented Tiling Support for Time-Windowed Aggregations ([#5724](#5724)) ([7a99166](7a99166)) * Offline Store historical features retrieval based on datetime range for spark ([#5720](#5720)) ([27ec8ec](27ec8ec)) * Offline Store historical features retrieval based on datetime range in dask ([#5717](#5717)) ([a16582a](a16582a)) * Production ready feast operator with v1 apiversion ([#5771](#5771)) ([49359c6](49359c6)) * Support for Map value data type ([#5768](#5768)) ([#5772](#5772)) ([b99a8a9](b99a8a9)) Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

ntkathole requested a review from a team as a code owner November 16, 2025 11:17

ntkathole force-pushed the aggregation_tiling branch from 6b05deb to 570debe Compare November 16, 2025 11:33

franciscojavierarceo requested a review from Copilot November 17, 2025 15:34

Copilot started reviewing on behalf of franciscojavierarceo November 17, 2025 15:34 View session

Copilot finished reviewing on behalf of franciscojavierarceo November 17, 2025 15:37

ntkathole force-pushed the aggregation_tiling branch from 570debe to dd83c13 Compare November 17, 2025 15:41

Copilot AI reviewed Nov 17, 2025

View reviewed changes

HaoXuAI reviewed Nov 20, 2025

View reviewed changes

ntkathole force-pushed the aggregation_tiling branch 2 times, most recently from 7189d22 to 839c7a4 Compare November 21, 2025 15:51

ntkathole added 2 commits November 21, 2025 21:24

feat: Implemented tiling support

0df22f3

Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>

chore: Created Aggregation interface

712d5f9

Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>

ntkathole force-pushed the aggregation_tiling branch from 839c7a4 to 712d5f9 Compare November 21, 2025 15:55

ntkathole added the ok-to-test label Nov 23, 2025

ntkathole requested review from HaoXuAI and franciscojavierarceo November 24, 2025 12:53

franciscojavierarceo approved these changes Nov 28, 2025

View reviewed changes

Merge branch 'master' into aggregation_tiling

49c2792

franciscojavierarceo merged commit 7a99166 into feast-dev:master Nov 28, 2025
19 checks passed

jfw-ppi pushed a commit to jfw-ppi/feast that referenced this pull request Nov 30, 2025

feat: Implemented Tiling Support for Time-Windowed Aggregations (feas…

c980db9

…t-dev#5724)

jfw-ppi pushed a commit to jfw-ppi/feast that referenced this pull request Nov 30, 2025

feat: Implemented Tiling Support for Time-Windowed Aggregations (feas…

04dc34f

…t-dev#5724) Signed-off-by: Jacob Weinhold <29459386+jfw-ppi@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Implemented Tiling Support for Time-Windowed Aggregations#5724

feat: Implemented Tiling Support for Time-Windowed Aggregations#5724
franciscojavierarceo merged 3 commits intofeast-dev:masterfrom
ntkathole:aggregation_tiling

ntkathole commented Nov 16, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

HaoXuAI Nov 20, 2025

Uh oh!

ntkathole Nov 21, 2025

Uh oh!

franciscojavierarceo left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		@@ -0,0 +1,27 @@
		"""
		Tiling for efficient time-windowed aggregations.

Conversation

ntkathole commented Nov 16, 2025

What this PR does / why we need it:

Sawtooth Window Tiling

Enable Tiling in StreamFeatureView

Architecture

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

HaoXuAI Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

ntkathole Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

franciscojavierarceo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants