Skip to content

fix: evaluate ManifestGroup file filters#664

Open
fallintoplace wants to merge 1 commit into
apache:mainfrom
fallintoplace:fix/manifest-group-filter-files-datafile-evaluator
Open

fix: evaluate ManifestGroup file filters#664
fallintoplace wants to merge 1 commit into
apache:mainfrom
fallintoplace:fix/manifest-group-filter-files-datafile-evaluator

Conversation

@fallintoplace
Copy link
Copy Markdown
Contributor

Summary

Fixes #663.

This implements ManifestGroup::FilterFiles() instead of accepting the filter as a silent no-op. The change adds a DataFile StructLike wrapper, binds and caches file-level evaluators per partition spec, and applies the evaluator to each manifest entry before returning it.

The reader projection now also includes file-filter referenced metadata columns when callers use Select(...), including widening nested partition references to the full partition struct so partition filters evaluate against the correct positional values.

Tests

  • uvx pre-commit run --files src/iceberg/manifest/manifest_group.cc src/iceberg/row/manifest_wrapper.cc src/iceberg/row/manifest_wrapper.h src/iceberg/test/manifest_group_test.cc
  • uvx cmake --build build --target manifest_test -j 8
  • ./build/src/iceberg/test/manifest_test --gtest_filter='ManifestGroupVersions/ManifestGroupTest.FilterFiles*'
  • ./build/src/iceberg/test/manifest_test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ManifestGroup::FilterFiles silently ignores file filters

1 participant