Fix: accept empty arrays in struct field lookup#997
Merged
sungwy merged 3 commits intoapache:mainfrom Aug 6, 2024
Merged
Conversation
Fixes apache#992. Empty `pyarrow` arrays are considered falsy, which caused a `ResolveError` for required fields during scan operations.
Fokko
approved these changes
Aug 5, 2024
Collaborator
|
@grobgl Thank you for the fix. Could you add a integration test to guarantee the expected behavior? |
This covers the issue reported in apache#992 where empty scan queries yielded a `ResolveError`. Specifically, this occurred under the following conditions: - a table with an ordered, non-nullable string column - a scan filtering for a non-existing value _within_ the range of the values in that particular column
Collaborator
|
Thank you, @grobgl. Could you please double-check your test implementation? I removed the fix and the test didn't break. |
Contributor
Author
Removing the fix does break the test for me. Possibly, this depends on your |
Collaborator
|
That's strange because I'm running your branch on my machine, so we shouldn't be seeing different behavior. |
Collaborator
sungwy
approved these changes
Aug 6, 2024
sungwy
pushed a commit
that referenced
this pull request
Aug 9, 2024
* Fix: accept empty arrays in struct field lookup Fixes #992. Empty `pyarrow` arrays are considered falsy, which caused a `ResolveError` for required fields during scan operations. * Integration test: empty scan on non-nullable ordered string column This covers the issue reported in #992 where empty scan queries yielded a `ResolveError`. Specifically, this occurred under the following conditions: - a table with an ordered, non-nullable string column - a scan filtering for a non-existing value _within_ the range of the values in that particular column * Lint (add missing newline)
sungwy
pushed a commit
that referenced
this pull request
Aug 9, 2024
* Fix: accept empty arrays in struct field lookup Fixes #992. Empty `pyarrow` arrays are considered falsy, which caused a `ResolveError` for required fields during scan operations. * Integration test: empty scan on non-nullable ordered string column This covers the issue reported in #992 where empty scan queries yielded a `ResolveError`. Specifically, this occurred under the following conditions: - a table with an ordered, non-nullable string column - a scan filtering for a non-existing value _within_ the range of the values in that particular column * Lint (add missing newline)
sungwy
pushed a commit
to sungwy/iceberg-python
that referenced
this pull request
Dec 7, 2024
* Fix: accept empty arrays in struct field lookup Fixes apache#992. Empty `pyarrow` arrays are considered falsy, which caused a `ResolveError` for required fields during scan operations. * Integration test: empty scan on non-nullable ordered string column This covers the issue reported in apache#992 where empty scan queries yielded a `ResolveError`. Specifically, this occurred under the following conditions: - a table with an ordered, non-nullable string column - a scan filtering for a non-existing value _within_ the range of the values in that particular column * Lint (add missing newline)
sungwy
pushed a commit
to sungwy/iceberg-python
that referenced
this pull request
Dec 7, 2024
* Fix: accept empty arrays in struct field lookup Fixes apache#992. Empty `pyarrow` arrays are considered falsy, which caused a `ResolveError` for required fields during scan operations. * Integration test: empty scan on non-nullable ordered string column This covers the issue reported in apache#992 where empty scan queries yielded a `ResolveError`. Specifically, this occurred under the following conditions: - a table with an ordered, non-nullable string column - a scan filtering for a non-existing value _within_ the range of the values in that particular column * Lint (add missing newline)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #992.
Empty
pyarrowarrays are considered falsy, which caused aResolveErrorfor required fields during scan operations.