fix!: Fix type equality between arro3 and pyarrow#483
Merged
kylebarron merged 5 commits intomainfrom Feb 23, 2026
Merged
Conversation
eq for python __eq__ duner"
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#462 looked like it should've been a simple refactor for nicer code, but it actually created #482, because the derived equality checking from pyo3 doesn't use the Arrow PyCapsule extraction. So it would only check for equality with other pyo3 objects, not any Arrow-like object.
This PR reverts #462 and adds tests that we can assert equality for all types (except for the lazy readers) with PyArrow.
The other part of equality that was biting me was #451, since I incorrectly was taking in references, not owned types.
Unfortunately, part of this change is that we now export
DataTypewithnullable: trueinstead ofnullable: false, so we probably need to put this in a new minor release, not a bugfix release.Change list
eqfor python__eq__duner #462__eq__calls the left side first. So ideally anyone who wants to compare arro3 types to other types should put the arro3 types first. This works reflexively with pyarrow for most types, because pyarrow returnsNotImplemented, which then tells Python to check the other direction for equality. This doesn't work forRecordBatchbecause apparently PyArrow has some custom__eq__there.DataTypethrough__arrow_c_schema__withnullable: true, which was required to match equality semantics with pyarrow.Reverts #462, Closes #482