Improve error handling for dataclass inheritance#13531
Merged
JukkaL merged 2 commits intopython:masterfrom Aug 27, 2022
Merged
Conversation
This pull request: 1. Fixes python#8334. Overriding a dataclass attribute with a method or property now results in an error message, not a crash. (Overriding an attribute with a non-attribute at runtime will result in either inconsistent behavior or an exception, so I think unconditionally disallowing this is fine.) 2. Makes mypy report an error if you try subclassing a frozen dataclass with a non-frozen one or vice versa. Attempting to do this subclassing at runtime will raise a TypeError.
Collaborator
Author
|
I'm not too enthused about the new error message for (1) -- it's a bit cryptic imo. I'm open to bikeshedding there. |
This comment has been minimized.
This comment has been minimized.
Contributor
|
According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉 |
JukkaL
approved these changes
Aug 27, 2022
Collaborator
JukkaL
left a comment
JukkaL
left a comment
There was a problem hiding this comment.
Thanks for the PR! Looks good. The error message is a little cryptic, but I couldn't quickly think of a clearly better one. We can update the error message later if somebody comes up with a better idea.
Michael0x2a
added a commit
to Michael0x2a/mypy
that referenced
this pull request
Aug 28, 2022
While working on python#13531, I noticed that DataclassTransformer's `collect_attributes` method was doing basically this: ```python all_attrs = [] known_attrs = set() for stmt in current_class: attr = convert_stmt_to_dataclass_attr(stmt) all_attrs.append(attr) known_attrs.add(attr.name) for info in current_class.mro[1:-1]: if info is not a dataclass: continue super_attrs = [] for attr in info.dataclass_attributes: # ...snip... if attr.name not in known_attrs: super_attrs.append(attr) known_attrs.add(attr.name) elif all_attrs: for other_attr in all_attrs: if other_attr.name == attr.name: all_attrs.remove(attr) super_attrs.append(attr) break all_attrs = super_attrs + all_attrs all_attrs.sort(key=lambda a: a.kw_only) validate all_attrs ``` Constantly searching through and removing items from `all_attrs`, then pre-pending the superclass attrs will result in worst-case quadratic behavior in the edge case where subtype is overriding every attribute defined in the supertype. This edge case is admittedly pretty unlikely to happen, but I wanted to clean up the code a bit by reversing the order in which we process everything so we naturally record attrs in the correct order. One quirk of the old implementation I found was that we do not sort the attrs list and move kw-only attrs to the end when none of the supertypes are dataclasses. I tried changing this logic so we unconditionally sort the list, but this actually broke a few of our tests for some reason. I didn't want to get too deep in the weeds, so opted to preserve this behavior.
JukkaL
pushed a commit
that referenced
this pull request
Aug 31, 2022
While working on #13531, I noticed that DataclassTransformer's `collect_attributes` method was doing basically this: ```python all_attrs = [] known_attrs = set() for stmt in current_class: attr = convert_stmt_to_dataclass_attr(stmt) all_attrs.append(attr) known_attrs.add(attr.name) for info in current_class.mro[1:-1]: if info is not a dataclass: continue super_attrs = [] for attr in info.dataclass_attributes: # ...snip... if attr.name not in known_attrs: super_attrs.append(attr) known_attrs.add(attr.name) elif all_attrs: for other_attr in all_attrs: if other_attr.name == attr.name: all_attrs.remove(attr) super_attrs.append(attr) break all_attrs = super_attrs + all_attrs all_attrs.sort(key=lambda a: a.kw_only) validate all_attrs ``` Constantly searching through and removing items from `all_attrs`, then pre-pending the superclass attrs will result in worst-case quadratic behavior in the edge case where subtype is overriding every attribute defined in the supertype. This edge case is admittedly pretty unlikely to happen, but I wanted to clean up the code a bit by reversing the order in which we process everything so we naturally record attrs in the correct order. One quirk of the old implementation I found was that we do not sort the attrs list and move kw-only attrs to the end when none of the supertypes are dataclasses. I tried changing this logic so we unconditionally sort the list, but this actually broke a few of our tests for some reason. I didn't want to get too deep in the weeds, so opted to preserve this behavior.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request:
Fixes Crash when overriding field as property/method on frozen derived class #8334. Overriding a dataclass attribute with a method or property now results in an error message, not a crash.
(Overriding an attribute with a non-attribute at runtime will result in either inconsistent behavior or an exception, so I think unconditionally disallowing this is fine.)
Makes mypy report an error if you try subclassing a frozen dataclass with a non-frozen one or vice versa. Attempting to do this subclassing at runtime will raise a TypeError.