-
Notifications
You must be signed in to change notification settings - Fork 28k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-47429][SQL] Rename error class to condition #46543
base: master
Are you sure you want to change the base?
[SPARK-47429][SQL] Rename error class to condition #46543
Conversation
since we stripped all the JSON from it in an earlier PR
This reverts commit 3ccde06.
@MaxGekk @HyukjinKwon @cloud-fan - Before I finish up this PR, I'd like to check in with you briefly to make sure you like where this is going. The main idea in this PR is to deprecate public classes and parameters that use the old "error class" terminology and instead direct users to use "error condition". I'm trying wherever possible to do this in a backwards-compatible way. Here's an example of the approach. In addition to being a bit gentler on users migrating to Spark 4.0, this will also help us avoid having to rename every instance of the Does this look good to you so far? In a similar vein, shall I keep incorrectly-named classes around but turn them into deprecated sub-classes of their correctly-named replacements? For example, |
What changes were proposed in this pull request?
On the Scala/Java side:
ErrorClassesJsonReader
toErrorConditionsJsonReader
.subClass
tosubCondition
inerror-conditions.json
and also in some private classes likeErrorInfo
.On the Python side:
error_classes.py
toerror_conditions.py
.ErrorClassesReader
toErrorConditionsReader
.PySparkException
andPySparkAssertionError
, add anerror_condition
parameter to the interface and deprecate theerror_class
parameter. (It would have been nice to be able to use@warnings.deprecated()
for this, but that's not available until Python 3.13.)TODO:
sub_condition
in Python butsubCondition
in Scala.ErrorConditionsJsonReader
methods likegetErrorMessage
,isValidErrorClass
, etc.SparkThrowable
SparkThrowableHelper
SparkException
SparkThrowable
sub-classes likeSparkRuntimeException
,SparkUpgradeException
, etc.KafkaIllegalStateException
Why are the changes needed?
This brings the code and interfaces in line with the updated terminology agreed on in SPARK-46810 and described in the errors README.
Deprecating some of the
errorClass
/error_class
parameters instead of renaming them in place is mainly so that we don't have to execute thousands of renames across the codebase all at once, and can instead do this incrementally.Does this PR introduce any user-facing change?
Yes, some evolving and developer APIs are changing.
The public interfaces for Spark's error reporting are also changing, though in a backwards-compatible manner.
How was this patch tested?
CI.
Was this patch authored or co-authored using generative AI tooling?
No.