Cleanlab on regression problems?

[mengqvist]

Hi!

I was wondering whether there's a natural way to extend cleanlab to identify label noise on regression problems, as opposed to classification problems? I am aware of some other frameworks, such as Shapley values, but it would be great if I could use cleanlab directly. I could bin the data, but that's not particularly elegant.

Thanks!

[mengqvist]

Ah! After poking about I found a note that extension to regression problems is planned as a future feature. Is this actively worked on?

[mglowacki100]

@jwmueller Are you still working on regression for cleanlab?
Have you consider following idea:

1. Let's start with 'ordinal regression', e.g. we rank movies with one, two or three stars. It could be relabelled to 0, 0.5, 1 and treated as ordinary regression. Next most of regression predictions would be between 0 and 1; predictions above 1 could be capped to 1, analogously predictions below 0 could be treated as 0. Those predictions could be treated as pred_probs for 'DataLab.find_issues(...)`. Obviously, this could be generalized to any number of "stars".
2. If idea from point 1 works, then general case of regression could be transformed to it by:

• scaling to [0,1]
• bucketing
  with some nitty-gritty details like outliers treatment and optimal number of buckets

[jwmueller]

@mglowacki100 Support for regression datasets is already available in the package today, just not in the stable release yet: https://docs.cleanlab.ai/master/tutorials/regression.html

To use it, simply do: pip install git+https://github.com/cleanlab/cleanlab.git

You can read about the method here: Detecting Errors in Numerical Data via any Regression Model

[mglowacki100]

@jwmueller Thanks for prompt reply! Do you plan to extend DataLab for regression?

[jwmueller]

Yes eventually, but it's not too high on the current priority list.

If you'd like to help with that extension, we could certainly use assistance!

[mglowacki100]

@jwmueller I've created separate issue for DataLab for regression task with some questions/initial investigations #765.