Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] Logging intermediate results in forecasting benchmarking #6440

Open
benHeid opened this issue May 17, 2024 · 1 comment
Open

[ENH] Logging intermediate results in forecasting benchmarking #6440

benHeid opened this issue May 17, 2024 · 1 comment
Labels
enhancement Adding new functionality module:metrics&benchmarking metrics and benchmarking modules

Comments

@benHeid
Copy link
Contributor

benHeid commented May 17, 2024

Is your feature request related to a problem? Please describe.
When applying long benchmarks with multiples of forecasters etc. It would be cool if intermediate results would be stored. This would have the following advantages:

  • In case of failures, not all benchmarks need to be executed again.
  • It would be possible to take a look into preliminary results and see how many benchmarks are already executed.
@benHeid benHeid added enhancement Adding new functionality module:metrics&benchmarking metrics and benchmarking modules labels May 17, 2024
@fkiraly
Copy link
Collaborator

fkiraly commented May 18, 2024

I think the first design by @ViktorKaz and myself, for classification, has that feature. It's fallen a bit out of step with the rest of the code base, but perhaps this might be a design inspiration.
(current benchmarking module)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Adding new functionality module:metrics&benchmarking metrics and benchmarking modules
Projects
None yet
Development

No branches or pull requests

2 participants