[optim] add missing 'maximize' parameter to LBFGS, NAdam and RAdam optimizers #126642
Labels
actionable
module: loss
Problem is related to loss function
module: optimizer
Related to torch.optim
triaged
This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Comments
drisspg
added
module: loss
|
Yes, it is known that these do not support the flag, and the documentation for each exclude mentioning maximize for that reason. It is totally easily actionable to add maximize for RAdam and NAdam + tests, perhaps easier than adding the docs lol. I may just add this for funsies later this week but if someone wants to do this, go for it! LBFGS I've gotta check, but off the top of my head it should work as well, but the implementation is not immediately doable in my head. |
pytorchmergebot
pushed a commit
that referenced
this issue
May 27, 2024
Fixes #[126642](#126642) I reference the maximize in `Adam` and add `Radam's` maximize flag. If this pr is OK, I will add another pr for `Nadam`. Pull Request resolved: #126765 Approved by: https://github.com/janeyx99
titaiwangms
pushed a commit
to titaiwangms/pytorch
that referenced
this issue
May 28, 2024
Fixes #[126642](pytorch#126642) I reference the maximize in `Adam` and add `Radam's` maximize flag. If this pr is OK, I will add another pr for `Nadam`. Pull Request resolved: pytorch#126765 Approved by: https://github.com/janeyx99
Aidyn-A
pushed a commit
to tinglvv/pytorch
that referenced
this issue
May 30, 2024
Fixes #[126642](pytorch#126642) I reference the maximize in `Adam` and add `Radam's` maximize flag. If this pr is OK, I will add another pr for `Nadam`. Pull Request resolved: pytorch#126765 Approved by: https://github.com/janeyx99
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
馃殌 The feature, motivation and pitch
Currently,
LBFGS,NAdamandRAdamoptimizers do not have themaximizeparameter in their constructors, which means they cannot be used with some loss functions (like R2Score).Alternatives
In the case
maximizecan't be implemented due to a constraint of the optimizer, it would be nice to make it clear thatmaximizeis intentionally not supported. Some possible suggestions:Additional context
No response
cc @vincentqb @jbschlosser @albanD @janeyx99 @crcrpar
The text was updated successfully, but these errors were encountered: