You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
During model optimization im getting the following warning:
scipy_minimizeterminated with status 3, displaying original message fromscipy.optimize.minimize`: ABNORMAL_TERMINATION_IN_LNSRCH. And recently, it tried several times after raising the error.
The traceback of the error falls here:
[288] mll = mll_class(likelihood=m.likelihood, model=m, **mll_options)
--> [289] fit_gpytorch_mll(mll)
[290] else:
[291] raise NotImplementedError(
[292] f"Model of type {m.__class__.__name__} is currently not supported."
[293]
[102] if optimizer is not None: # defer to per-method defaults
[103] kwargs["optimizer"] = optimizer
--> [105]return FitGPyTorchMLL(
[106] mll,
[107] type(mll.likelihood),
[108] type(mll.model),
[109] closure=closure,
[110] closure_kwargs=closure_kwargs,
[111] optimizer_kwargs=optimizer_kwargs,
[112] **kwargs,
[113]
File in Dispatcher.__call__(self, *args, **kwargs)
[91] func = self.__getitem__(types=types)
[92] try:
---> [93] return func(*args, **kwargs)
[94] except MDNotImplementedError:
[95] # Traverses registered methods in order, yields whenever a match is found
[96] funcs = self.dispatch_iter(*types)
File in _fit_fallback(mll, _, __, closure, optimizer, closure_kwargs, optimizer_kwargs, max_attempts, warning_handler, caught_exception_types, **ignore)
[280]if debug.off():
[281] msg = msg + " For more information, try enabling botorch.settings.debug mode."
--> [283]raise ModelFittingError(msg)
ModelFittingError: All attempts to fit the model have failed. For more information, try enabling botorch.settings.debug mode.
After some reading, I couldn't figure out why the error occurred. Suggestions online say it is most likely numerical stability + model fitting the data poorly. S
Even then I tried to see how to force extra retries and here is where I noticed that even that fit_gpytorch_mll allows for a max_attempts argument. There is no way to include it through Ax. So I'm playing with different choices of mll to see if it avoids this issue.
My question would be, if that error has come in the past, what suggestions do you have to go around it? It is worth noting that this only occurred after some rounds in the loop; the first rounds, things went smoothly. And I would really dislike changing model choices mid-optimization.
Thanks for your time, and please lmk if you would like a notebook to replicate the error :)
The text was updated successfully, but these errors were encountered:
@Jgmedina95 thanks for posting! Let me find someone internally that can answer your question and get back to you. In the mean time can you provide a bit more information as to when this is happening, e.g., on what kind of problem is this occurring? Please provide sufficient detail so we can recreate the problem locally if possible.
During model optimization im getting the following warning:
scipy_minimize
terminated with status 3, displaying original message from
scipy.optimize.minimize`: ABNORMAL_TERMINATION_IN_LNSRCH. And recently, it tried several times after raising the error.The traceback of the error falls here:
After some reading, I couldn't figure out why the error occurred. Suggestions online say it is most likely numerical stability + model fitting the data poorly. S
Even then I tried to see how to force extra retries and here is where I noticed that even that
fit_gpytorch_mll
allows for amax_attempts
argument. There is no way to include it through Ax. So I'm playing with different choices of mll to see if it avoids this issue.My question would be, if that error has come in the past, what suggestions do you have to go around it? It is worth noting that this only occurred after some rounds in the loop; the first rounds, things went smoothly. And I would really dislike changing model choices mid-optimization.
Thanks for your time, and please lmk if you would like a notebook to replicate the error :)
The text was updated successfully, but these errors were encountered: