You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In the Validation Prediction Insights, include the Zero-Epoch Response (i.e. how would the original foundation model response) as well as the BLEU (or other validation score) on the Zero-Epoch Response. This will show the user what the original foundation model would say vs their fine-tuned model as well as the validation performance improvement from fine-tuning.
Motivation
This will allow the user to quickly see the value of their fine-tuning exercise.
The text was updated successfully, but these errors were encountered:
This can be already solved by running a separate experiment and set epochs=0, then one can compare the outputs in separate windows, or by downloading the csvs and comparing outside.
One idea could be to save the output for each validation run and then display as separate columns in the insights, but it is already pretty hard to navigate there.
pascal-pfeiffer
changed the title
Compare Zero-Epoch Prediction with Fine-Tuned Prediction as well as Validaiton Score Comparison
Compare Zero-Epoch Prediction with Fine-Tuned Prediction as well as Validation Score Comparison
Jun 5, 2024
馃殌 Feature
In the Validation Prediction Insights, include the Zero-Epoch Response (i.e. how would the original foundation model response) as well as the BLEU (or other validation score) on the Zero-Epoch Response. This will show the user what the original foundation model would say vs their fine-tuned model as well as the validation performance improvement from fine-tuning.
Motivation
This will allow the user to quickly see the value of their fine-tuning exercise.
The text was updated successfully, but these errors were encountered: