BUG2

[Phani1609]

🚀 The feature, motivation and pitch

I propose adding a feature to improve the visualization of loss curves in LLaMA-Recipes. Currently, the loss curves may exhibit spikes or irregularities, making it challenging to interpret training progress accurately. By enhancing the loss curve visualization, users can better analyze model training dynamics, identify potential issues, and make informed decisions for model optimization.

Motivation:
The motivation for this proposal stems from the need for clearer and more informative loss curve visualization in LLaMA-Recipes. Clearer visualization can aid researchers and practitioners in understanding model behavior, diagnosing training issues, and fine-tuning training strategies to improve model performance.

Pitch:
The proposed feature involves implementing smoother and more informative loss curve visualization techniques in LLaMA-Recipes. This could include methods such as moving average smoothing, adaptive smoothing algorithms, or interactive visualization tools that allow users to explore and analyze loss curves dynamically.

Alternatives

Manual post-processing of loss curve data to achieve smoother visualization.
Using external visualization libraries or tools to visualize loss curves outside of LLaMA-Recipes.

Additional context

No response

[mreso]

Thanks for the suggestions, we welcome contributions from the community.

[HamidShojanazeri]

@Phani1609 would it be something you are interested to add to W&B implementation here? https://github.com/meta-llama/llama-recipes/blob/main/src/llama_recipes/finetuning.py#L52