Inference on tiny-llama

[MrAnayDongre]

Need some help here! I have fine-trained tiny-llama on my personal books. How do perform inference on it?
I have used AutoTrain-Advanced for fine-tuning.
My training.csv has following columns: prompt, question, answer, and text. The text column is a concatenation of the prompt, question, and answer columns.

Command I used:

`
autotrain llm --train --project-name josh-ops --model Maykeye/TinyLLama-v0 --data-path /kaggle/input/llamav2 --use-peft --quantization int4 --lr 2e-4 --train-batch-size 15 --epochs 50 --trainer sft

`

[abhishekkrthakur]

code to do inference can be found in README.md of the output repo/folder (josh-ops)

[MrAnayDongre]

I got it, thanks.