Skip to content

Code base for the paper "Instruction Tuned Models are Quick Learners".

License

Notifications You must be signed in to change notification settings

srsawant34/efficient_instruction_learning

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Instruction Tuned Models are Quick Learners

  • This repo releases the implementation for our experiments of the research paper: "Instruction Tuned Models are Quick Learners"
  • The experiments are run on Tk-instruct-3B-model, which was finetuned on data.

Requirements

The experiments and analysis are conducted on the following environment:

  • CUDA (11.1)
  • cuDNN (8.0)
  • Pytorch (1.10.0)
  • Transformers (4.18.0)
  • DeepSpeed

For cloning the environment and install the required python libraries run the following command:

pip install -r requirements.txt

Data

Our model are trained on the Super-NaturalInstructions English-only tasks on 119 test tasks. The data splits can be created by running the following python script.

python data_prep.py --num_examples 2 --ten True --onepercent True --hundred True --twohundred True --thousand True

Running the experiment

To run the experiment, run the following the command:

sh scripts/master.sh -t task.txt -s twohundred

The above command will finetune the model(s) and evaluate it.