-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use merge_lora_weights.py Error:AttributeError: 'TinyLlavaPhiConfig' object has no attribute 'attention_bias' #24
Comments
I have not encountered this problem, but I want to know why can you use lora to finetune the model? because I can't found the script. #23 |
Make sure your model name has ‘phi’ or ‘TinyLLaVA-3.1B’ in it. |
I provided an example with lora finetuning in #23 |
the error:
[2024-03-15 17:42:38,572] [INFO] [real_accelerator.py:110:get_accelerator] Setting ds_accelerator to cuda (auto detect) Loading LLaVA from base model... Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Traceback (most recent call last): File "/home/xmj/mycipan3/smartahc_algorithm/projects/TinyLLaVABench/scripts/merge_lora_weights.py", line 22, in <module> merge_lora(args) File "/home/xmj/mycipan3/smartahc_algorithm/projects/TinyLLaVABench/scripts/merge_lora_weights.py", line 8, in merge_lora tokenizer, model, image_processor, context_len = load_pretrained_model(args.model_path, args.model_base, model_name, device_map='cpu') File "/home/xmj/mycipan3/smartahc_algorithm/projects/TinyLLaVABench/tinyllava/model/builder.py", line 118, in load_pretrained_model model = TinyLlavaLlamaForCausalLM.from_pretrained(model_base, low_cpu_mem_usage=True, config=cfg_pretrained, File "/home/xmj/anaconda3/envs/tinyllava/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3594, in from_pretrained model = cls(config, *model_args, **model_kwargs) File "/home/xmj/mycipan3/smartahc_algorithm/projects/TinyLLaVABench/tinyllava/model/language_model/llava_llama.py", line 46, in __init__ self.model = TinyLlavaLlamaModel(config) File "/home/xmj/mycipan3/smartahc_algorithm/projects/TinyLLaVABench/tinyllava/model/language_model/llava_llama.py", line 38, in __init__ super(TinyLlavaLlamaModel, self).__init__(config) File "/home/xmj/mycipan3/smartahc_algorithm/projects/TinyLLaVABench/tinyllava/model/llava_arch.py", line 32, in __init__ super(LlavaMetaModel, self).__init__(config) File "/home/xmj/anaconda3/envs/tinyllava/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 958, in __init__ [LlamaDecoderLayer(config, layer_idx) for layer_idx in range(config.num_hidden_layers)] File "/home/xmj/anaconda3/envs/tinyllava/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 958, in <listcomp> [LlamaDecoderLayer(config, layer_idx) for layer_idx in range(config.num_hidden_layers)] File "/home/xmj/anaconda3/envs/tinyllava/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 758, in __init__ self.self_attn = LLAMA_ATTENTION_CLASSES[config._attn_implementation](config=config, layer_idx=layer_idx) File "/home/xmj/anaconda3/envs/tinyllava/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 315, in __init__ self.q_proj = nn.Linear(self.hidden_size, self.num_heads * self.head_dim, bias=config.attention_bias) File "/home/xmj/anaconda3/envs/tinyllava/lib/python3.10/site-packages/transformers/configuration_utils.py", line 265, in __getattribute__ return super().__getattribute__(key) AttributeError: 'TinyLlavaPhiConfig' object has no attribute 'attention_bias'
The text was updated successfully, but these errors were encountered: