mac m2 run air llm garage-bAInd/Platypus2-7B get error Input must be a file-like object opened in binary mode, or string

[wuxiongwei]

---

ValueError Traceback (most recent call last)
Cell In[23], line 2
1 import mlx.core as mx
----> 2 generation_output = model.generate(
3 mx.array(input_tokens['input_ids']),
4 max_new_tokens=3,
5 use_cache=True,
6 return_dict_in_generate=True)
8 print(generation_output)

File ~/opt/anaconda3/envs/air_llm_python_3_8/lib/python3.8/site-packages/airllm/airllm_llama_mlx.py:254, in AirLLMLlamaMlx.generate(self, x, temperature, max_new_tokens, **kwargs)
252 def generate(self, x, temperature=0, max_new_tokens=None, **kwargs):
253 tokens = []
--> 254 for token in self.model_generate(x, temperature=temperature):
255 tokens.append(token)
258 if len(tokens) >= max_new_tokens:

File ~/opt/anaconda3/envs/air_llm_python_3_8/lib/python3.8/site-packages/airllm/airllm_llama_mlx.py:281, in AirLLMLlamaMlx.model_generate(self, x, temperature, max_new_tokens)
278 mask = mask.astype(self.tok_embeddings.weight.dtype)
280 self.record_memory('before_loading_tok')
--> 281 update_weights = ModelPersister.get_model_persister().load_model(self.layer_names_dict['embed'], self.checkpoint_path)
283 self.record_memory('after_loading_tok')
284 self.tok_embeddings.update(update_weights['tok_embeddings'])
...
97 #available = psutil.virtual_memory().available / 1024 / 1024
98 #print(f"loaded {layer_name}, available mem: {available:.02f}")
100 layer_state_dict = map_torch_to_mlx(layer_state_dict)

ValueError: [load] Input must be a file-like object opened in binary mode, or string

[Verdagon]

I just had this same problem, and I think I have a fix. In mlx_model_persister.py, change:
mx.load(to_load_path)
to:
mx.load(str(to_load_path))

@wuxiongwei, can you help verify the fix works for you?

[8-momo-8]

@Verdagon , yes it works with converting it to string

mx.load(str(to_load_path))

[mustangs0786]

Help where i have to put above code.....

input_text = [
#'What is the capital of United States?',
'I like',
]

MAX_LENGTH = 128
input_tokens = model.tokenizer(input_text,
return_tensors="np",
return_attention_mask=False,
truncation=True,
max_length=MAX_LENGTH,
padding=False)

input_tokens

generation_output = model.generate(
mx.array(input_tokens['input_ids']),
max_new_tokens=3,
use_cache=True,
return_dict_in_generate=True)

print(generation_output)

[shahfasal]

@mustangs0786 same error for me where you able to figure it out?

[shahfasal]

@Verdagon where do i find this file mlx_model_persister.py?

[shahfasal]

Got it its under airllm/persist/