Skip to content

Commit

Permalink
chore(mlx_examples): fix wegiht typo in open_elm.py (#13)
Browse files Browse the repository at this point in the history
  • Loading branch information
jinyanzi committed May 2, 2024
2 parents f602cdc + 0c3d8f1 commit aaa14a6
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion mlx_examples/open_elm/open_elm.py
Original file line number Diff line number Diff line change
Expand Up @@ -631,7 +631,7 @@ def filter(layer: nn.Module, name: str, val: mx.array) -> bool:
for module in self.modules():
if isinstance(module, (nn.Linear)):
std = module.weight.shape[0] ** -0.5
module.apply(nn.init.normal(std=std), filter_fn=name_filter("wegiht"))
module.apply(nn.init.normal(std=std), filter_fn=name_filter("weight"))
module.apply(nn.init.constant(0.0), filter_fn=name_filter("bias"))
elif isinstance(module, (nn.Embedding)):
std = model_dim**-0.5
Expand Down

0 comments on commit aaa14a6

Please sign in to comment.