ENH Layer/model status shows devices now #1743
Conversation
For each adapter, show all the devices of this adapter's parameters. Also, while working on this, found a very minor bug in VeRA as its linear layer didn't implement its own __repr__.
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
ping @sayakpaul maybe also interesting for diffusers |
There was a problem hiding this comment.
Just a single comment. I like this feature a lot!
The tests are very comprehensive.
Would you mind adding a note about this (once this is merged) to https://huggingface.co/docs/diffusers/main/en/tutorials/using_peft_for_inference?
| elif ( | ||
| isinstance(adapter_module, torch.nn.ParameterDict) | ||
| or (adapter_module.__class__.__name__ == "BufferDict") # VeRA | ||
| ): |
There was a problem hiding this comment.
Should this be covered for DoRA too?
There was a problem hiding this comment.
Yes, this works for DoRA too. As DoRA is just a LoraModel, the shown status will be almost the same for LoRA and DoRA, except that DoRA has a few more parameters.
There was a problem hiding this comment.
Thank you @BenjaminBossan for adding the features for showing model and layer adapter details and improving that to add the device details! Left a couple nits.
| TunerLayerStatus(name='text_model.encoder.layers.0.self_attn.v_proj', | ||
| module_type='lora.Linear', | ||
| enabled=True, | ||
| active_adapters=['adapter-2'], | ||
| merged_adapters=[], | ||
| requires_grad={'adapter-1': False, 'adapter-2': True}, | ||
| available_adapters=['adapter-1', 'adapter-2']), | ||
| devices={'adapter-1': ['cpu'], 'adapter-2': ['cuda]}), |
There was a problem hiding this comment.
| devices={'adapter-1': ['cpu'], 'adapter-2': ['cuda]}), | |
| devices={'adapter-1': ['cpu'], 'adapter-2': ['cuda']}, |
| @@ -244,5 +246,6 @@ TunerModelStatus( | |||
| merged_adapters=[], | |||
| requires_grad={'adapter-1': False, 'adapter-2': True}, | |||
| available_adapters=['adapter-1', 'adapter-2'], | |||
| devices={'adapter-1': ['cpu'], 'adapter-2': ['cuda], | |||
There was a problem hiding this comment.
| devices={'adapter-1': ['cpu'], 'adapter-2': ['cuda], | |
| devices={'adapter-1': ['cpu'], 'adapter-2': ['cuda']}, |
For each adapter, show all the devices of this adapter's parameters.
Also, while working on this, found a very minor bug in VeRA as its linear layer didn't implement its own
__repr__.