-
Notifications
You must be signed in to change notification settings - Fork 8.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
examples: cache hf model when --model not provided #7353
Conversation
Also, wouldn't it be nice to have
On windows, the equivalent of Not sure what it is on macos. |
cache_directory = std::getenv("HOME") + std::string("/.cache/"); | ||
} | ||
#elif defined(__APPLE__) | ||
cache_directory = std::getenv("HOME") + std::string("/Library/Caches/"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i think on Mac os pretty much all apps also use ~/.cache
(so, same as linux)
Just my 2 cents
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't use macos very often so I'm not sure. Maybe @ggerganov knows more about this?
My assumption is that ~/.cache
widely used by applications ported from linux (maybe?), while ~/Library/Caches
is the conventional OS X way (according to this documentation).
In either way, having model cached under user's $HOME is already very nice.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see apps caches in both places on my machine - I guess either way is fine. Don't have an opinion
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's keep ~/Library/Caches
then (as it is referred in apple's documentation)
common/common.cpp
Outdated
cache_directory += "llama.cpp"; | ||
cache_directory += DIRECTORY_SEPARATOR; | ||
} | ||
const bool success = create_directory_with_parents(cache_directory); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe you forgot to remove this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My bad, it should be removed now :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should be good to merge after resolving: https://github.com/ggerganov/llama.cpp/pull/7353/files#r1607877007
if (!success) { | ||
throw std::runtime_error("failed to create cache directory: " + cache_directory); | ||
} | ||
params.model = cache_directory + string_split(params.hf_file, '/').back(); | ||
} | ||
} else if (!params.model_url.empty()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Quick question: Do you think we should also use cache dir under this condition params.model_url
? @ggerganov
For now only params.hf_repo
enables cache dir.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. I'd prefer to also support cache dir with --model-url
. But that's not very important for now.
#elif defined(__APPLE__) | ||
cache_directory = std::getenv("HOME") + std::string("/Library/Caches/"); | ||
#elif defined(_WIN32) | ||
cache_directory = std::getenv("APPDATA"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Btw, the cache directory on Windows is %LOCALAPPDATA%
(see platformdirs). %APPDATA%
will be synced synced between computers in an Active Directory domain ("Roaming").
* examples: cache hf model when --model not provided * examples: cache hf model when --model not provided * examples: cache hf model when --model not provided * examples: cache hf model when --model not provided * examples: cache hf model when --model not provided
* examples: cache hf model when --model not provided * examples: cache hf model when --model not provided * examples: cache hf model when --model not provided * examples: cache hf model when --model not provided * examples: cache hf model when --model not provided
Ref: #7252