How to work with local data

[pretidav]

I downloaded a dataset from hf. I want to load it locally, but it still tries to download it from hf and place it into the cache.
How can I use the local one I already downloaded?

Thank you.

[dmilcevski]

Here, the script first tries to load the data from HF, if it doesn't succeed, then it falls back to loading it from disk. All you need to do is to adapt your recipe like this:

dataset_mixer:
  /path/to/your/data/: 1.0