Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FR] - add provider for yfinance-cache #6159

Open
ValueRaider opened this issue Mar 2, 2024 · 2 comments
Open

[FR] - add provider for yfinance-cache #6159

ValueRaider opened this issue Mar 2, 2024 · 2 comments

Comments

@ValueRaider
Copy link

I hope the benefit of persistently caching data is obvious - speed for user, less load on provider.

I've been working on a persistent caching wrapper for yfinance - yfinance-cache. Basic idea is be smart about what & when to fetch. Supports a subset of yfinance - price history, calendar, shares outstanding, info. Currently finishing off financials caching.

Still needs some polish, but now might be a good time to start thinking about integrating into OpenBB. API intended as a drop-in replacement of yfinance so should be easy. @deeleeramone I see you mostly handle the yfinance provider, this feature might interest you. I did experiment creating a provider, but then discovered providers hardcoded anyway.

@deeleeramone
Copy link
Contributor

deeleeramone commented Mar 4, 2024

@ValueRaider, thanks for reaching out! We basically run a wrapper around the download function and Ticker class that connects it to our common parameters and parsing requirements. There's always things to add yet, but a caching solution sounds fantastic - especially helpful for things like reference data and symbol mapping.

It looks like a parameter for "use_cache" could be added where it is available, and then redirect the Python function accordingly.

A couple of questions for you:

  • Does this also redirect the Cookie and Crumb caching that is performed by the yFinance library?
  • Is there a function for setting the cache location?
  • What is the format the cache is stored in?
  • Is the cache itself able to be used asynchronously?

@ValueRaider
Copy link
Author

ValueRaider commented Apr 16, 2024

Hey, somehow I was never notified, only noticed via a PIP update. My bad, I had muted the repo

Does this also redirect the Cookie and Crumb caching that is performed by the yFinance library?

No.

Is there a function for setting the cache location?

Unofficially yes, implemented for unit tests. I'd have to add safety checks in case new location has old/incompatible data.

What is the format the cache is stored in?

Almost-entirely Pickled Python objects, each with a small metadata dict. Prices are Pickled Pandas Dataframes.

ls ~/.cache/py-yfinance-cache/AAPL 
annuals.pkl   dividends.pkl       events.log     full-release-dates.pkl  history-1h.pkl   info.pkl                   listing_date.json
calendar.pkl  earnings_dates.pkl  fast_info.pkl  history-1d.pkl          history-1wk.pkl  interim-release-dates.pkl  quarterlys.pkl

Is the cache itself able to be used asynchronously?

Not a clue, I work synchronously. If this is fast then do you really need async?

Btw since creating this FR, new release adds financials caching.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants