30 May 15:48

github-actions

9f189ac

v1.39.4 Latest

Latest

What's Changed

fix - UI submit chat on enter by @ishaan-jaff in #3916
Revert "Revert "fix: Log errors in Traceloop Integration (reverts previous revert)"" by @nirga in #3909

Full Changelog: v1.39.3...v1.39.4

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.39.4

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	120.0	135.98662418243552	6.404889633803229	0.0	1913	0	97.80563699996492	1663.1231360000243
Aggregated	Passed ✅	120.0	135.98662418243552	6.404889633803229	0.0	1913	0	97.80563699996492	1663.1231360000243

Contributors

nirga and ishaan-jaff

Assets 4

30 May 04:26

github-actions

v1.39.3

6f06315

v1.39.3

What's Changed

fix: Log errors in Traceloop Integration (reverts previous revert) by @nirga in #3846
Added support for Triton chat completion using trtlllm generate endpo… by @giritatavarty-8451 in #3895
Revert "Added support for Triton chat completion using trtlllm generate endpo…" by @ishaan-jaff in #3900
[Feat] Implement Logout Admin UI by @ishaan-jaff in #3901
Revert "fix: Log errors in Traceloop Integration (reverts previous revert)" by @krrishdholakia in #3908
feat(proxy_server.py): emit webhook event whenever customer spend is tracked by @krrishdholakia in #3906
fix(openai.py): only allow 'user' as optional param if openai model by @krrishdholakia in #3902
[Feat] UI update analytics tab to show human friendly usage vals by @ishaan-jaff in #3894
ui - fix latency analytics on completion_tokens by @ishaan-jaff in #3897
[Admin UI] Edit Internal Users by @ishaan-jaff in #3904
fix(proxy_server.py): fix end user object check when master key used by @krrishdholakia in #3910
[UI] Fix bug on Model analytics by @ishaan-jaff in #3913
feat - langfuse use key_alias as generation name on litellm proxy by @ishaan-jaff in #3911
fix pricing / price tracking for vertex_ai/claude-3-opus@20240229 by @ishaan-jaff in #3915

New Contributors

@giritatavarty-8451 made their first contribution in #3895

Full Changelog: v1.39.2...v1.39.3

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.39.3

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	110.0	133.96143579083153	6.347194412767075	0.0	1898	0	91.88108999995848	1459.6432470000025
Aggregated	Passed ✅	110.0	133.96143579083153	6.347194412767075	0.0	1898	0	91.88108999995848	1459.6432470000025

Contributors

nirga, krrishdholakia, and 2 other contributors

Assets 4

29 May 06:53

github-actions

v1.39.2

19551ef

v1.39.2

What's Changed

Update ollama.py for image handling by @rick-github in #2888
fix(anthropic.py): fix parallel streaming on anthropic.py by @krrishdholakia in #3883
feat(proxy_server.py): Time to first token Request-level breakdown by @krrishdholakia in #3886
[BETA-Feature] Add OpenAI v1/batches Support on LiteLLM SDK by @ishaan-jaff in #3882
feat - router add abatch_completion - N Models, M Messages by @ishaan-jaff in #3889
[Feat] LiteLLM Proxy Add POST /v1/files and GET /v1/files by @ishaan-jaff in #3888
[Feat] LiteLLM Proxy - Add support for POST /v1/batches , GET /v1/batches by @ishaan-jaff in #3885
feat(router.py): support fastest response batch completion call by @krrishdholakia in #3887

Full Changelog: v1.38.12...v1.39.2

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.39.2

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	72	83.46968387564114	6.529958043991633	0.0	1954	0	61.38368400002037	678.4462749999989
Aggregated	Passed ✅	72	83.46968387564114	6.529958043991633	0.0	1954	0	61.38368400002037	678.4462749999989

Contributors

rick-github, krrishdholakia, and ishaan-jaff

Assets 4

28 May 15:54

github-actions

v1.38.12

e6789a0

v1.38.12

What's Changed

feat(proxy_server.py): CRUD endpoints for controlling 'invite link' flow by @krrishdholakia in #3873
[Feat] Add, Test Email Alerts on Admin UI by @ishaan-jaff in #3874

Full Changelog: v1.38.11...v1.38.12

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.38.12

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	76	91.16258395147193	6.473952425752436	0.0	1937	0	62.406538999994154	1772.6057410000067
Aggregated	Passed ✅	76	91.16258395147193	6.473952425752436	0.0	1937	0	62.406538999994154	1772.6057410000067

Contributors

krrishdholakia and ishaan-jaff

Assets 4

28 May 03:25

github-actions

v1.38.11

4b0a8ff

v1.38.11

users#set-rate-limits

✨ NEW /Customer/update and /Customer/delete endpoints https://docs.litellm.ai/docs/proxy/users#set-rate-limits

📝 [Feat] Email alerting is now Free Tier: https://docs.litellm.ai/docs/proxy/email

🚀 [Feat] Show supported OpenAI params on LiteLLM UI model hub

✨ [Feat] Show Created at, Created by on Models Page

What's Changed

Clarifai-LiteLLM update docs by @mogith-pn in #3856
[Feat] Show supported OpenAI params on model hub by @ishaan-jaff in #3859
fix(parallel_request_limiter.py): fix user+team tpm/rpm limit check by @krrishdholakia in #3857
fix - Admin UI show activity by model_group by @ishaan-jaff in #3865
[Feat] Show Created at, Created by on Models Page by @ishaan-jaff in #3868
Improve validate-fallbacks method by @SujanShilakar in #3847
fix(model_dashboard.tsx): accurately show the input/output cost per token when custom pricing is set by @krrishdholakia in #3871
Admin UI - Public model hub by @krrishdholakia in #3869
[Feat] Rename /end/user/new -> /customer/new (maintain backwards compatibility) by @ishaan-jaff in #3870
[Feat] Make Email alerting Free Tier, but customizing emails enterprise by @ishaan-jaff in #3872

New Contributors

@SujanShilakar made their first contribution in #3847

Full Changelog: v1.38.10...v1.38.11

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.38.11

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	94	113.13091035154665	6.485092627447978	0.0	1940	0	80.4994959999874	735.4111310000064
Aggregated	Passed ✅	94	113.13091035154665	6.485092627447978	0.0	1940	0	80.4994959999874	735.4111310000064

Contributors

krrishdholakia, ishaan-jaff, and 2 other contributors

Assets 4

26 May 22:48

github-actions

v1.38.10

3be6a07

v1.38.10

What's Changed

[Feat] Model Hub by @ishaan-jaff in #3849

Full Changelog: v1.38.8...v1.38.10

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.38.10

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	130.0	152.41971991092666	6.452763997233594	0.0	1931	0	108.63601500000186	1150.9651800000142
Aggregated	Passed ✅	130.0	152.41971991092666	6.452763997233594	0.0	1931	0	108.63601500000186	1150.9651800000142

Contributors

ishaan-jaff

Assets 4

26 May 07:03

github-actions

v1.38.8-stable

bec13d4

v1.38.8-stable

Full Changelog: v1.38.8...v1.38.8-stable

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.38.8-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	73	85.31436445193742	6.640342227407584	0.0	1987	0	61.23339800001304	1299.6820050000224
Aggregated	Passed ✅	73	85.31436445193742	6.640342227407584	0.0	1987	0	61.23339800001304	1299.6820050000224

Assets 4

26 May 05:49

github-actions

v1.38.8

4e6ca3c

v1.38.8

What's Changed

feat(slack_alerting.py): enable provider-region based alerting by @krrishdholakia in #3844

Full Changelog: v1.38.7...v1.38.8

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.38.8

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	74	86.12069489644486	6.487708071155493	0.0	1941	0	62.97004400005335	733.9951239999891
Aggregated	Passed ✅	74	86.12069489644486	6.487708071155493	0.0	1941	0	62.97004400005335	733.9951239999891

Contributors

krrishdholakia

Assets 4

26 May 03:29

github-actions

v1.38.7-stable

1a63f4c

v1.38.7-stable

What's Changed

[Feat] - Admin UI - New Activity Tab by @ishaan-jaff in #3836
[Feat] Ui Enforce premium features on ui by @ishaan-jaff in #3840
fix(proxy_server.py): fix model check for /v1/models + /model/info endpoint when team has restricted access by @krrishdholakia in #3839
[Fix] Set budget_duration on /team/new and /team/update by @ishaan-jaff in #3842
[Feat] Reset Team Budgets on budget_reset_at by @ishaan-jaff in #3843
[Feature]: Attach litellm exception in error string by @ishaan-jaff in #3824
docs- email notifs by @ishaan-jaff in #3845

Full Changelog: v1.38.5...v1.38.7-stable

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.38.7-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	96	117.54187770999512	6.456232729004693	0.0	1931	0	80.74312700000519	802.6662359999932
Aggregated	Passed ✅	96	117.54187770999512	6.456232729004693	0.0	1931	0	80.74312700000519	802.6662359999932

Contributors

krrishdholakia and ishaan-jaff

Assets 4

26 May 01:44

github-actions

v1.38.7

1a63f4c

v1.38.7

😇 LiteLLM v1.38.7 - New Activity Tab, Track LLM API Requests & Total Tokens 👉 Start here: https://github.com/BerriAI/litellm

🔥 [Fix] Set budget_duration on /team/new and /team/update

🔥 [Feat] Supporting for Resetting Team Budgets on budget_reset_at https://docs.litellm.ai/docs/proxy/users

⚒️ [Feature]: Attach litellm exception in error string - ContentPolicyViolation, AuthenticationError

📧 [Docs]- setting up Email notifications https://docs.litellm.ai/docs/proxy/email

What's Changed

[Feat] - Admin UI - New Activity Tab by @ishaan-jaff in #3836
[Feat] Ui Enforce premium features on ui by @ishaan-jaff in #3840
fix(proxy_server.py): fix model check for /v1/models + /model/info endpoint when team has restricted access by @krrishdholakia in #3839
[Fix] Set budget_duration on /team/new and /team/update by @ishaan-jaff in #3842
[Feat] Reset Team Budgets on budget_reset_at by @ishaan-jaff in #3843
[Feature]: Attach litellm exception in error string by @ishaan-jaff in #3824
docs- email notifs by @ishaan-jaff in #3845

Full Changelog: v1.38.5...v1.38.7

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.38.7

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	110.0	127.76486134384693	6.465849619551454	0.0	1934	0	97.91651000000456	1353.8686059999918
Aggregated	Passed ✅	110.0	127.76486134384693	6.465849619551454	0.0	1934	0	97.91651000000456	1353.8686059999918

Contributors

krrishdholakia and ishaan-jaff

Assets 4

Releases: BerriAI/litellm

v1.39.4

What's Changed

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

Contributors

v1.39.3

What's Changed

New Contributors

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

Contributors

v1.39.2

What's Changed

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

Contributors

v1.38.12

What's Changed

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

Contributors

v1.38.11

💵 LiteLLM v1.38.11 Proxy 100+ LLMs AND Set Budgets for your customers https://docs.litellm.ai/docs/proxy/users#set-rate-limits

What's Changed

New Contributors

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

Contributors

v1.38.10

What's Changed

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

Contributors

v1.38.8-stable

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

v1.38.8

What's Changed

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

Contributors

v1.38.7-stable

What's Changed

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

Contributors

v1.38.7

😇 LiteLLM v1.38.7 - New Activity Tab, Track LLM API Requests & Total Tokens 👉 Start here: https://github.com/BerriAI/litellm

What's Changed

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

Contributors