[Feature]: 🎉 GPT-4o Day 1 Support

[krrishdholakia]

UPDATE

Live now

On SDK

from litellm import completion
import os

## set ENV variables
os.environ["OPENAI_API_KEY"] = "your-api-key"

response = completion(
  model="gpt-4o",
  messages=[{ "content": "Hello, how are you?","role": "user"}]
)

On Proxy

1. Add to yaml

model_list:
   - model_name: gpt-4o
      litellm_params:
          model: gpt-4o
          api_key: os.environ/MY_OPENAI_KEY

2. Start proxy

litellm --config /path/to/config.yaml

3. Test it!

curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--data ' {
      "model": "gpt-4o",
      "messages": [
        {
          "role": "user",
          "content": "what llm are you"
        }
      ]
    }
'

See all ways to test on proxy: https://docs.litellm.ai/docs/proxy/quick_start#using-litellm-proxy---curl-request-openai-package-langchain

The Feature

Creating parent issue to track gpt-4o support.

It's not out yet - https://platform.openai.com/docs/models/gpt-4-turbo-and-gpt-4

But should be in the next few weeks

Motivation, pitch

support all models

Twitter / LinkedIn details

No response

[mmmaia]

It's not out yet - https://platform.openai.com/docs/models/gpt-4-turbo-and-gpt-4

https://platform.openai.com/docs/models/gpt-4o

[hi019]

Also accessible in playground: https://platform.openai.com/playground/chat?mode=chat&model=gpt-4o&models=gpt-4o

[emsi]

It's absolutely out.

Why on earth there's a need to add new gpt models to litellm each time openai adds them? Can't it just route gpt.* to openai and at least in principle make it future proof?

[mmmaia]

Why on earth there's a need to add new gpt models to litellm each time openai adds them? Can't it just route gpt.* to openai and at least in principle make it future proof?

There's also the models endpoint, that could be used as a source.
https://platform.openai.com/docs/api-reference/models

curl https://api.openai.com/v1/models \
  -H "Authorization: Bearer $OPENAI_API_KEY"

    {
      "id": "gpt-4o",
      "object": "model",
      "created": 1715367049,
      "owned_by": "system"
    },
    {
      "id": "gpt-4o-2024-05-13",
      "object": "model",
      "created": 1715368132,
      "owned_by": "system"
    }

[ventz]

@krrishdholakia - added it already: #3615

(I have it working with OpenAI directly)

[ventz]

@mmmaia The main reason for adding the models "by hand" is due to their pricing/context windows. You would think OpenAI would create a json spec somewhere with this information so that others could automate.
But completely agree with you -- it feels like something that should be avoidable.

[krrishdholakia]

This is now live - #3613

Should be available for everybody to use (no updates needed), in the next 5-10 minutes

once cache is updated - https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json

[krrishdholakia]

On SDK

from litellm import completion
import os

## set ENV variables
os.environ["OPENAI_API_KEY"] = "your-api-key"

response = completion(
  model="gpt-4o",
  messages=[{ "content": "Hello, how are you?","role": "user"}]
)

On Proxy

1. Add to yaml

model_list:
   - model_name: gpt-4o
      litellm_params:
          model: gpt-4o
          api_key: os.environ/MY_OPENAI_KEY

2. Start proxy

litellm --config /path/to/config.yaml

3. Test it!

curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--data ' {
      "model": "gpt-4o",
      "messages": [
        {
          "role": "user",
          "content": "what llm are you"
        }
      ]
    }
'

See all ways to test on proxy: https://docs.litellm.ai/docs/proxy/quick_start#using-litellm-proxy---curl-request-openai-package-langchain

[ishaan-jaff]

Why on earth there's a need to add new gpt models to litellm each time openai adds them?

You don't need to wait for it to be on litellm json map - @emsi you could have just done model=openai/gpt-4o

cc @ConnorDoyle @hi019 @chanzer0

[krrishdholakia]

It's absolutely out.

Why on earth there's a need to add new gpt models to litellm each time openai adds them? Can't it just route gpt.* to openai and at least in principle make it future proof?

Hey @emsi

you do not need to add models to litellm to use new models.

Just append the provider to the model name - e.g. openai/gpt-4o, and it'll work. The model map, just allows people to avoid needing to explicitly provide the mapping for popular models.