feat: Custom OpenAI compatible server support with preserved inference parameters

[qnixsynapse]

Problem
Sometimes, people like me run an openai compatible server instead of running the model through Jan's nitro for better tailored to hardware compatibility and availability. The openai inference partially works when set like this:

However, Inference parameters, such as top_k, top_p, etc are not available. And some parameters such as temperature are not preserved. The model name is also incorrect as shown in the screenshot:

Success Criteria
It would be better to have support for a custom openai server. The list of models available can be queried through v1/models endpoint(if available).

[Van-QA]

hi @qnixsynapse,
You can achieve the same result by modifying the model.json value, take OpenAI GPT 4 turbo for example:

{
  "sources": [
    {
      "url": "https://openai.com"
    }
  ],
  "id": "gpt-4-turbo",
  "object": "model",
  "name": "OpenAI GPT 4 Turbo",
  "version": "1.2",
  "description": "OpenAI GPT 4 Turbo model is extremely good",
  "format": "api",
  "settings": {},
  "parameters": {
    "max_tokens": 4096,
    "temperature": 0.7,
    "top_p": 0.95,
    "stream": true,
    "stop": [],
    "frequency_penalty": 0,
    "presence_penalty": 0
  },
  "metadata": {
    "author": "OpenAI",
    "tags": [
      "General"
    ]
  },
  "engine": "openai"
}

The value from the settings will be visible in the UI and will be applied to the request

Related article:
https://jan.ai/docs/remote-models/generic-openai