Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bugfix: convert several providers stop_reason to valid openai finish_reason #363

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

unsync
Copy link

@unsync unsync commented May 14, 2024

Description:
Currently, we forward the anthropic stop_reason, which is not compatible with the openai finish_reason.

This can cause warnings from some OpenAI sdk.

The conversion function returns stop as a default value in case new stop_reason are added.

There was also a bug in the groq and google providers for the finish_reason

Side note:
It would probably be beneficial to update the types to we can get compilation-time validation :

/**
* The structure of a completion response for the 'complete' function.
* @interface
*/
export interface CompletionResponse extends CResponse {
choices: {
text: string;
index: number;
logprobs: null;
finish_reason: string;
}[];
}
/**
* The structure of a choice in a chat completion response.
* @interface
*/
export interface ChatChoice {
index: number;
message: Message;
finish_reason: string;
logprobs?: object | null;
}

Proposed change :

export interface ChatChoice {
  index: number;
  message: Message;
  finish_reason: 'stop' | 'length' | 'function_call' | 'content_filter' | null;
  logprobs?: object | null;
}
export interface CompletionResponse extends CResponse {
  choices: {
    text: string;
    index: number;
    logprobs: null;
    finish_reason: 'stop' | 'length' | 'function_call' | 'content_filter' | null;
  }[];
}

see https://platform.openai.com/docs/guides/text-generation/chat-completions-response-format

@unsync unsync force-pushed the feat/improve-anthropic-stop-reason branch 2 times, most recently from 1e5a6ee to ae228b9 Compare May 16, 2024 15:16
@unsync unsync changed the title Fix: convert anthropic stop_reason to finish_reason bugfix: convert several providers stop_reason to valid openai finish_reason May 16, 2024
@unsync unsync force-pushed the feat/improve-anthropic-stop-reason branch 3 times, most recently from 351fff8 to ff93296 Compare May 27, 2024 08:22
@unsync unsync force-pushed the feat/improve-anthropic-stop-reason branch from ff93296 to b5a650e Compare June 5, 2024 14:33
@unsync
Copy link
Author

unsync commented Jun 6, 2024

hello @vrushankportkey @VisargD, i'm not sure if there is a process after opening a PR to request a review, any chance this PR could be considered ?

@vrushankportkey
Copy link
Collaborator

Thank you so much @unsync! Yes, we're checking the PR and will share review if any in some time! Really appreciate you pushing this bugfix 🙌

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants