Add response_format support for openai and a way to set the functions #170

Arttii · 2024-02-23T16:07:47Z

Description

I needed support for the JSON output that OpenAI supports by setting the type. This is already supported by the underlying client.

Type of change

New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

No tests

Checklist:

I have read the Contributing documentation.
[ X] I have read the Code of conduct documentation.
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes
I have checked my code and corrected any misspellings

BabySid · 2024-04-10T08:00:56Z

hi, I do need this feature too. Is there a plan to merge this PR? thanks @henomis

henomis · 2024-04-10T08:15:24Z

hi, I do need this feature too. Is there a plan to merge this PR? thanks @henomis

@BabySid this PR needs some refactor.

BabySid · 2024-04-11T02:15:22Z

hi, I do need this feature too. Is there a plan to merge this PR? thanks @henomis

@BabySid this PR needs some refactor.

got. Thanks for the reply

Arttii · 2024-04-14T10:07:50Z

@henomis please let me know, what you think should be changed. Would be happy to incorporate it.

henomis · 2024-04-14T10:13:47Z

@henomis please let me know, what you think should be changed. Would be happy to incorporate it.

@Arttii just follow the review comments.

henomis · 2024-04-10T08:05:09Z

llm/openai/openai.go

+// WithFunctions sets the functions to use for the OpenAI instance.
+func (o *OpenAI) WithFunctions(functions map[string]Function) *OpenAI {
+	o.functions = functions
+	return o
+}
+


This is out of context of this PR.

Why is this out of scope?

I'm not 100% sure of this method and this change must be discussed.

henomis · 2024-04-10T08:06:51Z

llm/openai/openai.go

@@ -37,6 +37,7 @@ type OpenAI struct {
 	streamCallbackFn StreamCallback
 	toolChoice       *string
 	cache            *cache.Cache
+	responseFormat   openai.ChatCompletionResponseFormatType


Do not use go-openai type here.

Should we use a local alias for this or what is your preference?

Yes use an alias, see the comment below.

henomis · 2024-04-10T08:07:14Z

llm/openai/openai.go

 func (o *OpenAI) WithToolChoice(toolChoice *string) *OpenAI {
 	o.toolChoice = toolChoice
 	return o
 }

+func (o *OpenAI) WithResponseFormat(format openai.ChatCompletionResponseFormatType) *OpenAI {


Don't use go-openai types here

Lingoose's users must deal with lingoose types.

henomis · 2024-04-10T08:10:02Z

llm/openai/openai.go

+		temperature:    DefaultOpenAITemperature,
+		maxTokens:      DefaultOpenAIMaxTokens,
+		functions:      make(map[string]Function),
+		responseFormat: openai.ChatCompletionResponseFormatTypeText,


Don't use go-openai types here.

henomis · 2024-04-10T08:13:28Z

llm/openai/openai.go

@@ -299,6 +312,9 @@ func (o *OpenAI) buildChatCompletionRequest(t *thread.Thread) openai.ChatComplet
 		N:           DefaultOpenAINumResults,
 		TopP:        DefaultOpenAITopP,
 		Stop:        o.stop,
+		ResponseFormat: &openai.ChatCompletionResponseFormat{


ResponseFormat is omitempty in go-openai package. This means that you can use a default pointer to nil and eventually overwrite it using the internal value.

henomis · 2024-04-10T08:14:05Z

thread/thread.go

@@ -208,7 +208,7 @@ func (c *Content) Format(input types.M) *Content {
 	prompt := prompt.NewPromptTemplate(c.Data.(string))
 	err := prompt.Format(input)
 	if err != nil {
-		return c
+		panic(c)


Don't panic. Out of scope of this PR.

This is an interesting question as we should at least surface the error. This tripped me up a few times as the llm was just being prompted with an empty prompt essentially. Should we log or what is your preference?

henomis · 2024-04-14T10:14:45Z

go.sum

+github.com/sashabaranov/go-openai v1.21.0 h1:isAf3zPSD3VLd0pC2/2Q6ZyRK7jzPAaz+X3rjsviaYQ=
+github.com/sashabaranov/go-openai v1.21.0/go.mod h1:lj5b/K+zjTSFxVLijLSTDZuP7adOgerWeFyZLUhAKRg=


This is out of scope of this PR.

Fair enough, I will remove this.

henomis · 2024-04-14T10:15:01Z

llm/openai/common.go

+	GPT4Turbo             Model = openai.GPT4Turbo
+	GPT4Turbo20240409     Model = openai.GPT4Turbo20240409


Out of Scope

henomis · 2024-04-14T10:15:08Z

llm/openai/openai.go

@@ -324,7 +324,7 @@ func (o *OpenAI) getChatCompletionRequestTools() []openai.Tool {
 	for _, function := range o.functions {
 		tools = append(tools, openai.Tool{
 			Type: "function",
-			Function: openai.FunctionDefinition{
+			Function: &openai.FunctionDefinition{


Out of scope

henomis · 2024-04-15T07:08:08Z

llm/openai/openai.go

@@ -37,6 +37,7 @@ type OpenAI struct {
 	streamCallbackFn StreamCallback
 	toolChoice       *string
 	cache            *cache.Cache
+	responseFormat   openai.ChatCompletionResponseFormatType


Yes use an alias, see the comment below.

henomis · 2024-04-15T07:08:37Z

llm/openai/openai.go

 func (o *OpenAI) WithToolChoice(toolChoice *string) *OpenAI {
 	o.toolChoice = toolChoice
 	return o
 }

+func (o *OpenAI) WithResponseFormat(format openai.ChatCompletionResponseFormatType) *OpenAI {


Lingoose's users must deal with lingoose types.

henomis · 2024-04-15T07:10:54Z

llm/openai/openai.go

+// WithFunctions sets the functions to use for the OpenAI instance.
+func (o *OpenAI) WithFunctions(functions map[string]Function) *OpenAI {
+	o.functions = functions
+	return o
+}
+


I'm not 100% sure of this method and this change must be discussed.

BabySid · 2024-05-20T03:21:43Z

Hi, @henomis It seems that this PR has been on hold for a long time. Does the lingoose project have any plans to develop this feature? To be honest, this feature is still very important for openai. I also know that this PR is currently in a very awkward situation, and the author may not be able to continue development due to some unknown reasons. It would be a very commendable thing if the project can integrate this feature.

henomis · 2024-05-20T06:14:14Z

@BabySid
I'll open a new PR for this feature.

henomis · 2024-05-20T06:31:58Z

Implemented here: #199

BabySid · 2024-05-20T08:19:35Z

@BabySid I'll open a new PR for this feature.

Many thanks:)

Arttii added 2 commits February 23, 2024 17:04

add response_format

97f022b

add format panic

0046ce2

Merge branch 'main' into main

991aab9

bump opepnai, add new turbo model

cd328eb

henomis requested changes Apr 14, 2024

View reviewed changes

henomis requested changes Apr 15, 2024

View reviewed changes

henomis closed this May 20, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add response_format support for openai and a way to set the functions #170

Add response_format support for openai and a way to set the functions #170

Arttii commented Feb 23, 2024 •

edited

BabySid commented Apr 10, 2024

henomis commented Apr 10, 2024

BabySid commented Apr 11, 2024

Arttii commented Apr 14, 2024

henomis commented Apr 14, 2024

henomis Apr 10, 2024

Arttii Apr 14, 2024

henomis Apr 15, 2024

henomis Apr 10, 2024

Arttii Apr 14, 2024

henomis Apr 15, 2024

henomis Apr 10, 2024

henomis Apr 15, 2024

henomis Apr 10, 2024

henomis Apr 10, 2024

henomis Apr 10, 2024

Arttii Apr 14, 2024

henomis Apr 14, 2024

Arttii Apr 14, 2024

henomis Apr 14, 2024

henomis Apr 14, 2024

henomis Apr 15, 2024

henomis Apr 15, 2024

henomis Apr 15, 2024

BabySid commented May 20, 2024

henomis commented May 20, 2024

henomis commented May 20, 2024

BabySid commented May 20, 2024

		github.com/sashabaranov/go-openai v1.21.0 h1:isAf3zPSD3VLd0pC2/2Q6ZyRK7jzPAaz+X3rjsviaYQ=
		github.com/sashabaranov/go-openai v1.21.0/go.mod h1:lj5b/K+zjTSFxVLijLSTDZuP7adOgerWeFyZLUhAKRg=

		GPT4Turbo Model = openai.GPT4Turbo
		GPT4Turbo20240409 Model = openai.GPT4Turbo20240409

Add response_format support for openai and a way to set the functions #170

Add response_format support for openai and a way to set the functions #170

Conversation

Arttii commented Feb 23, 2024 • edited

Description

Type of change

How Has This Been Tested?

Checklist:

BabySid commented Apr 10, 2024

henomis commented Apr 10, 2024

BabySid commented Apr 11, 2024

Arttii commented Apr 14, 2024

henomis commented Apr 14, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BabySid commented May 20, 2024

henomis commented May 20, 2024

henomis commented May 20, 2024

BabySid commented May 20, 2024

Arttii commented Feb 23, 2024 •

edited