Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

include_callback_events=True with add_routes() not working with agentExecutor #540

Open
giraudmartin opened this issue Mar 15, 2024 · 2 comments
Open

include_callback_events=True with add_routes() not working with agentExecutor #540

giraudmartin opened this issue Mar 15, 2024 · 2 comments

Comments

@giraudmartin
Copy link

Hello 馃憢

I have trying to add a callback to my AgentExecutor with langserve but I got some errors. I have an already implemented route in my backend so I've added the parameter include_callback_events=True to this route :

class Input(BaseModel):
    input: str
    chat_history: List[Union[HumanMessage, AIMessage, FunctionMessage]]

class Output(BaseModel):
    output: Any


# Create chat model from Azure AI
chatModel = AzureChatOpenAI(
   openai_api_version="2023-07-01-preview",
   azure_deployment="gpt-35-turbo_1106",
)

# Create agent with create_openai_tools_agent
tools = [myTool]
agent = create_openai_tools_agent(chatModel, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools)

add_routes(
    app,
    agent_executor.with_types(input_type=Input, output_type=Output),
    path="/backend",
    include_callback_events=True
)

I have a React client that calls this route thanks to the RemoteRunnable import :

import { RemoteRunnable } from "langchain/runnables/remote";

const result: { output: string } = await remoteChain.invoke({
  input: question,
  chat_history: chatHistory
}, { callbacks: [langfuseHandler] });

And I have the following errors from langserve, (note that it happens also even without callback from client) :

Traceback (most recent call last):
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/uvicorn/protocols/http/h11_impl.py", line 412, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/middleware/cors.py", line 91, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/middleware/cors.py", line 146, in simple_response
    await self.app(scope, receive, send)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/routing.py", line 758, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/routing.py", line 778, in app
    await route.handle(scope, receive, send)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/routing.py", line 299, in handle
    await self.app(scope, receive, send)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/routing.py", line 79, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/starlette/routing.py", line 74, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 278, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/langserve/server.py", line 464, in invoke
    return await api_handler.invoke(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/langserve/api_handler.py", line 719, in invoke
    self._InvokeResponse(
  File "/Users/martin.giraud/Documents/MyProject/MyProjectGPT/translation-assistant/backend/.venv/lib/python3.12/site-packages/pydantic/v1/main.py", line 341, in __init__
    raise validation_error
pydantic.v1.error_wrappers.ValidationError: 27 validation errors for backendInvokeResponse
callback_events -> 15 -> __root__ -> serialized
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_chain_start' (type=value_error.const; given=on_agent_finish; permitted=('on_chain_start',))
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_chain_end' (type=value_error.const; given=on_agent_finish; permitted=('on_chain_end',))
callback_events -> 15 -> __root__ -> error
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_chain_error' (type=value_error.const; given=on_agent_finish; permitted=('on_chain_error',))
callback_events -> 15 -> __root__ -> serialized
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> messages
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_chat_model_start' (type=value_error.const; given=on_agent_finish; permitted=('on_chat_model_start',))
callback_events -> 15 -> __root__ -> serialized
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> prompts
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_llm_start' (type=value_error.const; given=on_agent_finish; permitted=('on_llm_start',))
callback_events -> 15 -> __root__ -> response
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_llm_end' (type=value_error.const; given=on_agent_finish; permitted=('on_llm_end',))
callback_events -> 15 -> __root__ -> serialized
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> input_str
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_tool_start' (type=value_error.const; given=on_agent_finish; permitted=('on_tool_start',))
callback_events -> 15 -> __root__ -> output
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_tool_end' (type=value_error.const; given=on_agent_finish; permitted=('on_tool_end',))
callback_events -> 15 -> __root__ -> error
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_tool_error' (type=value_error.const; given=on_agent_finish; permitted=('on_tool_error',))
callback_events -> 15 -> __root__ -> serialized
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> query
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_retriever_start' (type=value_error.const; given=on_agent_finish; permitted=('on_retriever_start',))
callback_events -> 15 -> __root__ -> documents
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_retriever_end' (type=value_error.const; given=on_agent_finish; permitted=('on_retriever_end',))
callback_events -> 15 -> __root__ -> error
  field required (type=value_error.missing)
callback_events -> 15 -> __root__ -> type
  unexpected value; permitted: 'on_retriever_error' (type=value_error.const; given=on_agent_finish; permitted=('on_retriever_error',))

It seems quite related to this ticket, but it doesn't have any fix : #283

Thanks a lot for your help !
@eyurtsev
Copy link
Collaborator

  1. Apologies don't use callbacks yet, callback code is still untested so I don't have confidence in it working.
  2. For now work-around if you need token counting is to use astream_events and count tokens client side with tik-token. I'll address this at some point, but token counting is not high priority right now

@giraudmartin
Copy link
Author

@eyurtsev Ok noted, thanks for your return 馃憤

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@eyurtsev @giraudmartin