ChatTogether and JsonOuputFunctionParser

[fabiogomez11c]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

The following python code:

from pydantic import BaseModel, Field
from langchain_together import ChatTogether
from langchain_core.prompts import PromptTemplate
from langchain_core.output_parsers.openai_functions import JsonOutputFunctionsParser
from langchain_community.utils.openai_functions import convert_pydantic_to_openai_function

model = ChatTogether(model="mistralai/Mixtral-8x7B-Instruct-v0.1", temperature=0.0)


class SQLQuery(BaseModel):
    query: str = Field(..., description='SQL query to answer the question')


query = "Create a sample SQL query to answer the question: What is the average age of users?"

prompt = PromptTemplate(
    template="Answer the question: {question}",
    input_variables=["question"]
)
parser = JsonOutputFunctionsParser()
openai_functions = [convert_pydantic_to_openai_function(SQLQuery)]

res = prompt | model.bind(functions=openai_functions) | parser
res.invoke({"question": query})
print(res)

Error Message and Stack Trace (if applicable)

Traceback (most recent call last):
  File "/opt/miniconda3/envs/cotsql/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/miniconda3/envs/cotsql/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/Users/fh/.cursor/extensions/ms-python.python-2023.22.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39, in <module>
    cli.main()
  File "/Users/fh/.cursor/extensions/ms-python.python-2023.22.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
    run()
  File "/Users/fh/.cursor/extensions/ms-python.python-2023.22.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "/Users/fh/.cursor/extensions/ms-python.python-2023.22.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/Users/fh/.cursor/extensions/ms-python.python-2023.22.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/Users/fh/.cursor/extensions/ms-python.python-2023.22.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "/Users/fh/code/sql_cot/together_poc.py", line 26, in <module>
    res.invoke({"question": query})
  File "/opt/miniconda3/envs/cotsql/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 2499, in invoke
    input = step.invoke(
  File "/opt/miniconda3/envs/cotsql/lib/python3.10/site-packages/langchain_core/output_parsers/base.py", line 169, in invoke
    return self._call_with_config(
  File "/opt/miniconda3/envs/cotsql/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 1626, in _call_with_config
    context.run(
  File "/opt/miniconda3/envs/cotsql/lib/python3.10/site-packages/langchain_core/runnables/config.py", line 347, in call_func_with_variable_args
    return func(input, **kwargs)  # type: ignore[call-arg]
  File "/opt/miniconda3/envs/cotsql/lib/python3.10/site-packages/langchain_core/output_parsers/base.py", line 170, in <lambda>
    lambda inner_input: self.parse_result(
  File "/opt/miniconda3/envs/cotsql/lib/python3.10/site-packages/langchain_core/output_parsers/openai_functions.py", line 78, in parse_result
    raise OutputParserException(f"Could not parse function call: {exc}")
langchain_core.exceptions.OutputParserException: Could not parse function call: 'function_call'

Description

I'm trying to use the JsonOutputFunctionParser and realized that this is very specific for the OpenAI response and not for the Together API response:

OpenAI message.additional_kwargs["function_call"]:
{'function_call': {'arguments': '{"query":"SELECT AVG(age) AS average_age FROM users"}', 'name': 'SQLQuery'}}

Together message.additional_kwargs["function_call"]:
None -> Error
And this is because from function calls we receive this:
{'tool_calls': [{'id': 'call_1x6sobhg8l5q95h7cozs28kq', 'function': {'arguments': '{"query":"SELECT AVG(age) FROM users"}', 'name': 'SQLQuery'}, 'type': 'function'}]}

So the JsonOutputFunctionParser is not getting the function key-value. Will it be good to have a different JsonOutputFunctionParser for Together? It is easy to parse from Together since the output is very similar to the OpenAI response, it will need to get the function key instead of function_call.

I can work on that if you want.

System Info

langchain==0.1.19
langchain-community==0.0.38
langchain-core==0.1.52
langchain-openai==0.1.6
langchain-text-splitters==0.0.1
langchain-together==0.1.1

Platform:
Mac

Python version:
Python 3.10.14