-
Notifications
You must be signed in to change notification settings - Fork 699
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add new ChatMemory implementation to be used for stateful data extraction #1067
Conversation
Hi @mariofusco, thanks a lot! Will try to review it asap |
Do you have any news about this? I'm seeing that this pull request could be relevant also in the light of broader feature requests related to chat memories. More in general we may want to introduce some sort of minimal SPI to facilitate the pluggability of custom chat memory implementations and maybe rewrite the existing memories in terms of this SPI. I could sketch this idea in a different pull request if you're interested, or maybe did you already have something similar in mind? |
@mariofusco sorry, I did not have time to look at it yet, I will try to do it today |
Hi @mariofusco! If I understand correctly, this use case assumes interactive conversation with the user to collect all the needed details, right? But in the test there is no response from the model (with guidance what information should be provided) because it outputs a |
That's correct, this chat memory is designed to be used with extractors so it cannot provide any message for the user. This means that a second AI service needs to be used together with this in order to give some feedback to user. Here you can see an example of how I used a very similar strategy. In fact here for each state of the conversation I had to use both a |
Hi @mariofusco, one thing that comes to my mind is to provide an LLM with tools to save/validate customer details. Something like: @Tool
String saveUserName(String name, @ToolMemoryId String memoryId) {
Customer customer = getCustomer(memoryId);
customer.setName(name);
return customer.isValid() ? "successfully collected all customer details" : "some customer details are missing"; // maybe specify what exactly is missing
}
@Tool
String saveUserAge(int age, @ToolMemoryId String memoryId) {
Customer customer = getCustomer(memoryId);
customer.setAge(age);
return customer.isValid() ? "successfully collected all customer details" : "some customer details are missing"; // maybe specify what exactly is missing
} Then the same AI Service/LLM can be used to drive the conversation and collect details. Regarding this PR: |
I will give a try to what you suggested, even though it seems a bit more cumbersome since (if I understand correctly) it requires to configure a different tool for each field of the domain object to be retrieved from the chat. Regarding this PR it is ok for me to close it, but I still believe that there should be a way to implement and configure chat memories in a more fine grained way. In case you will make any progress on this or I can help with something please keep me updated. |
I would try a single tool per domain object with all the fields, it might work as well. Ideally all tool params should be optional, there was a PR, I need to finish it.
I completely agree, I am just not sure at this point if it should be a template/vars or something different (e.g. separate
OK! Thank you! |
This pull request implements the
ChatMemory
that I have discussed and proposed here. This should be a good fit to be used for stateful data extraction, so for instance the included test case produces the following output:In essence what this
ChatMemory
does is simply concatenating the values of variables sent by a user at each iteration, recreating at each step the user message from the original prompt template and those concatenated variables. In this way the user message sent to the LLM at the 3rd prompt of my example above will be something like:"Extract information about a customer from this text 'hi. my name is Mario Fusco. I'm 50'. The response must contain only the JSON with customer's data and without any other sentence. You must answer strictly in the following JSON format: {\n"firstName": (type: string),\n"lastName": (type: string),\n"age": (type: integer)"
In order to implement this feature I had to add to the
UserMessage
both the prompt template and the set of variables from which it has been created. I believe that carrying those information can be useful also beyond the specific needs of this pull request. In reality probably it would be an ever better design if theUserMessage
would know how to render itself and use thePromptTemplate
internally instead of having a text populated from the outside as it does now. I'm open to also implement this further improvement, but for now I just wanted to demonstrate the general idea with the smallest possible set of changes./cc @sebastienblanc