Replies: 4 comments 3 replies
-
🤖 Hey @marklysze, great to see you diving into new queries with the CitationQueryEngine! How's everything going on your end? Based on the information provided in the LlamaIndex repository, it is indeed necessary to include a prompt template within the query text when running a query using the CitationQueryEngine. The CitationQueryEngine class uses two prompt templates: CITATION_QA_TEMPLATE and CITATION_REFINE_TEMPLATE. These templates are used to format the query and context for the response synthesizer. Here's the relevant code snippet from the response_synthesizer = response_synthesizer or get_response_synthesizer(
llm=llm,
service_context=index.service_context,
text_qa_template=citation_qa_template,
refine_template=citation_refine_template,
response_mode=response_mode,
use_async=use_async,
streaming=streaming,
) As you can see, the Regarding your issue with Google's new Gemma model, it's possible that the model has a different requirement or format for the prompt template. It would be helpful to check the documentation or any available resources for the Gemma model to understand its specific requirements. If you're still encountering issues, it would be helpful to provide the error message or any additional information about the crash when using the Gemma model's prompt template. This could help in identifying the root cause of the issue. You can find more details about the CitationQueryEngine and its methods in the LlamaIndex repository. Sources
|
Beta Was this translation helpful? Give feedback.
-
@dosu-bot, thanks for your response. I don't think it answers my question, specifically. Can you please give me a full example that:
|
Beta Was this translation helpful? Give feedback.
-
@dosu-bot, so you don't need to use the prompt template on a CitationQueryEngine query? |
Beta Was this translation helpful? Give feedback.
-
I'm afraid it's still not clear to me. Anyone else? |
Beta Was this translation helpful? Give feedback.
-
When I run a query using the CitationQueryEngine do I need to provide the prompting template within the query text?
Here's my code using the CitationQueryEngine and running .query:
I was updating some code of mine and when I tried using Google's new Gemma model it crashed when I tried using it's prompt template and worked fine without any template, putting in just the question.
For some reason, a few months ago it seemed to need the template in the query, but perhaps I'm mistaken.
Beta Was this translation helpful? Give feedback.
All reactions