VectorQueryEngine issue with loading artifacts from query

[qwe321389yfs8t7huqd89q]

• I have read and agree to the contributing guidelines.

Describe the bug
Using this example and Redis I'm getting error:
Response: error querying vector store: string indices must be integers, not 'str'

[05/14/24 13:22:38] INFO     Subtask 14c519c5ea9747a381ef8aa97f4aa055
                             Response: error querying vector store: string indices must be integers, not 'str'
[05/14/24 13:22:53] INFO     Subtask f787fb764f1e4e9ea70d684662820fb3
                             Thought: The error message indicates a persistent issue with the JSON structure, and my previous
                             corrections have not resolved it. I need to re-evaluate the JSON object I am submitting to ensure it
                             adheres to the schema. I will carefully review the JSON schema for the griptape_tool action and make
                             sure that the structure of my action request is correct.
                             Actions: [{"tag": "griptape_info_search_corrected", "name": "griptape_tool", "path": "search",
                             "input": {"values": {"query": "What is Griptape?"}}}]

I don't have any other Vector DB so not sure if same thing happens with other drivers, looking at source code it should behave same as for Redis.

Tracked it down to this line:

griptape/griptape/engines/query/vector_query_engine.py

Line 36 in a899f5a

for artifact in [BaseArtifact.from_json(r.meta["artifact"]) for r in result if r.meta]

It assumes that meta is dict, but in fact query returns string from DB.

Possible solutions

1. Fix it in one place, in VectorQueryEngine (only a quick fix to make example code work with Redis):

[...]
import json
[...]
artifacts = [
            artifact
            for artifact in [BaseArtifact.from_json((r.meta["artifact"] if isinstance(r.meta, dict) else json.loads(r.meta)["artifact"])) for r in result if r.meta]
            if isinstance(artifact, TextArtifact)
        ]
[...]

2. Modify each Vector Driver so that query returns meta as dict.
3. Modify VectorQueryEngine to use load_entry method from Vector Driver before loading artifact.

I think best would be second option, as it won't affect LocalVectorStoreDriver.

Desktop (please complete the following information):

• OS: Win 11
• Griptape Version 0.25.0

[dylanholmes]

Hi @qwe321389yfs8t7huqd89q, thanks for surfacing this issue!

I've reproduced this using the following steps:

1. Started a local redis server with the redis search module:
   docker run -p 6379:6379 redislabs/redisearch:latest
2. Created an index.
   redis-cli FT.CREATE idx:griptape ON hash PREFIX 1 "griptape:" SCHEMA tag TAG vector VECTOR FLAT 6 TYPE FLOAT32 DIM 1536 DISTANCE_METRIC COSINE
3. Ran the example, replacing LocalVectorStoreDriver with RedisVectorStoreDriver.

Full Program:

from dotenv import load_dotenv
from griptape.structures import Agent
from griptape.tools import VectorStoreClient, TaskMemoryClient
from griptape.loaders import WebLoader
from griptape.engines import VectorQueryEngine
from griptape.drivers import RedisVectorStoreDriver, OpenAiEmbeddingDriver, OpenAiChatPromptDriver


load_dotenv()

engine = VectorQueryEngine(
    prompt_driver=OpenAiChatPromptDriver(model="gpt-3.5-turbo"),
    vector_store_driver=RedisVectorStoreDriver(
        host="localhost",
        port=6379,
        password="",
        index="idx:griptape",
        embedding_driver=OpenAiEmbeddingDriver(),
    ),
)

engine.upsert_text_artifacts(
    WebLoader().load("https://www.griptape.ai"),
    namespace="griptape"
)

vector_db = VectorStoreClient(
    description="This DB has information about the Griptape Python framework",
    query_engine=engine,
    namespace="griptape"
)

agent = Agent(
    tools=[vector_db, TaskMemoryClient(off_prompt=False)]
)

agent.run(
    "what is Griptape?"
)

Full Output:

[05/15/24 09:14:48] INFO     ToolkitTask 7fded25e6c244321a9233a3badbb4fa6                                                  
                             Input: what is Griptape?                                                                      
[05/15/24 09:14:52] INFO     Subtask 9c9d60f4761d432986781d48fe724a43                                                      
                             Thought: I need to search the vector database for information about the Griptape Python       
                             framework.                                                                                    
                             Actions: [                                                                                    
                               {                                                                                           
                                 "name": "VectorStoreClient",                                                              
                                 "path": "search",                                                                         
                                 "input": {                                                                                
                                   "values": {                                                                             
                                     "query": "What is Griptape Python framework?"                                         
                                   }                                                                                       
                                 },                                                                                        
                                 "tag": "search_griptape"                                                                  
                               }                                                                                           
                             ]                                                                                             
[05/15/24 09:14:53] INFO     Subtask 9c9d60f4761d432986781d48fe724a43                                                      
                             Response: error querying vector store: string indices must be integers, not 'str'             
[05/15/24 09:14:58] INFO     Subtask 71dac42b65464856831bdb342604789d                                                      
                             Thought: There seems to be an error with the way I structured the action. I need to correct   
                             the input format and try again.                                                               
                                                                                                                           
                             Actions: [                                                                                    
                               {                                                                                           
                                 "name": "VectorStoreClient",                                                              
                                 "path": "search",                                                                         
                                 "input": {                                                                                
                                   "values": {                                                                             
                                     "query": "What is Griptape Python framework?"                                         
                                   }                                                                                       
                                 },                                                                                        
                                 "tag": "search_griptape"                                                                  
                               }                                                                                           
                             ]                                                                                             
                    INFO     Subtask 71dac42b65464856831bdb342604789d                                                      
                             Response: error querying vector store: string indices must be integers, not 'str'             
[05/15/24 09:15:05] INFO     Subtask 71d47cb90fe8491184186bb9dd47824a                                                      
                             Thought: The same error occurred again. It seems there might be an issue with the             
                             VectorStoreClient action. I will try to use the TaskMemoryClient action to search the memory  
                             content for information about Griptape.                                                       
                                                                                                                           
                             Actions: [{"tag": "query_griptape", "name": "TaskMemoryClient", "path": "query", "input":     
                             {"values": {"memory_name": "griptape_info", "artifact_namespace": "griptape", "query": "What  
                             is Griptape Python framework?"}}}]                                                            
                    INFO     Subtask 71d47cb90fe8491184186bb9dd47824a                                                      
                             Response: memory not found                                                                    
[05/15/24 09:15:10] INFO     Subtask b760ab6da0f1473685c8bca126477928                                                      
                             Thought: The memory does not contain information about Griptape. I will try to summarize the  
                             memory content to see if there is any relevant information.                                   
                             Actions: [{"tag": "summarize_memory", "name": "TaskMemoryClient", "path": "summarize",        
                             "input": {"values": {"memory_name": "griptape_info", "artifact_namespace": "griptape"}}}]     
                    INFO     Subtask b760ab6da0f1473685c8bca126477928                                                      
                             Response: memory not found                                                                    
[05/15/24 09:15:15] INFO     ToolkitTask 7fded25e6c244321a9233a3badbb4fa6                                                  
                             Output: I'm sorry, but I am currently unable to provide information about Griptape.