[Question]:

[ddealwis09]

Question Validation

• I have searched both the documentation and discord for an answer.

Question

Using SimpleDirectoryReader for my new document or nodes, how can I load them into an existing FAISS vector store that I have persisting in ./storage? The index.insert and index.insert_nodes methods are not working.
The docstore.json and index_store.json are not being updated in the ./storage directory. Thanks!

[dosubot[bot]]

🤖

Hello,

Based on the context provided, it seems like you're trying to load new documents or nodes into an existing FAISS vector store that is persisting in ./storage. The index.insert and index.insert_nodes methods are not updating the docstore.json and index_store.json in the ./storage directory.

You can use the add method of the FaissVectorStore to add new nodes to the FAISS vector store. Then, you can use the persist method of the SimpleIndexStore to save the current state of the vector store to a file. Here is an example:

from llama_index.vector_stores.faiss import FaissVectorStore
from llama_index.schema import TextNode
from llama_index.storage.index_store.simple_index_store import SimpleIndexStore

# Create a new node
node = TextNode(text="new node", id_="d4", metadata={"node": "info"})

# Load the existing vector store from the persisted file
vector_store = FaissVectorStore.from_persist_path('./storage/index_store.json')

# Add the new node to the vector store
vector_store.add([node])

# Load the existing index store
index_store = SimpleIndexStore.from_persist_path('./storage/index_store.json')

# Persist the updated index store
index_store.persist('./storage/index_store.json')

This will update the index_store.json file in the ./storage directory with the new node.

As for the index.insert method, it is used to insert a sequence of nodes into the index graph. It does this by creating an instance of the TreeIndexInserter class and calling its insert method with the nodes as an argument. The TreeIndexInserter is responsible for inserting the nodes into the index structure.

The index.insert_nodes method does not exist in the provided context. Therefore, I cannot provide information about it.

I hope this helps! If you have any other questions, feel free to ask.

Best,
Dosu

Sources

• [Question]: Is it possible to append new nodes to an existing index without overriding the whole index?
• tests/storage/docstore/test_simple_docstore.py
• llama_index/vector_stores/faiss.py
• llama_index/storage/index_store/simple_index_store.py
• llama_index/indices/tree/inserter.py
• llama_index/indices/tree/base.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.