Skip to content

A project written in the Rust language with the goal of offline load of small LLM Model, specifically RAG (Retrieval Augmented Generation) on mobile devices.

License

Notifications You must be signed in to change notification settings

smalltong02/keras-rag-chatbot

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

keras-rag-chatbot

image

Please downloaded openchat_3.5.Q4_K_M.gguf and tokenizer.json from Huggingface:

download TheBloke/openchat_3.5-GGUF openchat_3.5.Q4_K_M.gguf
download openchat/openchat_3.5 tokenizer.json

Copy them to the folder ./models/openchat-3.5-gguf/

and then

cargo run --release

About

A project written in the Rust language with the goal of offline load of small LLM Model, specifically RAG (Retrieval Augmented Generation) on mobile devices.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages