Run LLMs on weak devices or make powerful devices even more powerful by distributing the workload and dividing the RAM usage.

The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM models, execute structured function calls and get structured output. Works also with models not fine-tuned to JSON output and function calls.

the AI-native open-source embedding database

Get up and running with Llama 3, Mistral, Gemma, and other large language models.

Chat with your legal assistant.

User-friendly WebUI for LLMs (Formerly Ollama WebUI)

Self-hosted AI coding assistant

Sora 的中文指南🔥，Sora 中文调教指南，指令指南，应用开发指南，精选资源清单，Sora 开发者精选工具框架 🚀

Access large language models from the command-line

Work with LLMs on a local environment using containers

🤖 Collect practical AI repos, tools, websites, papers and tutorials on AI. 实用的AI百宝箱 💎

Structured Text Generation

VT.ai - Multi-modal AI Chatbot Assistant

TypeScript SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)

Finetune Llama 3, Mistral & Gemma LLMs 2-5x faster with 80% less memory

Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, with more integrations coming..

Build LLM-powered Dart/Flutter applications.

A simple LLM client for Emacs

RepairLLaMA: Efficient Representations and Fine-Tuned Adapters for Program Repair

Pretrain, finetune, deploy 20+ LLMs on your own data. Uses state-of-the-art techniques: flash attention, FSDP, 4-bit, LoRA, and more.