Projects to test AI chat bots locally

Many people is interested in trying AI (Artificial Intelligence) chat bots locally, but they don't know how to start. These are some projects that I've found that can be used to test different LLMs (Language Learning Models) and chat bots:

• Text generation web UI. It tries to be the AUTOMATIC1111/stable-diffusion-webui of text generation. Based on Gradio, it provides a web interface to test different LLMs.

• Serge. It's an interface based on Svelte as web framework and llama.cpp for running mmodels. It's dockerized. It also provides an API.

• openplayground. Works with local models and remote APIs. Can be installed as a Python package or run with Docker. It's implemented as a Flask application with a React frontend.

• ChatAI. It is a desktop application for Windows and Ubuntu developed in Python with PyQt. It uses a Google T5-Flan model and it is based on A. I. Runner, a framework to run AI models.

• ColossalChat. ColossalAI provides a set of tools to develop deep learning models, and it includes ColossalChat, which is a chat bot based on LLaMA models, working for English and Chinese. They say in this article it's the first open source solution with a whole RLHF (Reinforcement Learning with Human Feedback) pipeline to improve the chat bot, in the same way as ChatGPT. It looks like the focus is on providing tools to speed up training and inference.

• LocalAI. Based on llama.cpp, gpt4all and ggml, it provides a local API to LLMs. Basically, it uses docker to deploy a web server developed in Go.

• MLC LLM. It's a project to deploy LLMs on everyone's devices. It compiles the LLMs using Apache TVM Unity. They even have a project to deploy LLMs on web browsers, Web LLM.