Running Ollama in a Linux Environment Without Root Privileges

Running applications without root privileges is a common requirement for security and compliance reasons, especially on shared systems or in environments where users don't have administrative access. This guide will walk you through setting up ollama on a Linux system under these constraints, using some tools like uv to manage Python dependencies and configuring networking correctly. Maybe, this is not the easiest way and using Docker could be simpler, but it's a good exercise to understand how to manage Python applications without root privileges.

Prerequisites

• A functional Python environment.
• Basic command-line navigation skills.
• Network access for downloading necessary files and configuring network settings.

Step-by-step Guide

1. Install uv

Firstly, we need a tool to handle Python applications efficiently without root privileges. The tool I used is uv. Here’s how you can install it:

curl -LsSf https://astral.sh/uv/install.sh | sh

This command downloads and executes the installation script for uv. After installing, verify its presence using:

which uv

You might need to restart …

From my previous post about projects to test AI chat bots locally, the field has not been standing still.

I've found that the easiest way to try LLMs locally is to use llamafile: download a file, run it and that's it.

I've also tried LM studio, which is more powerful, but it's also more complicated to use.

Many people is interested in trying AI (Artificial Intelligence) chat bots locally, but they don't know how to start. These are some projects that I've found that can be used to test different LLMs (Language Learning Models) and chat bots:

• Text generation web UI. It tries to be the AUTOMATIC1111/stable-diffusion-webui of text generation. Based on Gradio, it provides a web interface to test different LLMs.

• Serge. It's an interface based on Svelte as web framework and llama.cpp for running mmodels. It's dockerized. It also provides an API.

• openplayground. Works with local models and remote APIs. Can be installed as a Python package or run with Docker. It's implemented as a Flask application with a React frontend.

• ChatAI. It is a desktop application for Windows and Ubuntu developed in Python with PyQt. It uses a Google T5-Flan model and it is based on A. I. Runner, a framework to run AI models.

• ColossalChat. ColossalAI provides a set of tools to develop deep learning models, and it includes ColossalChat, which is a chat bot based …