Running Ollama in a Linux Environment Without Root Privileges

Running applications without root privileges is a common requirement for security and compliance reasons, especially on shared systems or in environments where users don't have administrative access. This guide will walk you through setting up ollama on a Linux system under these constraints, using some tools like uv to manage Python dependencies and configuring networking correctly. Maybe, this is not the easiest way and using Docker could be simpler, but it's a good exercise to understand how to manage Python applications without root privileges.

Prerequisites

• A functional Python environment.
• Basic command-line navigation skills.
• Network access for downloading necessary files and configuring network settings.

Step-by-step Guide

1. Install uv

Firstly, we need a tool to handle Python applications efficiently without root privileges. The tool I used is uv. Here’s how you can install it:

curl -LsSf https://astral.sh/uv/install.sh | sh

This command downloads and executes the installation script for uv. After installing, verify its presence using:

which uv

You might need to restart …

I have been using LaTeX for a while now and I have always been annoyed by the fact that I have to run the pdflatex command multiple times to get the references and the table of contents right. Today, I colleague of mine told me about the latexmk command. It is a tool that takes care of all the dependencies and runs the necessary commands to generate the PDF file.

You can run:

latexmk -pdf yourfile.tex

And it will take care of everything. It will run pdflatex multiple times if necessary, it will run bibtex (or 'biber', as in my case) if you have a bibliography, and it will generate the PDF file.

I'm working on a Jetson Nano and I want to profile my code. Nvidia provides several tools, being Nsight Compute the most powerful one. However, it cannot be run on the Jetson Nano, so I have resorted to using nvprof.

nvprof is a command-line profiler that can be used to profile CUDA applications. It is included in the CUDA Toolkit, so you should have it installed if you have CUDA installed. However, you need to be root to run it. As root, I don't have the CUDA environment set up, so I need use the full path. These are two commands that I've found useful:

/usr/local/cuda/bin/nvprof --print-gpu-trace ./my_program

This shows the GPU trace of the program. It is useful to see how the GPU is being used.

/usr/local/cuda/bin/nvprof --metrics all ./my_program

This shows all the metrics that nvprof can measure. It is useful to see how the program is using the GPU.

In addition, you can use the --log-file option to save the output to a file …

I've found this interesting post from 2022 by Nvidia about GPU sharing in Kubernetes.

The main GPU sharing technologies can be summarized in this table:

The post also explains how GPUs are advertised as schedulable resources in Kubernetes with the device plugin framework, but it is a integer-based resource, so it does not allow for oversuscription. They describe a way of achieving this with time-slicing APIs.

I've been reading about the history of GPU computing in chapter 2 of "Massively Parallel Processors" by David B. Kirk and Wen-mei W. Hwu. It's a fascinating story of how the GPU came to be used for general-purpose computing.

The story begins in the 1990s, when the first consumer 3D graphics cards were being developed. These cards were designed to accelerate the rendering of 3D graphics for video games. They were able to do this by offloading the rendering work from the CPU to the GPU, which was specifically designed for this task.

The first GPUs were fixed-function, meaning that they could only perform a limited set of operations. However, as the demand for more realistic and complex graphics grew, the capabilities of the GPU were expanded. This led to the development of programmable shaders, which allowed developers to write custom code to control the rendering process. In the beginning, these shaders were still limited to graphics-related tasks and there were different kinds, such as vertex shaders and pixel shaders.

One of the questions I …