Devuan

Devuan is a GNU+Linux operating system fork of Debian without systemd that allows users to reclaim control over their system by avoiding unnecessary entanglements and ensuring Init Freedom. Devuan runs LLM's.

After install

# apt-get install aptitude vim curl wget ca-certificates

In this case machine have a AMD/ATI GPU, machine is old and the propose is to test / prototype. Drivers seem to be installed with base install.

# apt-get install radeontop clinfo

Set your default editor;

$ update-alternatives --config editor
$ su -
# update-alternatives --config editor

Add user to sudo group;

$ su -
# sudo usermod -aG sudo username

Make sure ssh server only accepts key login and not passwords;

$ sudo vim /etc/ssh/sshd_config

Change following lines;

PasswordAuthentication no
PubkeyAuthentication yes
UsePAM no

Install Ollama

Install Ollama;

$ curl -fsSL https://ollama.com/install.sh | sh

For more information on how to finish setup read ollama page.

Install Hermes

Read hermes page for more information.

Performance

You can check machine hardware. Hardware is old for today standards but shows how it is possible to test and run models on modest hardware or how to save in computing and energy bills.