Ollama

How to setup Ollama on hardware-constrained system to develop agents with RAG capabilities. We will change configuration as we test and setup all the tools.

Setup service

#!/bin/sh
### BEGIN INIT INFO
# Provides:          ollama
# Required-Start:    $network $local_fs $remote_fs
# Required-Stop:     $network $local_fs $remote_fs
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Ollama AI service
# Description:       Ollama local AI model server
### END INIT INFO

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/local/bin/ollama
NAME=ollama
DESC="Ollama AI service"
PIDFILE=/var/run/ollama.pid
USER=ollama
LOGFILE=/var/log/ollama.log
OLLAMA_HOST=192.168.1.4:11434
OLLAMA_KEEP_ALIVE=-1
OLLAMA_REQUEST_TIMEOUT=3600
export OLLAMA_HOST
export OLLAMA_KEEP_ALIVE
export OLLAMA_REQUEST_TIMEOUT

# Check if daemon exists
test -x $DAEMON || exit 0

. /lib/lsb/init-functions

case "$1" in
  start)
    log_daemon_msg "Starting $DESC" "$NAME"
    start-stop-daemon --start --quiet --background --make-pidfile \
        --pidfile $PIDFILE --chuid $USER --exec $DAEMON -- serve >> $LOGFILE 2>&1
    log_end_msg $?
    ;;
  stop)
    log_daemon_msg "Stopping $DESC" "$NAME"
    start-stop-daemon --stop --quiet --pidfile $PIDFILE --retry 5
    rm -f $PIDFILE
    log_end_msg $?
    ;;
  restart|force-reload)
    $0 stop@ ]
    sleep 1
    $0 start
    ;;
  status)
    status_of_proc -p $PIDFILE $DAEMON $NAME && exit 0 || exit $?
    ;;
  *)
    echo "Usage: /etc/init.d/$NAME {start|stop|restart|force-reload|status}" >&2
    exit 1
    ;;
esac

exit 0

Above remove space at "$0 stop@ ]" to be "$0 stop@]" this was due to not finding how to escape it. Make it executable and setup:

$ sudo chmod +x /etc/init.d/ollama
$ sudo update-rc.d ollama defaults

Create the ollama user:

$ sudo useradd -r -s /bin/false -d /home/ollama ollama

Create log file:

$ sudo touch /var/log/ollama.log
$ sudo chown ollama:ollama /var/log/ollama.log

Start the service:

$ sudo /etc/init.d/ollama start

Check status:

$ sudo /etc/init.d/ollama status

Setup user

Service will run on IP set in script and if user calls ollama client will get following error;

$ ollama list
Error: ollama server not responding - could not connect to ollama server, run 'ollama serve' to start it

Add this to your user's ~/.bash_aliases (not the ollama user's):

alias ollama-admin='sudo -u ollama OLLAMA_HOST=http://192.168.1.4:11434 ollama'

Bash ~/.bashrc should contain following lines;

# AI application
export OLLAMA_HOST=192.168.1.4
# keep the models loaded
export OLLAMA_KEEP_ALIVE=-1
export PATH="$HOME/.npm-global/bin:$PATH"

# Alias definitions.
# You may want to put all your additions into a separate file like
# ~/.bash_aliases, instead of adding them here directly.
# See /usr/share/doc/bash-doc/examples in the bash-doc package.

if [ -f ~/.bash_aliases ]; then
    . ~/.bash_aliases
fi

Source it;

$ . ~/.bashrc

Pull model

To pull models become user service is running as and then run pull command:

$ ollama-admin pull smollm:135m 
$ ollama-admin pull qwen3:0.6b
$ ollama-admin pull qwen3:1.7b
$ ollama-admin pull qwen3:4b
$ ollama-admin list

Test run

Run as a normal user:

$ ollama-admin  qwen3:4b