From 14d22902a74ea18302affe313083792e636ae844 Mon Sep 17 00:00:00 2001 From: Yuri Khrustalev Date: Wed, 10 Jul 2024 13:15:00 +0000 Subject: [PATCH 1/4] Configure the systemd service via a separate file. Instead of changing the systemd unit file, use a separate config for the configuration. refs #3516 --- docs/faq.md | 16 +++++----------- docs/linux.md | 7 ++++++- scripts/install.sh | 13 +++++++++++++ 3 files changed, 24 insertions(+), 12 deletions(-) diff --git a/docs/faq.md b/docs/faq.md index da1848f7..e48507d3 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -73,23 +73,17 @@ If Ollama is run as a macOS application, environment variables should be set usi ### Setting environment variables on Linux -If Ollama is run as a systemd service, environment variables should be set using `systemctl`: +If Ollama is run as a systemd service, environment variables should be set via the file specified in the `EnvironmentFile` option of the systemd unit: -1. Edit the systemd service by calling `systemctl edit ollama.service`. This will open an editor. - -2. For each environment variable, add a line `Environment` under section `[Service]`: +1. Edit the file `/etc/ollama/serve.conf`: ```ini - [Service] - Environment="OLLAMA_HOST=0.0.0.0" + OLLAMA_HOST=0.0.0.0 ``` -3. Save and exit. - -4. Reload `systemd` and restart Ollama: +2. Restart Ollama: ```bash - systemctl daemon-reload systemctl restart ollama ``` @@ -272,4 +266,4 @@ The following server settings may be used to adjust how Ollama handles concurren - `OLLAMA_NUM_PARALLEL` - The maximum number of parallel requests each model will process at the same time. The default will auto-select either 4 or 1 based on available memory. - `OLLAMA_MAX_QUEUE` - The maximum number of requests Ollama will queue when busy before rejecting additional requests. The default is 512 -Note: Windows with Radeon GPUs currently default to 1 model maximum due to limitations in ROCm v5.7 for available VRAM reporting. Once ROCm v6.2 is available, Windows Radeon will follow the defaults above. You may enable concurrent model loads on Radeon on Windows, but ensure you don't load more models than will fit into your GPUs VRAM. \ No newline at end of file +Note: Windows with Radeon GPUs currently default to 1 model maximum due to limitations in ROCm v5.7 for available VRAM reporting. Once ROCm v6.2 is available, Windows Radeon will follow the defaults above. You may enable concurrent model loads on Radeon on Windows, but ensure you don't load more models than will fit into your GPUs VRAM. diff --git a/docs/linux.md b/docs/linux.md index ec730656..ad64679c 100644 --- a/docs/linux.md +++ b/docs/linux.md @@ -102,7 +102,7 @@ sudo chmod +x /usr/bin/ollama ## Installing specific versions -Use `OLLAMA_VERSION` environment variable with the install script to install a specific version of Ollama, including pre-releases. You can find the version numbers in the [releases page](https://github.com/ollama/ollama/releases). +Use `OLLAMA_VERSION` environment variable with the install script to install a specific version of Ollama, including pre-releases. You can find the version numbers in the [releases page](https://github.com/ollama/ollama/releases). For example: @@ -141,3 +141,8 @@ sudo rm -r /usr/share/ollama sudo userdel ollama sudo groupdel ollama ``` + +Remove the config +```bash +sudo rm -f /etc/ollama +``` diff --git a/scripts/install.sh b/scripts/install.sh index 2a06c350..2f4edb21 100644 --- a/scripts/install.sh +++ b/scripts/install.sh @@ -99,6 +99,18 @@ configure_systemd() { status "Adding current user to ollama group..." $SUDO usermod -a -G ollama $(whoami) + if $SUDO test -f "/etc/ollama/serve.conf" ; then + status "Skip creating ollama serve config file..." + else + status "Creating ollama serve config file..." + $SUDO mkdir -p /etc/ollama/ + cat </dev/null +# The list of supported env variables https://github.com/ollama/ollama/blob/main/envconfig/config.go +#OLLAMA_DEBUG=1 +#OLLAMA_HOST=0.0.0.0:11434 +EOF + fi + status "Creating ollama systemd service..." cat </dev/null [Unit] @@ -112,6 +124,7 @@ Group=ollama Restart=always RestartSec=3 Environment="PATH=$PATH" +EnvironmentFile=/etc/ollama/serve.conf [Install] WantedBy=default.target From 17b29a2d0189c44dfb5cba0815466f07ffd55490 Mon Sep 17 00:00:00 2001 From: Yuri Khrustalev Date: Mon, 15 Jul 2024 22:52:16 -0400 Subject: [PATCH 2/4] Update linux.md --- docs/linux.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/linux.md b/docs/linux.md index ad64679c..b85cd670 100644 --- a/docs/linux.md +++ b/docs/linux.md @@ -144,5 +144,5 @@ sudo groupdel ollama Remove the config ```bash -sudo rm -f /etc/ollama +sudo rm -rf /etc/ollama ``` From 96883f49278ac0c53ce467d840d0b3e77c67d432 Mon Sep 17 00:00:00 2001 From: Yuri Khrustalev Date: Fri, 26 Jul 2024 15:30:14 -0400 Subject: [PATCH 3/4] Update install.sh --- scripts/install.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/install.sh b/scripts/install.sh index 2f4edb21..226535ee 100644 --- a/scripts/install.sh +++ b/scripts/install.sh @@ -105,7 +105,7 @@ configure_systemd() { status "Creating ollama serve config file..." $SUDO mkdir -p /etc/ollama/ cat </dev/null -# The list of supported env variables https://github.com/ollama/ollama/blob/main/envconfig/config.go +# The list of supported env variables could be found by running: ollama serve --help #OLLAMA_DEBUG=1 #OLLAMA_HOST=0.0.0.0:11434 EOF From a384a5bd24b608606219ad2baf35c23088c6f03f Mon Sep 17 00:00:00 2001 From: Yuri Khrustalev Date: Tue, 30 Jul 2024 20:09:47 -0400 Subject: [PATCH 4/4] Update faq.md