forked from third-party-mirrors/ollama
7893ccb68c
This commit aims to provide the Ollama maintainers with maximum control of the distribution build process by creating a cross-platform shim. Currently, we have no flexibility, or control of the process (pre and post) or even the quality of the build. By introducing a shim, and propagating it out to Homebrew, et al., we can soon after ensure that the build process is consistent, and reliable. This also happens to remove the requirement for go generate and the build tag hacks, but it does still support go generate in the flow, at least until we can remove it after the major distribution use the new build process. About the script: Beyond giving the Ollama maintainers drastically more control over the build process, the script also provides a few other benefits: - It is cross-platform, and can be run on any platform that supports Go (a hard requirement for building Ollama anyway). - It can can check for correct versions of cmake, and other dependencies before starting the build process, and provide helpful error messages to the user if they are not met. - It can be used to build the distribution for any platform, architecture, or build type (debug, release, etc.) with a single command. Currently, it is two commands. - It can skip parts of the build process if they are already done, such as build the C dependencies. Of course there is a -f flag to force rebuild. - So much more!
143 lines
4.7 KiB
Markdown
143 lines
4.7 KiB
Markdown
# Development
|
|
|
|
Install required tools:
|
|
|
|
- cmake version 3.24 or higher
|
|
- go version 1.22 or higher
|
|
- gcc version 11.4.0 or higher
|
|
|
|
### MacOS
|
|
|
|
```bash
|
|
brew install go cmake gcc
|
|
```
|
|
|
|
Optionally enable debugging and more verbose logging:
|
|
|
|
```bash
|
|
# At build time
|
|
export CGO_CFLAGS="-g"
|
|
|
|
# At runtime
|
|
export OLLAMA_DEBUG=1
|
|
```
|
|
|
|
Get the required libraries and build the native LLM code:
|
|
|
|
```bash
|
|
go run build.go
|
|
```
|
|
|
|
Now you can run `ollama`:
|
|
|
|
```bash
|
|
./ollama
|
|
```
|
|
|
|
### Rebuilding the native code
|
|
|
|
If at any point you need to rebuild the native code, you can run the
|
|
build.go script again using the `-f` flag to force a rebuild, and,
|
|
optionally, the `-d` flag to skip building the Go binary:
|
|
|
|
```bash
|
|
go run build.go -d -s
|
|
```
|
|
|
|
### Linux
|
|
|
|
#### Linux CUDA (NVIDIA)
|
|
|
|
_Your operating system distribution may already have packages for NVIDIA CUDA. Distro packages are often preferable, but instructions are distro-specific. Please consult distro-specific docs for dependencies if available!_
|
|
|
|
Install `cmake` and `golang` as well as [NVIDIA CUDA](https://developer.nvidia.com/cuda-downloads)
|
|
development and runtime packages.
|
|
|
|
Typically the build scripts will auto-detect CUDA, however, if your Linux distro
|
|
or installation approach uses unusual paths, you can specify the location by
|
|
specifying an environment variable `CUDA_LIB_DIR` to the location of the shared
|
|
libraries, and `CUDACXX` to the location of the nvcc compiler. You can customize
|
|
a set of target CUDA architectures by setting `CMAKE_CUDA_ARCHITECTURES` (e.g. "50;60;70")
|
|
|
|
Then build the binary:
|
|
|
|
```
|
|
go run build.go
|
|
```
|
|
|
|
#### Linux ROCm (AMD)
|
|
|
|
_Your operating system distribution may already have packages for AMD ROCm and CLBlast. Distro packages are often preferable, but instructions are distro-specific. Please consult distro-specific docs for dependencies if available!_
|
|
|
|
Install [CLBlast](https://github.com/CNugteren/CLBlast/blob/master/doc/installation.md) and [ROCm](https://rocm.docs.amd.com/en/latest/) development packages first, as well as `cmake` and `golang`.
|
|
|
|
Typically the build scripts will auto-detect ROCm, however, if your Linux distro
|
|
or installation approach uses unusual paths, you can specify the location by
|
|
specifying an environment variable `ROCM_PATH` to the location of the ROCm
|
|
install (typically `/opt/rocm`), and `CLBlast_DIR` to the location of the
|
|
CLBlast install (typically `/usr/lib/cmake/CLBlast`). You can also customize
|
|
the AMD GPU targets by setting AMDGPU_TARGETS (e.g. `AMDGPU_TARGETS="gfx1101;gfx1102"`)
|
|
|
|
Then build the binary:
|
|
|
|
```
|
|
go run build.go
|
|
```
|
|
|
|
ROCm requires elevated privileges to access the GPU at runtime. On most distros you can add your user account to the `render` group, or run as root.
|
|
|
|
#### Advanced CPU Settings
|
|
|
|
By default, running `go run build.go` will compile a few different variations
|
|
of the LLM library based on common CPU families and vector math capabilities,
|
|
including a lowest-common-denominator which should run on almost any 64 bit CPU
|
|
somewhat slowly. At runtime, Ollama will auto-detect the optimal variation to
|
|
load. If you would like to build a CPU-based build customized for your
|
|
processor, you can set `OLLAMA_CUSTOM_CPU_DEFS` to the llama.cpp flags you would
|
|
like to use. For example, to compile an optimized binary for an Intel i9-9880H,
|
|
you might use:
|
|
|
|
```
|
|
OLLAMA_CUSTOM_CPU_DEFS="-DLLAMA_AVX=on -DLLAMA_AVX2=on -DLLAMA_F16C=on -DLLAMA_FMA=on" go run build.go
|
|
```
|
|
|
|
#### Containerized Linux Build
|
|
|
|
If you have Docker available, you can build linux binaries with `./scripts/build_linux.sh` which has the CUDA and ROCm dependencies included. The resulting binary is placed in `./dist`
|
|
|
|
### Windows
|
|
|
|
Note: The Windows build for Ollama is still under development.
|
|
|
|
First, install required tools:
|
|
|
|
- MSVC toolchain - C/C++ and cmake as minimal requirements
|
|
- Go version 1.22 or higher
|
|
- MinGW (pick one variant) with GCC.
|
|
- [MinGW-w64](https://www.mingw-w64.org/)
|
|
- [MSYS2](https://www.msys2.org/)
|
|
- The `ThreadJob` Powershell module: `Install-Module -Name ThreadJob -Scope CurrentUser`
|
|
|
|
Then, build the `ollama` binary:
|
|
|
|
```powershell
|
|
$env:CGO_ENABLED="1"
|
|
go run build.go
|
|
```
|
|
|
|
#### Windows CUDA (NVIDIA)
|
|
|
|
In addition to the common Windows development tools described above, install CUDA after installing MSVC.
|
|
|
|
- [NVIDIA CUDA](https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html)
|
|
|
|
|
|
#### Windows ROCm (AMD Radeon)
|
|
|
|
In addition to the common Windows development tools described above, install AMDs HIP package after installing MSVC.
|
|
|
|
- [AMD HIP](https://www.amd.com/en/developer/resources/rocm-hub/hip-sdk.html)
|
|
- [Strawberry Perl](https://strawberryperl.com/)
|
|
|
|
Lastly, add `ninja.exe` included with MSVC to the system path (e.g. `C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja`).
|