LocalAI 部署

介紹

LocalAI 是免費(fèi)的開源 OpenAI 替代方案。 LocalAI 充當(dāng) REST API 的直接替代品，與本地推理的 OpenAI API 規(guī)范兼容。 它無需 GPU，還有多種用途集成，允許您使用消費(fèi)級硬件在本地或本地運(yùn)行 LLM、生成圖像、音頻等等，支持多個模型系列。

啟動方式

1. Linux AMD64 docker 啟動

helm repo add go-skynet https://go-skynet.github.io/helm-charts/
helm search repo go-skynet
helm pull go-skynet/local-ai
tar -xvf local-ai-3.1.0.tgz && cd local-ai
vim value.yaml
# 取消下面截圖的注釋

helm install --create-namespace local-ai . -n local-ai -f values.yaml

2. Mac M2 手動啟動

# install build dependencies
brew install abseil cmake go grpc protobuf wget# clone the repo
git clone https://github.com/go-skynet/LocalAI.gitcd LocalAI# build the binary
make build
# make BUILD_TYPE=metal build
## Set `gpu_layers: 1` to your YAML model config file and `f16: true`
## Note: only models quantized with q4_0 are supported!# Download gpt4all-j to models/
wget https://gpt4all.io/models/ggml-gpt4all-j.bin -O models/ggml-gpt4all-j# Use a template from the examples
cp -rf prompt-templates/ggml-gpt4all-j.tmpl models/# Run LocalAI
./local-ai --models-path=./models/ --debug=true

使用

# Now API is accessible at localhost:8080
curl http://localhost:8080/v1/models
curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{"model": "ggml-gpt4all-j","messages": [{"role": "user", "content": "How are you?"}],"temperature": 0.9}'

官方編譯啟動文檔

FQA

Q1: 編譯報錯日志 sources/go-llama/llama.go:372:13: undefined: min

binding.cpp:333:67: warning: format specifies type 'size_t' (aka 'unsigned long') but the argument has type 'int' [-Wformat]
binding.cpp:809:5: warning: deleting pointer to incomplete type 'llama_model' may cause undefined behavior [-Wdelete-incomplete]
sources/go-llama/llama.cpp/llama.h:60:12: note: forward declaration of 'llama_model'
# github.com/go-skynet/go-llama.cpp
sources/go-llama/llama.go:372:13: undefined: min
note: module requires Go 1.21
make: *** [backend-assets/grpc/llama] Error 1

需要使用 go 1.21 版本

brew install mercurial
# 安裝 gvm
bash < <(curl -s -S -L https://raw.githubusercontent.com/moovweb/gvm/master/binscripts/gvm-installer)
# gvm直接生效
source ~/.gvm/scripts/gvm
# 查看版本
gvm install go1.21.7
gvm use go1.21.7

Q2: The link interface of target “protobuf::libprotobuf” contains: absl::absl_check, but the target was not found

CMake Error at /opt/homebrew/lib/cmake/protobuf/protobuf-targets.cmake:71 (set_target_properties):The link interface of target "protobuf::libprotobuf" contains:absl::absl_checkbut the target was not found.  Possible reasons include:* There is a typo in the target name.* A find_package call is missing for an IMPORTED target.* An ALIAS target is missing.Call Stack (most recent call first):/opt/homebrew/lib/cmake/protobuf/protobuf-config.cmake:16 (include)examples/grpc-server/CMakeLists.txt:34 (find_package)

需要更新一下 protobuf 和 abseil 版本

brew uninstall protobuf abseil
sudo port install re2 grpc abseil