run model and add to openclaw
i run:
docker run -d \
--name llama-server \
--restart unless-stopped \
--privileged \
--network=host \
--device=/dev/kfd \
--device=/dev/dri \
--group-add video \
--group-add render \
--cap-add=SYS_PTRACE \
--security-opt seccomp=unconfined \
--ipc=host \
--shm-size 32G \
-v ~/models:/data \
docker.io/mixa3607/llama.cpp-gfx906:full-b7091-rocm-6.3.3 \
--server \
-m /data/nanbeige4.1-3b-q8_0.gguf \
--host 0.0.0.0 \
--port 8080 \
-c 131072 \
-n 4096 \
--temp 0.6 \
--top-p 0.95 \
-ngl 999 \
--parallel 2 \
then at openclaw i try add:
QuickStart โโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ โ
โ Gateway port: 18789 โ
โ Gateway bind: Loopback (127.0.0.1) โ
โ Gateway auth: Token (default) โ
โ Tailscale exposure: Off โ
โ Direct to chat channels. โ
โ โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โ
โ Model/auth provider
โ Custom Provider
โ
โ API Base URL
โ http://192.168.1.59:8080/v1
โ
โ API Key (leave blank if not required)
โ 123
โ
โ Endpoint compatibility
โ OpenAI-compatible
โ
โ Model ID
โ /data/nanbeige4.1-3b-q8_0.gguf
โ
โ Verification failed: This operation was aborted
โ
โ What would you like to change?
โ โ Change base URL
โ โ Change model
โ โ Change base URL and model
docker run -d \
--name llama-server \
--restart unless-stopped \
--privileged \
--network=host \
--device=/dev/kfd \
--device=/dev/dri \
--group-add video \
--group-add render \
--cap-add=SYS_PTRACE \
--security-opt seccomp=unconfined \
--ipc=host \
--shm-size 32G \
-v ~/models:/data \
docker.io/mixa3607/llama.cpp-gfx906:full-b7091-rocm-6.3.3 \
--server \
-m /data/nanbeige4.1-3b-q8_0.gguf \
--host 0.0.0.0 \
--port 8080 \
-c 131072 \
-n 4096 \
--temp 0.6 \
--top-p 0.95 \
-ngl 999 \
--parallel 2 \
then at openclaw i try add:
QuickStart โโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ โ
โ Gateway port: 18789 โ
โ Gateway bind: Loopback (127.0.0.1) โ
โ Gateway auth: Token (default) โ
โ Tailscale exposure: Off โ
โ Direct to chat channels. โ
โ โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โ
โ Model/auth provider
โ Custom Provider
โ
โ API Base URL
โ http://192.168.1.59:8080/v1
โ
โ API Key (leave blank if not required)
โ 123
โ
โ Endpoint compatibility
โ OpenAI-compatible
โ
โ Model ID
โ /data/nanbeige4.1-3b-q8_0.gguf
โ
โ Verification failed: This operation was aborted
โ
โ What would you like to change?
โ โ Change base URL
โ โ Change model
โ โ Change base URL and model
The AI that actually does things. Emails, calendar, home automation - all from your favorite chat app. New shell, ready to help. The lobster way.
131,281Members
Resources
Similar Threads
Was this page helpful?
