llamactl

llamactl - Local GGUF model operations

Small Go CLI that manages GGUF models, presets, downloads, and llama.cpp server lifecycle for local LLM inference.

As part of my daily-driving, local-AI stack, I was running several GGUF models with llama.cpp. I wanted one place for model files, presets, server processes, logs, and health checks. So, I created this Go CLI. The CLI downloads models from Hugging Face, starts and stops llama-server , follows logs, checks VRAM, and sends quick prompts through llama-cli . It works with native llama.cpp, CPU containers, CUDA 13, and rootless NVIDIA CDI. Models can stay loaded or start on demand behind an OpenAI-compatible API.

Open source project · 2026 · Solo developer