Run LLMs on-premise, on GPU servers you already control.

On-premise AI used to mean a team of engineers SSH-ing into GPU boxes, compiling inference servers and keeping model versions in sync by hand. Loadcore turns that into one agent per server and one console for the whole fleet, while inference stays on your hardware.

Open the console Read the docs

What on-premise means with Loadcore

Loadcore separates inference traffic from control traffic:

Inference traffic goes from your users and tools straight to the Loadcore Agent on your server. Prompts, completions and streaming responses never leave your network.
Control traffic goes from the agent to the Loadcore control plane: heartbeats with OS, GPU and memory, download progress, instance status and API key checks.

So your data stays on-premise, while you get a managed console without running your own management stack.

From bare GPU box to serving tokens

Register a server in the console and copy its one-time server key.
Start the agent with that key. It installs llama.cpp and starts sending heartbeats.
Pull a model from Hugging Face in the quantization that fits your VRAM.
Start an instance with your port, context size and GPU layers.
Hand out keys. Members create personal API keys and use any OpenAI-compatible tool.

For GPU servers in your own cloud account, the console gives you a startup script. Paste it into the provider's user data or cloud-init field and the server shows up in the console on first boot.

Built for a fleet, not a single box

Health checks on every running instance every 15 seconds.
Heartbeats every 30 seconds, so you see which servers are online, what they run and how much memory is free.
Per-model access through departments, enforced by the agent on every request.
Distributed instances that split one model across up to eight servers using llama.cpp RPC, so a 70B+ model fits on several 24 GB cards you already own.

Open formats, no lock-in

Loadcore runs GGUF models with llama.cpp, the most widely used open stack for local inference. Your weights live in a plain directory on your server, and the API is OpenAI-compatible. If you ever leave, your models and integrations keep working.

Read how Loadcore fits a GDPR-friendly AI setup, or compare costs with the LLM cost calculator.

Frequently asked questions

Is Loadcore fully on-premise?

Inference is. Models, prompts and completions run and stay on your own servers. The control plane that tracks servers, models, departments and keys is a managed service run by Loadcore, so the agent needs an outbound connection to it.

What network access does the agent need?

An outbound connection to the Loadcore control plane for heartbeats and key checks, which the agent caches for five minutes, and to Hugging Face when you download a model. Your users reach the agent inside your own network, and prompts never leave it.

Which servers are supported?

Linux servers with a GPU that llama.cpp supports. Your own racks, a colocation box or a GPU server rented in your own cloud account all work the same way.

How do we scale beyond one server?

Run the same model on several servers for more throughput, run different models on different servers, or split one large model across up to eight servers on your local network with a distributed instance.

Run your first model today.