How Loadcore secures the connection between agents, API and relay
A look at the security design behind Loadcore: signed commands, short-lived tokens, an outbound-only relay tunnel, hardened agents and signed updates.
Loadcore lets you run models on your own GPU servers while managing them from a hosted control plane. That raises a fair question for any security team: if a cloud service can start and stop processes on my servers, what stops someone else from doing the same? This article describes how the three parts of Loadcore talk to each other and which safeguards sit on each path.
The three parts
- The API (control plane). Hosted by Loadcore. It stores your organization, servers, models, access rules and usage figures. It never receives prompts, responses or model weights.
- The agent. Runs on each of your GPU servers. It downloads models, starts and stops llama.cpp instances and serves an OpenAI-compatible API to your users.
- The relay. Hosted by Loadcore. It gives the console and your tools a way to reach agents that sit behind a firewall or NAT, without opening inbound ports.
The guiding principle: no single part should be able to take over a server on its own, and every credential should be as narrow and short-lived as possible.
Commands from the API to an agent are signed
Every command the control plane sends to an agent, such as starting an instance or downloading a model, carries an Ed25519 signature. The signature covers:
- the server it is meant for
- the HTTP method, path and query
- a timestamp
- a random 16-byte nonce
- a SHA-256 hash of the request body
The agent rejects a command when the signature does not verify, when the timestamp is more than 60 seconds off, or when it has already seen the nonce. A captured command can therefore not be replayed, redirected to another server or altered in transit.
Each signature also starts with a fixed label for its purpose. A signature made for a command cannot be passed off as any other kind of token, and the reverse is also true.
The signing key lives only in the control plane. Agents and the relay only hold public keys. Keys carry an identifier, so a new key can be introduced while agents still accept the old one during the switch.
The relay can forward, but not command
The agent opens an outbound, TLS-encrypted WebSocket connection to the relay and keeps it open. Your server needs no inbound ports, which is often the difference between "approved" and "needs a firewall change request".
To open that tunnel, the agent presents a relay token signed by the control plane. The token is bound to one server and expires after 5 minutes, and the relay checks it with public keys only.
Because the relay has no signing key, it can pass traffic along but cannot create valid commands. Even someone with full control over the relay could not start a process on your server, because the agent would reject every unsigned or replayed command.
Further measures on the relay:
- Control-plane traffic to agents uses a separate internal route that requires its own secret.
- Repeated failed attempts from one address are blocked for 5 minutes.
- Only the Loadcore console origin is allowed for browser requests.
- The relay does not store request or response content.
User access to models
People reach models through the agent's OpenAI-compatible API, directly or through the relay. Two kinds of credentials are accepted.
Personal API keys (sk-lc-...) contain 256 random bits. The control plane only stores a SHA-256 hash of each key, so a database leak would not reveal usable keys. The agent checks a key with the control plane, caches the answer for 60 seconds and only allows the models the user's departments grant. Keys can be revoked one by one, and failed attempts from one address are rate limited.
Chat tokens let the console chat work without anyone pasting a key into the browser. When you open the chat, the control plane issues a token that:
- is signed with its own label
- is bound to a single server
- lists exactly the models you may use there
- expires after 10 minutes
The agent verifies chat tokens locally and refuses tokens for other servers or with a lifetime over 15 minutes.
Signing in to the console uses a hosted login with PKCE. Password logins directly through the API are disabled, and passwords never pass through the Loadcore API.
Server keys
Each agent authenticates to the control plane with its own server key, which is stored only as a hash. You can rotate a server key from the console at any time:
- Planned changes: the old key keeps working for 24 hours.
- Suspected leaks: you can revoke the old key immediately. This also closes the agent's relay tunnel.
Deleting a server instructs the agent to shut itself down.
Hardening on the server
On Linux the install script sets the agent up as a hardened systemd service:
- Dedicated user. The agent runs as its own unprivileged user, not as root.
- Locked-down service. It has no Linux capabilities, cannot gain new privileges and sees a read-only view of the system and no home directories.
- Protected model servers. Each llama.cpp instance listens only on the loopback interface and requires an instance-specific key. A firewall rule allows only the agent's user to open connections to those ports, so other users and processes on the machine cannot use the models directly.
- Docker. The Docker image runs the agent as a non-root user.
For models split across several servers, the servers talk to each other only over private network ranges, and every connection starts with an HMAC proof based on a secret unique to that instance.
Signed updates
Agents keep themselves up to date, which is convenient and also the most sensitive feature an agent has. Loadcore protects it as follows:
- A separate signing key. Releases are signed with a key used only for releases. It is kept in the build pipeline, apart from the control plane's command key.
- Built-in public key. The agent ships with the release public key built in. It only installs a release whose signature, version and platform match, and never moves to an older version.
- A second check as root. On Linux the unprivileged agent cannot replace its own binary. A small root service checks the signature again before installing, keeps the previous version, and rolls back automatically if the new agent does not report healthy within 3 minutes.
- Per-server control. You can switch automatic updates off per server and update manually when you choose.
What we do not see
The control plane holds the data it needs to manage your fleet: accounts, organization structure, server details reported by the agent, access rules, key metadata and usage counts. It never receives prompts, responses or model files. Requests that go through the relay are encrypted on the way to the relay and inside the agent's tunnel, and the relay does not store them.
If you prefer that no traffic leaves your network at all, connect your tools straight to the agent on your own network and skip the relay.
Reviewing it yourself
Security claims are only useful if you can check them. Test the behaviour on your own server: send a modified command, replay an old one, or try a key from another organization, and watch the agent refuse it. If your security team has questions about the design, contact us.
For deployment options, see the on-premise LLM page, the GDPR-compliant AI overview and the docs.