How to Give Your Team a Private ChatGPT

If you want to give your team a private ChatGPT, you need four things: a model, a GPU server, a place to chat and a way to control access. This guide shows the setup and the decisions your IT and data protection teams will face along the way. For the broader product comparison, see the self-hosted ChatGPT alternative page.

Why teams want a private version

Employees are already using AI, often through personal accounts. Each time someone pastes a contract, a customer email or source code into a public tool, that data leaves your control. Banning the tools rarely works. Offering a sanctioned alternative does.

A private deployment means prompts and completions are processed on a GPU server you choose, in the location you choose. That matters to your DPO (see GDPR-compliant AI) and to regulated teams such as law firms and healthcare.

The architecture in plain terms

Component	What it does	Where it runs
Loadcore Agent	Runs and supervises llama.cpp, serves an OpenAI-compatible API on port 9090	Your Linux GPU server
Loadcore Console	Web app for servers, models, members, keys and chat	console.loadcore.io
Loadcore API	Managed control plane for orgs, roles, departments and key validation	Loadcore

Prompts travel from the user's tool straight to your server. The control plane receives only heartbeats, download progress, instance status and key checks. It never proxies inference and never stores prompts or completions.

Step 1: Choose a model

Match the model to your hardware and your team's work. Examples from the model library:

• Gemma 4 31B: general chat and image input, fits one 48 GB GPU
• Qwen3.8 27B: 17.6 GB at Q4
• gpt-oss 120B: 63.4 GB in MXFP4, fits one 80 GB GPU
• Qwen3 Coder Next: 48.5 GB at Q4, for developers

A mid-sized general model is a good starting point for a pilot.

Step 2: Stand up the server

Rent a GPU server from a provider and region you trust, or use your own. In the Console, copy the startup script into your cloud server's user data or cloud-init field. On first boot the server installs the agent as a system service. On your own machine, run:

./loadcore-agent --api-key="lc_..."

Then search for a model in the Console, download it to the server and start an instance with the context size and GPU layers you need.

Step 3: Set up people and departments

1. Create your organization and invite members. Sign-in works with password, Google or Apple.
2. Assign admin or member roles.
3. Create departments, such as Legal, Engineering and Support.
4. Grant each department the models it should use.

Now Legal can have one model and Engineering another, and nobody sees what they are not granted.

Step 4: Give people a way to chat

You have two options, and you can use both:

• In the Console. Members chat in the browser with the models they are permitted to use. No installation.
• Existing tools. Any OpenAI-compatible client works, including the OpenAI SDKs, LangChain, opencode, aider and Continue. Set the base URL and key:

base_url = http://your-server:9090/v1
api_key  = sk-lc-your-key
model    = your-model.gguf

Each person creates a personal sk-lc- key and can see their own usage statistics. Keys can be revoked individually.

Step 5: Roll out in stages

A rollout that works in practice:

1. Pilot (2 to 4 weeks). One department, 10 to 20 people, one model.
2. Collect feedback. Which tasks work, which do not. Compare against your current hosted tool, as described in Can Open-Source AI Replace ChatGPT in Your Company?.
3. Add models. Add a coding model for developers or a larger model for specific teams.
4. Expand by department. Grant access as each team is ready.
5. Add capacity. When a model gets busy, add a server and run the same model as another instance.

What to tell employees

Tell them what they gain: an assistant they may use with real company documents, without worrying about where the data goes. It handles everyday drafting, summarizing, translation and Q&A, and developers can plug the same models into their own tools. Make your policy clear on which tools are approved for which data.

What to tell your DPO

• Prompts and completions are processed on the servers you run and are not sent to Loadcore or a model provider.
• The control plane stores account data, organization structure, server details and hashed API key metadata.
• You choose the server location.
• Compliance still depends on your own purposes, legal bases and retention. No tool makes you compliant by itself.

Grow with demand

When the assistant becomes business critical, run the same model on a second server for extra capacity and a spare. Read How to Manage a GPU Fleet for LLM Inference for operations and How Much Does It Cost to Self-Host an LLM for 200 Employees? for budgeting. Setup details are in the docs, and the on-premise LLM page covers deployment options.