milkinfrastructure.com/docs

Start with Milk.

Milk Parlor is the gateway: it forwards model calls and saves eligible requests and answers. Milk Man is the agent: give it a goal, and it uses commands and cloud jobs to operate models, measure results, and improve them using your data.

Two programs

Milk Parlor

The Rust gateway. It authenticates, forwards, streams, and saves selected complete request + response pairs.

Milk Man

The agent runs from Bash, with an optional local dashboard. Jobs select models, storage, and cloud resources through environment variables.

Connect an app

There is no Milk SDK. Use the official OpenAI package.

  1. Install the official OpenAI package.
pip install openai
# or: npm install openai
  1. Point it at Milk Parlor with your operator-issued Milk key.
export OPENAI_BASE_URL=https://parlor.milkinfrastructure.com/v1
export OPENAI_API_KEY='your Milk key'
  1. Keep the application call unchanged.
from openai import OpenAI

client = OpenAI()
answer = client.responses.create(model="your-model", input="Hello")
print(answer.output_text)

Milk supports POST /v1/responses and POST /v1/chat/completions, including streaming. It does not claim the rest of the OpenAI API.

Which key goes where?

Your app uses the Milk key issued by the gateway operator. The operator configures the actual model-provider key on Parlor. When you run Milk Man yourself, its jobs use the provider and storage credentials you configure locally. Keep secret values out of chat, screenshots, and Git.

What happens

your app Milk Parlor your model object store Milk Man
1. Forward
Parlor sends the request to the configured model and streams the answer back.
2. Save
After completion, Parlor writes both sides to local or S3-compatible storage without holding up the answer.
3. Process
Milk Man counts saved exchanges, groups related tasks, and saves summary checkpoints before making training examples.
4. Propose
Milk Man may prepare a traffic rule for a new model. A person must approve and sign it.

Develop locally

Clone both repositories. Milk Parlor needs Rust. Milk Man needs Bash, Python 3, Git, and curl. Processing captured records also needs zstd.

git clone https://github.com/milkinfrastructure/milk-parlor.git
git clone https://github.com/milkinfrastructure/milk-man.git

Run the gateway

Set its Milk key list, model URLs and keys, route public key, and storage values. The gateway README contains one complete local setup.

cd milk-parlor
cargo run --locked

open the gateway setup

Run Milk Man

cd milk-man
export LLM_API_URL=https://parlor.milkinfrastructure.com/v1/responses
export LLM_API_MODE=responses
export LLM_MODEL=gpt-6-astra
export LLM_API_KEY='your Milk key'

bin/man run --workspace milk-man="$PWD" -- \
  "Read the goal and report the next unfinished item."

This creates a saved session and keeps its heartbeat running. For a different OpenAI-compatible provider, change the URL, model, API format, and key. The URL includes the complete endpoint path.

In a second terminal, use the same environment and start the dashboard:

cd milk-man
bin/man dashboard
# open http://127.0.0.1:8765

Chat resumes that session. Closing the page does not stop Milk Man. Idle heartbeat checks make no model calls; a new instruction or registered change can resume work.

What is happening now?

Chat shows replies and commands. The heartbeat shows working, waiting, or offline. A lost dashboard connection does not prove the task stopped.

What has been saved?

Data shows summaries. Experiments shows results. Tools shows required environment names, not secret values or proof that a provider is running.

Use MILK_DASHBOARD_PORT to change the local port and MILK_PARLOR_BASE_URL for the gateway health check. Restart after changing the environment.

Run a job without chat

export MILK_SCOPE_ID=11111111-1111-4111-8111-111111111111
export MILK_STORE_KIND=local
export MILK_STORE_ROOT="$PWD/.milk-objects"

bin/milk status
bin/milk operate --once

This local example has no captured traffic yet. Point both programs at the same scope and storage to process real exchanges. operate --once runs ready data jobs and exits; it is separate from the agent's persistent heartbeat.

open the complete job setup

Saved data

Milk gives each customer or test run one unique ID. Folder-like object storage, such as Cloudflare R2 or Amazon S3, keeps the full history under that ID:

milk/v2/scopes/<scope-uuid>/
├── c/           request–response exchanges
├── s/           summary checkpoints
├── readiness/   the next allowed step
├── e/           model-test cases
├── d/           training and evaluation splits
├── m/           model records
├── v/           comparison results
├── p/           unsigned route proposals
└── research/    objective and experiment history

Saved records are never changed. A small current.json points to the latest version. Model weights stay outside Git and container images.

Read next