Inference Adapter

The Inference Adapter is the service that runs Enprivacy’s local machine learning models: document embedding, search reranking, and named-entity detection. The Web and Job services send it every such request over a private TLS connection. It serves encoder models only; the system LLM is a separate service.

It is mandatory. The Web and Job services refuse to start without an adapter endpoint and key, and there is no in-process fallback.

Image	ghcr.io/enprivacyhq/enprivacy-inference-adapter, pinned by @sha256: digest
Listener	HTTPS on port 8443, private network only
Models	Baked into the image — nothing is downloaded at runtime
Callers	Web and Job, authenticated with an API key

Before you start

You need three things, all generated once per deployment:

1. An API key pair — one half for the adapter, one for the platform.
2. A TLS certificate and private key for the adapter’s listener.
3. The image digest for the release, from Enprivacy.

The API key pair

Mint it with the adapter’s own image. It needs no network and no configuration:

docker run --rm ghcr.io/enprivacyhq/enprivacy-inference-adapter@sha256:<digest> \
  keygen --name platform

It prints one line of JSON with three fields:

{"name": "platform", "key": "platform.<secret>", "entry": "platform.<sha256>"}

Field	Goes to	As
key	The platform (Web and Job)	EP_INFERENCE_API_KEY
entry	The adapter	EP_INF_API_KEYS

• Keep both halves from the same run. The adapter stores only the hash in entry; a key from a different run never matches, and every call answers 401. Nothing can recover a lost key from entry — mint a new pair.
• Treat key as a secret. entry is a hash, but keep it out of source control as well.
• --name identifies the caller in the adapter’s logs. Letters, digits, hyphens and underscores, up to 64 characters; no dots.

The TLS certificate

The adapter refuses to listen on a routable address without a certificate. The platform is its only client and trusts this exact certificate, so a self-signed one is the expected shape — no public CA is involved.

The certificate’s subject alternative names must include the hostname in EP_INFERENCE_URL, or the platform refuses the connection. For a Docker Compose service named inference-adapter:

openssl req -x509 -newkey rsa:2048 -nodes -days 825 \
  -keyout inference.key -out inference.crt \
  -subj "/CN=inference-adapter" \
  -addext "subjectAltName=DNS:inference-adapter"

• The adapter mounts both files; the key must be readable by its user (UID 10001).
• Web and Job mount only the certificate, as their CA — never the key.
• On Kubernetes, use the Service’s DNS names.
• On Azure, skip this step. The Azure Terraform bundle has Key Vault generate the certificate, for the adapter’s app name, when it applies the app module. See Deploy on Azure with Terraform.

Adapter settings

Variable	Default	Set to
EP_INF_LISTEN	127.0.0.1:8443	0.0.0.0:8443 in a container, so other services can reach it
EP_INF_TLS_CERT / EP_INF_TLS_KEY	—	Paths to the mounted certificate and key. Required on any non-loopback bind
EP_INF_API_KEYS	—	The entry value. Several entries may be listed, comma- or newline-separated, one per caller name
EP_INF_API_KEYS_FILE	—	Alternative to EP_INF_API_KEYS: a file of entries. Set one or the other
EP_INF_MODELS_CONFIG	the image’s registry	Leave unset unless Enprivacy supplies a registry
EP_INF_AIRGAP	true	Leave on. The models are in the image
EP_INF_CPU_BUDGET	read from the container’s CPU limit	Override only if the limit cannot be read
EP_INF_MAX_QUEUE_WAIT_MS	30 000	How long a request may wait for capacity before a 503
EP_INF_LOG_LEVEL	info	debug for troubleshooting

Give the container a CPU limit. The adapter sizes its thread pools from the limit it reads at boot, and logs the number; without one it sizes for the whole host. The Azure reference runs 4 vCPU / 8 GiB, which holds the embedder and rerankers resident with headroom.

Platform settings

On both the Web and Job services:

Variable	Default	Set to
EP_INFERENCE_URL	—	https://<adapter host>:8443 — the hostname must be in the certificate
EP_INFERENCE_API_KEY	—	The key value
EP_INFERENCE_CA_FILE	—	Path to the mounted adapter certificate (a PEM bundle is accepted)
EP_INFERENCE_VERIFY_TLS	true	Leave on. Disable only against a loopback development listener
EP_INFERENCE_OPEN_TIMEOUT / EP_INFERENCE_READ_TIMEOUT	5 / 120 s	Raise the read timeout only if large rerank requests time out
EP_INFERENCE_EMBEDDER_ALIAS / EP_INFERENCE_RERANKER_ALIAS	default-embedder / default-reranker	Leave unset. They name models in the adapter’s registry

Docker Compose example

services:
  inference-adapter:
    image: ghcr.io/enprivacyhq/enprivacy-inference-adapter@sha256:<digest>
    environment:
      EP_INF_LISTEN: "0.0.0.0:8443"
      EP_INF_TLS_CERT: /tls/tls.crt
      EP_INF_TLS_KEY: /tls/tls.key
      EP_INF_API_KEYS: ${INFERENCE_ADAPTER_ENTRY}
    volumes:
      - ./inference-tls:/tls:ro
    cpus: 4
    mem_limit: 8g

  web:
    environment:
      EP_INFERENCE_URL: https://inference-adapter:8443
      EP_INFERENCE_API_KEY: ${INFERENCE_ADAPTER_KEY}
      EP_INFERENCE_CA_FILE: /inference-tls/tls.crt
    volumes:
      - ./inference-tls/tls.crt:/inference-tls/tls.crt:ro
    depends_on:
      inference-adapter:
        condition: service_healthy
  # jobs: the same three variables and mount as web

The image carries a health check; service_healthy waits for it.

Start-up and health

• /ready turns healthy once the adapter has loaded its always-resident models — the embedder and rerankers, about 3 GB from the image. Allow several minutes on a cold node.

• /health and /ready are the only unauthenticated paths. Everything else, including /openapi.json, needs the key.

• Start the adapter before Web and Job, and let it become ready. They refuse to boot without the endpoint configured, and search and detection fail while it is unreachable.

• Confirm every model loads with selftest, which loads each model in the registry and prints what it measured. /ready covers only the always-resident ones:

  docker compose exec inference-adapter inference-adapter selftest

• The boot log names its CPU budget, for example 4 core(s) from cgroup v2 cpu.max. It should match the container’s limit.

From a Web container, this one call exercises DNS, TLS, the certificate’s names and the key together:

curl --cacert "$EP_INFERENCE_CA_FILE" \
  -H "Authorization: Bearer $EP_INFERENCE_API_KEY" \
  "$EP_INFERENCE_URL/v1/models"

Symptom	Cause
Every call 401	key and entry from different keygen runs, or a truncated value
TLS or hostname error from Web/Job	The hostname in EP_INFERENCE_URL is not in the certificate, or EP_INFERENCE_CA_FILE points at a different certificate
Adapter exits at boot: TLS required	EP_INF_TLS_CERT / EP_INF_TLS_KEY unset on a non-loopback EP_INF_LISTEN
Adapter exits at boot: unreadable file	The key is not readable by UID 10001, or the volume did not mount
Web/Job exit at boot: inference requires	EP_INFERENCE_URL or EP_INFERENCE_API_KEY unset

Rotation

Both can be rotated with no outage.

The API key. The adapter accepts several named keys at once:

1. Mint a new pair with a new name (--name platform2).
2. Set the adapter’s EP_INF_API_KEYS to both entries, comma-separated, and restart it.
3. Set the platform’s EP_INFERENCE_API_KEY to the new key and restart Web and Job.
4. Remove the old entry from the adapter and restart it.

The certificate. EP_INFERENCE_CA_FILE accepts a bundle:

1. Generate the new certificate and key.
2. Give Web and Job a CA file holding both the old and new certificates, and restart them.
3. Switch the adapter to the new certificate and key, and restart it.
4. Reduce the CA file to the new certificate, and restart Web and Job.

Upgrading

Pin the new release’s digest and restart the adapter. Its models come with the image, so nothing needs staging, and each model’s weights are verified against a pinned digest as it loads.

Changing the detection model changes which personal information is found. Enprivacy treats a model change as a release of its own, not a dependency bump.