[01] WHAT LAYA IS

Laya runs fully on a CPU-only VPS

Laya is an open source System 1 decision model. It answers typed questions in one forward pass. It never generates text, so it cannot invent an answer.

• "Type 1 style" means fast decisions, not long written reasoning.
• Every answer carries a calibrated probability from 0.0 to 1.0.
• It classifies, routes, scores, and moderates text in over 100 languages.
• The license is Apache-2.0, so commercial use is allowed.
• Release 0.4.22 was current in November 2026.
• The GitHub repository has about 35,000 stars.

↓  NEXT · PgDn

[02] HARDWARE CHECK

Plan 4 GB RAM, 8 GB disk, and 200 to 450 ms per decision

• No GPU is needed, because one decision is one forward pass.
• The server keeps two checkpoints in RAM by default.
• Set LAYA_MAX_LOADED=1 to keep only one resident.
• Reserve at least 8 GB of disk for the system, torch, and one checkpoint.
• CPU speed peaks near 16 worker threads; extra threads slow it down.
• The published timings came from a 56-core server, so a small VPS answers slower.
• A 2 vCPU, 4 GB VPS runs one checkpoint comfortably.

Timings are four-question CPU requests from the benchmarks and known limits page. Total process memory sits above the weights figure.

[03] INSTALL

Install laya with the CPU-only PyTorch wheel

$ python3 -m venv .venv
$ source .venv/bin/activate
$ pip install torch --index-url https://download.pytorch.org/whl/cpu
$ pip install "laya[serve]"
$ python -c "import laya; print(laya.__version__)"

• Laya needs Python 3.10 or newer.
• The CPU wheel index skips the CUDA download.
• The serve extra installs laya-serve and its API stack.
• The last command prints the installed version.

$ uv pip install --torch-backend=cpu "laya[serve]"

Platform-specific steps live in the installation details section of the README.

[04] MODEL DOWNLOAD · QUANTIZATION

Use fp32 weights first; the Q8 GGUF cuts disk to 449 MB

• The router downloads a checkpoint on first use.
• Downloads land in ~/.cache/huggingface on Linux.
• Pick the multilingual checkpoint on a CPU VPS; it is smaller and twice as fast.
• Keep fp32 weights first, because quantization shifts calibrated probabilities.
• ggml-org/Laya-GGUF publishes BF16 (844 MB) and Q8_0 (449 MB) files.
• llama.cpp serves GGUF on the same /v1/systemone protocol.
• The laya[onnx] extra is a lighter CPU runtime option.

$ hf download convaiinnovations/laya-multilingual

$ llama serve -hf ggml-org/Laya-GGUF   # from the official GGUF card

[05] SERVE

laya-serve answers on port 8000 behind a bearer key

$ LAYA_DEVICE=cpu LAYA_MODELS=multilingual LAYA_MAX_LOADED=1 \
  LAYA_THREADS=4 LAYA_API_KEY=pick-a-long-random-key \
  laya-serve

• It exposes POST /v1/systemone and GET /health.
• It preloads its checkpoints at startup by default.
• LAYA_THREADS should stay at or below the physical core count.
• LAYA_API_KEY enables bearer checks and hides the docs pages.
• The default bind address is 0.0.0.0.
• Set LAYA_HOST=127.0.0.1 behind a reverse proxy for public traffic.
• Run the server under systemd for a long-lived service.

Every environment variable is documented on the HTTP API guide.

[06] VERIFY

A health probe and one request prove the install

$ curl -s http://127.0.0.1:8000/health

$ curl -s http://127.0.0.1:8000/v1/systemone \
  -H "content-type: application/json" \
  -H "authorization: Bearer pick-a-long-random-key" \
  -d '{
    "state": "I was charged twice this month. I want my money back.",
    "questions": {
      "queue": {
        "type": "choice",
        "instructions": "Which team should reply?",
        "criteria": {
          "billing": "billing and refunds",
          "tech": "login and app issues",
          "other": "everything else"
        }
      }
    }
  }'

{
  "answers": { "queue": { "choice": "billing",
    "probabilities": { "billing": 0.95, "tech": 0.03, "other": 0.02 } } },
  "routing": { "model": "multilingual" } }

• The health endpoint should answer with JSON.
• The queue answer should say billing.
• The documentation example scored billing at 0.9519.
• The routing block names the checkpoint that answered.

$ python -c "from laya import Router; print(Router().predict(\
'Refund my duplicate charge today.', {'refund': {'type': 'noul', \
'instructions': 'Does the user request a refund?'}})['answers']['refund']['noul'])"

The Router path prints a probability without the HTTP server.

[07] DOCUMENTATION

These official links cover setup, serving, and limits

CHECKED NOVEMBER 2026 · ALL LINKS RETURN HTTP 200 · FACTS FROM THE SOURCES ABOVE