Skip to content
CryptoDecentral

Pillar · DeAI

Decentralized AI

Open weights, local inference, and compute that is not locked behind a single API key — a field guide to patterns, not a token shopping list.

Start with residency, not vibes

Most people meet AI as a hosted chat box. That is a data-residency decision: prompts leave your building under someone else’s policy. Local and peer-to-peer patterns exist so you can choose differently. Essay-length framing: Local and Peer-to-Peer AI: Data Residency Without the Hype.

Threat models worth writing down

Before choosing a runtime, name who can see what — and what breaks when they do:

  • Provider logger — SaaS APIs that retain prompts, tool traces, or uploaded files under a policy you did not write.
  • Jurisdiction hop — data leaving the country (or the building) even when marketing says “enterprise secure.”
  • Swarm peer — collaborative inference where other machines see activations or intermediate state on a public swarm.
  • Supply-chain impostor — fake weight archives, trojaned runners, or lookalike hub uploads that pass a casual download.
  • Endpoint compromise — malware on the laptop that hosts your “local” model. Residency does not invent a secure OS.
  • Policy / continuity shock — sudden account bans, region blocks, model ID sunsets, or quota cliffs that brick a product built only against one API.

Match the stack to that list. Local open weights answer provider and many jurisdiction concerns; they do not answer endpoint malware. Public P2P answers hardware fit; it does not answer swarm confidentiality.

Layer 1 — Open weights

Weights you can download, license-check, and hash-pin. Open is not automatically safe, uncensored, or well-documented. Read the model card: license, intended use, training notes, evaluation claims, and whether the upload is an official org or a random fork. Prefer checksums and signed releases when offered.

“Open weights” means you can inspect and re-host under the license — not that a full audit already happened.

Layer 2 — Local inference

Runtimes such as llama.cpp and friendlier wrappers like Ollama load weights onto hardware you control. Prompts need not take a mandatory cloud round trip — unless the UI silently phones home. On first run, watch outbound connections. Telemetry, browser companions, and retrieval plugins can undo residency.

Local is not automatically private or correct. Measure tokens/sec on your machine; evaluate on your documents and languages. Practical checklist: Local AI starter.

Layer 3 — Collaborative and peer compute

When models exceed a single consumer GPU, research systems explore splitting work across machines. Petals-style swarms (see arXiv:2209.01188) illustrate collaborative inference with different trust assumptions than a hyperscaler API. You inherit questions about who sees activations, availability when peers leave, and prompt sensitivity mid-network.

A quieter middle path: run a private org server. Laptops talk to hardware you administer. Prompts may leave the laptop but not the organization. Do not collapse “private server,” “laptop local,” and “public swarm” into one marketing word.

Tokenized GPU markets exist. We discuss architecture, not tickers. No “top DeAI coins.”

Selection heuristic

  1. Sensitivity — Would a prompt leak harm a person, client, or investigation?
  2. Sovereignty — Do law, contract, or politics require data to stay in-country or on-prem?
  3. Capability — Does the task need a frontier model that will not fit locally?
  4. Continuity — Must this keep working offline or under platform pressure?
  5. Retrieval trust — If you pull documents live, who hosts the index and what leaves the room?
  6. License fit — Can you redistribute outputs or fine-tunes the way your org needs?

If sensitivity or sovereignty dominate, start local (or a tightly controlled private deployment). If capability dominates and data is low-sensitivity, cloud APIs remain rational. If continuity dominates, keep a local fallback even when you use the cloud day-to-day. A smaller offline model under a clear license can beat a frontier API for the workflows that matter.

Failure modes (field notes)

  • Quantisation that “fits” but quietly destroys accuracy on your domain language or documents — always eval on your corpus.
  • Telemetry or companion apps that undo residency while the chat UI still says “local.”
  • Pinning nothing: silent runner upgrades that change behaviour under a product you already shipped.
  • Treating model cards as security guarantees instead of structured publisher claims to check against hashes and your own tests.

Agent trust

Agents that browse, fetch feeds, or wield credentials amplify mistakes. If an agent can spend or sign, treat it like a hot wallet: small blast radius, clear permissions. CryptoDecentral publishes agent etiquette and feeds so machines can cite this hub without inventing custody or advice APIs. Pair with wallets and Nostr when identity or tips enter the picture.

Verify habits

  • Checksum or signature-verify runners and weight archives.
  • Read Hugging Face (or other hub) model cards before trusting uploads.
  • Pin versions you evaluated; upgrades are behavior changes.
  • Apply UI phishing discipline from verifying frontends whenever money or keys appear beside a model chat.

What we refuse to do

No token shopping lists. No custody of GPU credits. No swap UI. Directory listings for education and runners stay educational; sponsorships are labelled when present.

Related