Skip to content

Governed Inference Doctrine

Inference is a model-portable kernel capability in service of personal AI continuity. Providers are implementation surfaces behind role policy; they do not own cognition, routing, authority, approval, execution, memory, identity, or source-of-truth status.

The purpose of model portability is not provider abstraction for its own sake. It keeps Luna's reasoning style, operator relationship, and continuity from being trapped inside one model, vendor, or local runtime.

Current operating posture is local-first for coding and implementation lanes, with frontier escalation through governance for interactive conversation and eligible inference roles. Frontier models are infrastructure, not identity. Rubick preserves posture continuity across model and provider changes by making prompt/profile posture, cognitive modes, model mode, provider path, and runtime abstraction inspectable and governable. Meepo protects transition continuity when model/provider changes are part of an approved state transition; it revalidates the transition instead of selecting or judging models itself.

Role Registry

Canonical inference roles are defined in app/inference/roles.py and mirrored for tooling in docs/machine/inference-roles.json.

Current canonical roles:

  • conversational_reply
  • oracle_synthesis
  • operational_narration
  • architecture_summary
  • coordination_analysis
  • route_explanation
  • draft_generation
  • bounded_reasoning
  • memory_fact_extraction
  • retrieval_query_embedding
  • architecture_evidence_embedding

Compatibility roles remain available for existing callers: operational_clarification, task_intent_classification, embedding_generation, and memory_fact_embedding.

Each role declares allowed providers, local/frontier eligibility, structured-output eligibility, cost class, fallback policy, replay metadata requirements, forbidden uses, and degradation behavior.

Provider Arbitration

app/inference/arbitration.py selects providers deterministically from role policy, provider readiness, model mode, local-first preference, structured-output requirements, cost policy, fallback eligibility, and degradation state.

The default provider set is:

  • local Ollama through luna-inference
  • OpenAI-compatible frontier provider, disabled unless governed metadata allows it
  • Anthropic-compatible frontier provider, disabled unless governed metadata allows it
  • disabled provider state for fail-closed surfaces

No live credential is required for readiness inspection or tests.

Underlord Escalation

Paid frontier inference is blocked unless explicit Underlord approval metadata or a bounded no-approval budget exists for the role cost class. Silent frontier fallback is forbidden. Open-ended spend is forbidden.

Replay And Observability

Provider arbitration emits deterministic lineage records with role, selected provider, fallback chain, policy decision, cost class, approval requirement, replay hash, degraded state, structured-output flag, advisory-only flag, and model usage lineage.

Lineage records do not store prompts, outputs, credentials, API keys, bearer tokens, or secrets.

Oracle consumes build_inference_readiness_surface() to answer which roles are available, which providers are degraded, whether frontier inference is allowed, which roles require Underlord approval, why providers were selected or denied, and what fallback path exists.

Migrated Runtime Surfaces

app/inference/client.py now performs provider arbitration before any luna-inference completion or embedding gateway call. Failed arbitration returns explicit degraded or fail-closed output and records lineage in the response audit metadata.

app/bot.py migrates the first live chat model-call surfaces by tagging legacy route labels with governed role intent:

  • Oracle questions use oracle_synthesis.
  • Operational readiness, provider, blocked, degraded, and status narration uses operational_narration.
  • Route questions that ask why or how routing happened use route_explanation.
  • Coordination questions use coordination_analysis.
  • Architecture, subsystem, topology, coupling, and component-boundary questions use architecture_summary.

The legacy dictionary contract of run_model_attempt() remains intact while the request payload sent to luna-inference carries the selected governed role. Oracle readiness reports expose these migrated role statuses and lineage hashes without making a model call.

Remaining Legacy Surfaces

The compatibility wrappers call_openwebui(), call_ollama(), and generic conversation fallback still exist for older callers. They route through governed inference but intentionally remain on broad compatibility roles until their callers can provide narrower intent safely.

Non-model HTTP calls to Keeper, luna-tools, Zeus, luna-router, governance interceptor, ops, and Naga publication adapters are outside Era 8 inference role migration. They remain governed by their existing provider or capability contracts rather than by model inference roles.

Safety Boundaries

Inference output remains advisory-only. It cannot approve execution, mutate state, bypass governance, override deterministic routing, expand provider authority, silently become source of truth, or transfer runtime authority.