OMEGA Core started with one API. It now has two. The decision to split them is a small one in implementation terms — a new directory, a new compose stack, another port — but it surfaced a question that I think matters a great deal when you’re building AI platforms seriously: what is this service actually for?
The first API
The Evidence API was built to answer one question: is the platform healthy, and can I prove it?
That drove every decision about it. It is read-only. It collects live state from Docker, PostgreSQL, Ollama and the host system. It runs background loops that write time-series data to InfluxDB. It exposes endpoints for service health, model inventory, system metrics, database state. It is an observer. It does not change anything. It does not instruct anything. It watches, collects, and reports.
That constraint is not accidental. When you are building an AI platform and your primary concern is security and governance, having an authoritative read-only evidence layer — something that tells you the actual state of the system without touching it — is foundational. You cannot audit what you cannot observe. The Evidence API is the thing that makes the platform auditable.
The obvious next question
Once you have a working AI runtime with ten models installed and an Evidence API that can enumerate them, the obvious question arrives: can we send a prompt to one of them from outside the platform?
You could answer that question by adding a /query endpoint to the Evidence API. It already talks to Ollama. The dependency is there. The implementation would be straightforward.
But a /query endpoint does not belong in the Evidence API.
The Evidence API is a read-only observer. Adding an endpoint that sends prompts to a language model and returns responses would change what the service is. It would no longer be observing the platform. It would be actively using it. The conceptual boundary — the thing that makes the Evidence API trustworthy as an audit layer — would dissolve.
More practically: evidence collection and AI inference have different characteristics. Evidence endpoints are fast, lightweight, side-effect-free. Inference is slow, stateful in a session sense, and carries risk surfaces that read-only state queries do not. Authentication requirements differ. Timeout profiles differ. The GAIPS Domain 3 security assessment for these two services will look completely different. Conflating them into one service would make both harder to reason about and harder to secure.
The right answer was a second service.
Naming the first thing properly
Once the inference service existed, a problem appeared: the original API was named omegacore-api.
That name made sense when there was only one API. Once there are two, it is ambiguous. Which one is “the API”? A name that means everything means nothing.
So before building the inference service, the Evidence API was renamed. Container, Python package, compose stack, directory — all of it changed from omegacore-api and api/ to omegacore-evidence and evidence/. The new directory structure puts both services under api/ at the repository root: api/evidence/ and api/inference/.
This is the kind of rename that is cheap to do early and expensive to do late. At two services and a few months of development, the change was a commit. At ten services and a year of accumulated tooling, scripts, monitoring rules and documentation, it becomes a project.
The inference service
The Inference API is a FastAPI application running at port 8001, bound to the LAN interface and connected to omega_internal. It exposes three endpoints:
POST /query— accepts a model name and a prompt, returns the model response and durationGET /models— lists available Ollama models so callers can discover what is installedGET /healthandGET /versionfor standard operational checks
It communicates with omegacore-ollama via the shared internal Docker network using container name resolution. No static IPs. No direct port exposure to the model runtime.
Model selection is explicit. You do not send a prompt and get whatever the default is. You name the model you want. This matters for a platform with multiple specialist role models — the difference between sending a security prompt to omega-threat-analyst and sending it to llama3.2:1b is significant, and that choice should be intentional.
What this enables
There is an engineering detail about this that I find interesting enough to state directly.
Claude Code — the AI assistant I use to build and maintain OMEGA Core — can now query the platform’s own models via the Inference API. It has visibility of the installed model inventory through /models, and can send prompts to specific models and observe their responses as part of engineering work.
This is not a trivial capability in the context of what OMEGA Core is for. The GAIPS Domain 1 lab work involves probing models for hallucination and overconfidence, mapping attack surfaces, comparing responses across models for the same adversarial prompt. I can now do that work collaboratively — with Claude querying omega-threat-analyst and llama3.2:1b against the same prompt and returning both responses as evidence.
The contrast is already illustrative. Ask llama3.2:1b to define prompt injection in one sentence and it tells you it is a variant of SQL injection. Ask omega-threat-analyst the same question and it gives you an accurate definition with a MITRE ATLAS reference. Same question. Same infrastructure. Different model. Very different output.
That difference is evidence. The Inference API is how you collect it programmatically rather than manually.
Lesson learned: model confidence is not model accuracy
The llama3.2:1b response to the prompt injection question is worth dwelling on.
The model said, with no hesitation, that prompt injection is a variant of SQL injection — an attack where an attacker injects malicious input into a database query to manipulate or modify data. That is a definition of SQL injection. It has nothing to do with prompt injection. The model was wrong, it was confident, and nothing in the response indicated any uncertainty.
This is hallucination on a security topic. And it is the most dangerous kind — not a fabricated citation or an invented statistic, but a plausible-sounding technical answer that maps the right terminology onto the wrong concept. Someone without prior knowledge of prompt injection would have no reason to doubt it.
The lesson for anyone deploying AI in a security context is straightforward: a model’s confidence in its response tells you nothing about the accuracy of that response. Small models in particular will pattern-match on terminology and produce answers that sound authoritative and are factually incorrect. On general knowledge topics that is an inconvenience. On security topics it is a liability.
Model selection matters. omega-threat-analyst uses a larger base model and a system prompt that anchors it to the MITRE ATLAS framework. It got the same question right, with a precise definition and a correct framework reference. That is not a coincidence — it is the result of choosing the right base model for the task and conditioning it appropriately.
The Inference API makes explicit model selection mandatory. You cannot send a prompt without naming the model. That constraint exists precisely because the choice of model is a security decision, not a preference.
The broader principle
The architectural decision here is not complicated. Read-only observation and active inference are different concerns. They should be different services. Name your services after what they actually do. Make the separation early, while the cost is low.
These are not novel ideas. They are standard separation-of-concerns reasoning applied to an AI platform context. What I find useful about having done it on a real running system — rather than discussing it abstractly — is that the reasoning becomes concrete. You can see exactly where the boundary is and why it matters. You can test it. You can assess the security posture of each service independently.
That is what OMEGA Core is for. Not accumulating technology. Understanding it — at the level where you can explain why each decision was made, what it protects against, and what it leaves open.
The inference service opens new attack surface. The next step is to characterise it.