Self-Hosted LLM Platform & AI-in-SDLC Auto-Remediation
Open LLMs on bare metal, a multi-provider AI router, and an auto-remediation pipeline that detects, diagnoses, and ships its own fixes behind a QA gate.
The problem. Sending internal code and logs to a third-party API is a non-starter for some workloads, and paying per-token for high-volume internal tooling doesn't scale. Separately, the slowest part of an incident is rarely the fix itself — it's the human round-trip from "something's wrong" to "here's the diagnosis" to "here's a PR."
The approach. I run open models (DeepSeek, Qwen) self-hosted on the on-prem bare-metal cluster via Ollama — powering an internal chatbot and a Copilot-style coding assistant with grounded RAG over pgvector. A custom AI router sits in front of everything, handling prompt caching, model routing, vision, and abort-signal cancellation across Anthropic Claude, Azure OpenAI, and Azure Speech. On top of that runs an AI-in-SDLC auto-remediation pipeline: an LLM reads the error from logs, diagnoses the root cause, generates a fix, opens a PR, and lets automated QA gate the release.
The outcome. Sensitive workloads never leave the data center, high-volume internal AI runs at zero marginal token cost, and the detect→PR loop measurably reduced MTTR — humans review the fix instead of authoring it from scratch.
An LLM watches the log/alert stream and classifies real errors out of the noise — the entry point of the remediation loop.
Grounded against the codebase + prior incidents via RAG, the model proposes a root-cause hypothesis rather than just restating the symptom.
The model produces a concrete patch scoped to the diagnosed cause — a diff, not a suggestion.
The fix lands as a pull request with the diagnosis attached, so a human reviews an explained change instead of a raw alert.
Automated QA must pass before the change is eligible to release — the safety valve that makes auto-generated fixes trustworthy.
Gated changes release through the normal pipeline. The loop turns incident time from author-from-scratch into review-and-approve.
- Self-hosted firstOpen models on-prem via Ollama for anything sensitive or high-volume — data stays in the data center, no per-token bill.
- One router, many providersA single AI router abstracts Claude, Azure OpenAI, and Azure Speech — prompt caching, routing, and abort-signal handling live in one place.
- Grounded, not guessingRAG over pgvector so assistants answer from the actual codebase and docs instead of hallucinating.
- Auto-remediation, gatedLLMs open PRs, but automated QA — not the model — decides what ships. Human reviews an explained diff, not a raw alert.