</style></head><bodydata-astro-cid-5hce7sga><navdata-astro-cid-5hce7sga><aclass="brand"href="/"data-astro-cid-5hce7sga><imgsrc="/logo-white.svg"alt="dd0c.net"width="40"height="40"data-astro-cid-5hce7sga></a><ahref="/"data-astro-cid-5hce7sga>Home</a><ahref="/about"data-astro-cid-5hce7sga>About</a><ahref="/services"data-astro-cid-5hce7sga>Services</a><divclass="spacer"data-astro-cid-5hce7sga></div><aclass="external"href="https://github.com/ddoc"target="_blank"rel="noopener"data-astro-cid-5hce7sga>GitHub</a></nav><maindata-astro-cid-5hce7sga><aclass="back"href="/"data-astro-cid-gjtny2mx>← All posts</a><h1data-astro-cid-gjtny2mx>Self-Hosted AI: Running LLMs, TTS, and Whisper on Consumer Hardware</h1><pclass="post-meta"data-astro-cid-gjtny2mx>2026-03-24</p><divclass="post-body"data-astro-cid-gjtny2mx><p>I wanted a personal AI assistant that I actually controlled. Not a SaaS product with a monthly bill and a privacy policy I’d never read. Something running on my own hardware, connected to my own accounts, with no vendor in the middle deciding what it could or couldn’t do.</p>
<p>I didn’t buy new hardware for this. I used what I had: a TrueNAS server in my home lab and an M1 Pro MacBook. Not a data center. Just the stuff already sitting on my desk.</p>
<li><strong>LLM</strong>: Claude via a local reverse proxy that routes requests through my own Anthropic API key. No vendor lock-in, no shared rate limits, full visibility into what’s being sent.</li>
<li><strong>TTS</strong>: <ahref="https://github.com/rhasspy/piper">Piper TTS</a> running on TrueNAS via the <ahref="https://github.com/rhasspy/wyoming">Wyoming protocol</a>. Also <ahref="https://huggingface.co/Qwen/Qwen3-TTS">Qwen3-TTS</a> on the Mac for voice cloning.</li>
<li><strong>STT</strong>: <ahref="https://github.com/openai/whisper">Whisper</a> server on TrueNAS, exposed as an OpenAI-compatible <code>/v1/audio/transcriptions</code> endpoint.</li>
<p>The whole thing runs on hardware I already owned. The only recurring cost is the LLM API.</p>
<h2id="getting-it-running">Getting It Running</h2>
<p>Piper and Whisper both run in Docker on TrueNAS. Piper is straightforward — pick a voice model, point it at the Wyoming port, done. Whisper took a bit more tuning. I’m running <code>small.en</code> which is a good balance of speed and accuracy for English transcription.</p>
<p>The agent framework is what ties it together. It handles the conversation loop, routes messages to the right tools, and manages the Telegram integration. Skills are just directories with a <code>SKILL.md</code> and some scripts — easy to add, easy to audit.</p>
<p>The LLM proxy sits between the agent and the Anthropic API. It adds request logging, lets me swap models without touching agent config, and gives me a single place to manage the API key.</p>
<p>Qwen3-TTS on the Mac runs via <ahref="https://github.com/Blaizzy/mlx-audio">mlx-audio</a>, which uses Apple Silicon’s Neural Engine. The voice cloning feature is genuinely impressive — give it a 3-second audio sample and it’ll match the voice reasonably well.</p>
<h2id="what-surprised-me">What Surprised Me</h2>
<p><strong>Piper is fast.</strong> I expected local TTS to be slow and robotic. Piper is neither. On TrueNAS (not a powerful machine), it generates speech at realtime speed or better. The voice quality isn’t Eleven Labs, but it’s completely usable for a personal assistant.</p>
<p><strong>Whisper <code>small.en</code> is accurate enough.</strong> I was skeptical about running a smaller model, but it handles my voice well — probably 95% accuracy on normal speech. The only failures are proper nouns and technical jargon, which is expected. For a personal assistant that mostly hears the same vocabulary over and over, it’s fine.</p>
<p><strong>The real cost is tokens, not compute.</strong> I assumed the bottleneck would be CPU/GPU. It’s not. The hardware handles everything comfortably. What actually costs money is the LLM API. I pulled traces from <ahref="https://github.com/comet-ml/opik">Opik</a> and found 75 million input tokens across 100 traces. Context window size is the killer — long conversations with tool call history get expensive fast.</p>
<p><strong>V8 memory limits in Docker will silently kill your agent.</strong> The agent framework runs on Node.js. Docker containers have a default memory limit, and Node’s V8 heap will hit it and crash without a clear error. The fix is <code>NODE_OPTIONS=--max-old-space-size=4096</code> in your container environment. I lost a few hours to this before finding it.</p>
<p>The 75M input tokens I mentioned came from a period of heavy testing with long context windows. Normal usage is significantly lower. The lesson: be deliberate about what you include in the system prompt and conversation history. Every token in context costs money on both ends of the conversation.</p>
<h2id="whats-next">What’s Next</h2>
<p>A few things I’m working on:</p>
<ul>
<li>WhatsApp integration — Telegram works great but WhatsApp is where most people actually are</li>
<li>Multi-tenant hosting — running this for a small group, not just myself</li>
<li>Thermal mass modeling for HVAC — using the agent framework to build something that actually reasons about home energy, not just schedules</li>
</ul>
<p>The infrastructure is solid enough that I’m spending more time on what the agent <em>does</em> than on keeping it running. That’s the right place to be.</p>
<p>If you’re thinking about building something similar, the barrier is lower than you’d expect. You don’t need a GPU server. You don’t need a cloud budget. You need a machine that can run Docker, an API key, and a weekend.</p></div></main><footerdata-astro-cid-5hce7sga><ahref="/privacy"data-astro-cid-5hce7sga>Privacy Policy</a> ·
<ahref="/terms"data-astro-cid-5hce7sga>Terms of Service</a> ·
