Designing My AI Stack

I find it nearly impossible to give up working with physical, on-premise computers and devices. It's the same reason I prefer working in a lab environment—there's something uniquely rewarding about bringing up new hardware or wrestling with a jumble of software. No matter how many one-click, well-documented cloud features become available, I can't shake my attachment to real hardware. For me, it comes down to trust and a sense of autonomy.

When I first looked at available options for running local LLMs, I wasn't convinced of their usefulness for Agentic AI. But as they say about New England weather, here in USA, 'if you don't like it, wait a minute.' The AI market changes just as rapidly. I didn't wait a minute—I waited a week. Then the DeepSeek 3.2 and Mistral3 press releases dropped on Dec 2nd, nudging me closer to exploring local LLM development. What sealed my decision was Nvidia's Nemotron 3 family of models announcement on Dec 15th. Be sure to checkout the YouTube video associated with the blog titled "How to fine-tune LLM on Nvidia GPU's with Unsloth".

Designing a AI solution that includes a locally-deployed LLM is emerging as a realistic option. Having a sovereign Agentic AI stack may appeal to customers concerned about intellectual property exposure.

What's your experience with local LLMs for production use?

#LocalLLMs #AgenticAI #SovereignAI