Modern LLMs like Ollama are technically ground-breaking but suffer from significant thermal and energy inefficiencies on resource-constrained hardware. This is often an overlooked cost of LLM’s deterministic nature of token generation along with the heavy, unoptimized CPU operations within the underlying math engines.

High energy demand translates to high water usage and thermal dissipation needs. For many communities, this environmental footprint makes local AI inaccessible or unsustainable. To solve this, we must adopt a more “frugal” philosophy, the way Android development does.

This talk explores the forensics of loading Llama 3.2 (1B) onto a Raspberry Pi 4 to emulate resource-constrained conditions. Through the lens of Android and Kotlin Multiplatform development — molded and developed in its nature of resource-constrained hardware — we will audit Ollama’s source code and profile how it performs in real time. We will move past the high-level Go wrappers and into the unforgiving C++ threading and memory management, identifying the leaks and bottlenecks leading AI to drown in thermal throttling and hallucinations.

What you’ll take away:

• How to apply mobile performance patterns (like scoped locking) to AI engines
• How to monitor real-time hardware telemetry (thermal/RAM) against token generation speed
• Why “frugal computing philosophy” is an ethical necessity for sustainable AI deployment needed to save our fresh water resources

Amanda Hinchman-Dominguez is an Android developer, co-author of “Programming Android with Kotlin” and other works. She specializes in the Kotlin compiler, Android, and intersectional topics. As an active participant in the Kotlin community, Amanda organizes the Chicago KUG, writes tech blogs, and contributes to a variety of OSS projects. In her spare time, she practices hot yoga and plays kickball.