People tend to stick with the microcontroller unit (MCU) they are most familiar with. However, on-device Generative artificial intelligence introduces resource limitations that necessitate a more strategic selection process.
The challenges posed by on-device Generative AI
Microcontrollers have limited computational resources compared to typical processors. They typically have a few kilobytes of random access memory (RAM) and a clock speed measured in megahertz, whereas more powerful devices have gigabytes of RAM and gigahertz speeds.
Power management is another design challenge. Resource-hungry AI models will quickly drain batteries if not carefully managed, and real-time processing exacerbates the issue. Electronics engineers must use MCUs and algorithms that can operate within these stringent constraints.
Simply avoiding AI is not an option, as its popularity is growing exponentially. In the 2025 CFO Outlook Survey, 32% of respondents reported they were actively working with a vendor to develop or access generative AI solutions.
Key selection criteria for MCUs with on-device AI
There are five key selection criteria for MCUs used in AI-powered Edge systems.
Processing power
An 8-bit MCU is more affordable, energy-efficient, easy to use and readily available than its 32-bit counterpart. Its memory capacity is limited, and its processing speed is slow, so it can’t execute complex tasks or handle large volumes of data nearly as well.
A 32-bit MCU may be better suited for AI applications because it can transfer more in a single clock cycle. However, its high power consumption is not ideal for Edge devices. Also, it may be difficult to program and debug, so it is not the best choice for novice electronics engineers.
Memory capacity
Generally, it is better to have more memory than not enough. Electrically erasable programmable ROM combines RAM and ROM features. It can be read and written like RAM but retains its contents without power like ROM, but it is more expensive than alternatives.
Other suitable options for on-device Generative AI include RAM and Flash memory, which offer fast read access and high density at a low cost when combined. It is suitable for storing a vast amount of data for various applications.
Power consumption
All electronics face power management challenges, but the restrictions are especially noticeable among wireless MCUs running on rechargeable batteries. Typically, the solution is a trade-off between processing performance and electricity consumption.
Instruction set architecture
Microcontroller processors are based on complex instruction set computer (CISC) or reduced instruction set computer (RISC) architectures. The only pure CISC devices left are Intel’s 32-bit x86 microcontrollers, so it is not much of a choice.
Traditionally, CISC is easier to implement and more memory-efficient. It typically has around 80 instructions and 12 to 24 addressing modes. In comparison, RISC has about 30 instructions and three to five addressing modes. RISC offers high throughput and low power consumption that’s ideal for edge devices but introduces software-side complexities and increased overhead.
AI acceleration
Support for AI acceleration techniques is essential for overcoming hardware limitations. An integrated AI accelerator enables microcontrollers to run heavy workloads at the Edge, facilitating advanced AI applications.
The value of balancing performance and efficiency
Electronics engineers should balance performance, efficiency and cost to find a reliable long-term solution. Their top pick should be able to run Generative AI while meeting the target application’s specific requirements, such as real-time processing.
The perfect fit may be challenging to find or fall outside of the project’s budget. Professionals should choose an MCU that best fulfills their needs at the optimal price-to-performance ratio. The ease of integration is another consideration. Edge computing can increase compute and hardware expenses, so a cost-effective solution is necessary.
Aside from focusing on key criteria, professionals should consider optimising algorithms. In 2022, MIT researchers developed a two-algorithm solution that trains AI models in minutes using under 157 kilobytes of memory – less than an MCU’s typical 256-kilobyte capacity.
The first algorithm identifies the most important weights to update, while the second rounds 32-bit weights to 8-bit weights through quantisation. It effectively cuts the memory for both training and inference by adjusting the ratio between the weight and gradient. Strategies like this could help ease some of the selection burden.
Overcoming the challenges posed by AI integration
Individuals tend to choose the microcontrollers they used in previous projects because they are already familiar with the ins and outs. They may be able to reuse old code and avoid past pitfalls, saving time and preventing headaches. However, AI presents resource constraints and compatibility issues.
Not all microcontrollers are suitable for running machine learning models, and vice versa. Careful selection is necessary to ensure the model runs smoothly.
About the author:
Devin Partida is the Editor-in-Chief of ReHack.com, and a freelance writer. Though she is interested in all kinds of technology topics, she has steadily increased her knowledge of niches such as biztech, medtech, fintech, IoT, and cybersecurity.
