For embedded system designers, the important question about artificial intelligence is increasingly not simply whether to use it, but where the intelligence should reside.
A machine, camera, or sensor can send data to a Cloud platform for analysis. It can make some decisions locally while using the Cloud for wider analysis and management. Or it can perform inference within the sensor or embedded device itself.
Each approach changes the system around it. Latency, bandwidth, power consumption, connectivity, security, and the information travelling across the network are all affected by where a decision is made. That makes Edge AI as much an architectural decision as an AI one.
Why Cloud-first is not always the answer
Cloud computing remains valuable for AI. Centralised infrastructure provides substantial processing and storage resources and suits model development, long-term analysis, fleet management, and software distribution.
But a Cloud-first architecture assumes the required data can reach that infrastructure whenever a decision needs to be made.
An autonomous mobile robot encountering a person in a warehouse needs to respond immediately. An industrial monitoring system may need to recognise a developing fault while its external connection has failed. A smart camera may continuously produce far more information than the wider system needs.
In such applications, sending data elsewhere before deciding what it means can introduce latency, consume communications capacity, and make the function dependent on network availability.
Microcontrollers, processors, neural processing units (NPUs), AI accelerators, and intelligent sensors can now perform useful inference close to the source of data. The result is not the disappearance of the Cloud, but a different division of labour between device, Edge, and Cloud.
Start with the decision, not the processor
It is tempting to begin an Edge AI project by comparing processor performance, model sizes, or tera-operations per second. A more useful starting point is the decision the system needs to make. What must it detect or classify? How quickly must it respond? What happens if the answer is wrong? How often must the decision be made, and what information is needed to make it reliably?
Consider an industrial camera monitoring access to a restricted area. If its task is to determine whether somebody has crossed a boundary, continuously transporting video to a remote service may be unnecessary. A local model can analyse the images and produce the required event.
A condition-monitoring sensor presents a similar choice. Its purpose may be to identify abnormal vibration rather than provide another system with an uninterrupted waveform. A useful principle is to process selectively and transmit only the information needed by the wider system. Rather than stream every frame or measurement, an Edge device can monitor locally and send an event, classification, anomaly, diagnostic indicator, or concise summary only when wider action or analysis is needed. The requirement is not to move the maximum amount of data to the most powerful processor. It is to place enough processing at the appropriate point in the system to make the required decision reliably.
Where should Edge intelligence run?
Inference can sit at several levels in a practical embedded system:
- Inside or alongside the sensor: processing close to the physical measurement can minimise data movement and provide rapid responses. It suits tightly defined tasks and power-constrained devices, although processing and memory resources are limited
- In the embedded controller: a microcontroller or application processor can combine sensing, control, and inference. This offers more flexibility and can allow several inputs to contribute to one decision
- In a local gateway or industrial computer: more demanding models, multiple sensors, or machine-vision applications may justify greater local compute capability. A gateway can aggregate data and perform sensor fusion without depending on a remote data centre
- In the Cloud: central infrastructure remains appropriate when a task benefits from substantial computing resources, long-term historical data, fleet-wide comparison, or central management
- Most useful systems use more than one layer: a device might identify an anomaly immediately while a Cloud application examines trends across hundreds of deployed machines. The engineering task is to assign each responsibility to the layer where it makes most sense
Latency matters, but it is not the whole story
Removing a network round trip can be important in robotics, automation, and other systems where the physical process cannot wait for a remote server. But latency alone does not justify every Edge implementation.
Connectivity can be equally important. A remote monitor may operate where wireless coverage is intermittent or expensive. A factory system may need to retain essential functionality when an external connection is interrupted. Local inference allows a device to continue observing and responding when the wider network is unavailable.
Privacy and intellectual-property concerns can also influence the architecture. Keeping raw video, biometric information, or factory-process data within a device or site reduces the amount of sensitive material crossing a network.
That does not make an Edge device inherently secure. Secure boot, device identities, encryption, controlled software updates, and protection for stored models and data remain essential. Moving inference changes the attack surface; it does not eliminate it. For connected products, cybersecurity and lifecycle requirements should be considered alongside hardware and model selection, not left to a later software stage.
Choose hardware for the job
Moving computation closer to the sensor introduces constraints. Processing consumes energy; models require memory; accelerators occupy silicon or board space; and higher performance can affect cost and thermal design. Reduced data movement can also reduce network load and, depending on the architecture, may simplify some supporting hardware requirements.
For narrowly defined tasks, TinyML and compact models can allow inference on modest microcontrollers. Demanding vision, robotics, or multi-sensor applications may require dedicated accelerators or higher-performance processors.
The useful measure is not peak AI performance in isolation. It is whether the platform can make the required decision within the required time, power, and cost budget, under the conditions the product will encounter.
Platform selection also includes the surrounding development ecosystem. Evaluation kits, pre-integrated modules, reference designs, model-conversion tools, and software libraries can shorten the route from proof of concept to a production architecture. The question is not only whether a processor can run a model, but whether the team can develop, validate, secure, and maintain the complete system around it.
Sensor fusion adds context and complexity
A camera may provide excellent information in good conditions but become less dependable in poor light, dust, reflections, or occlusion. Combining vision with LiDAR, UWB, or other sensing methods can give a system more context.
Industrial condition monitoring presents the same opportunity. One measurement may indicate that something has changed; several related measurements can help establish what the change means.
Sensor fusion can improve confidence, but it increases processing and data-handling requirements. Designers need to decide whether fusion belongs inside a device, in a nearby controller, or at a gateway. The right location for fusion therefore depends on the decision, the available compute, and the cost of moving the underlying data.
Plan for uncertainty, failure, and change
An AI demonstration usually concentrates on successful inference. A deployable system must also consider what happens when the model is uncertain or wrong.
False positives can trigger unnecessary interventions. False negatives can be more serious in safety, quality, or machine-health applications. Conditions also change after deployment: lighting varies, machines wear, processes change, and users may behave differently from the development dataset.
A practical Edge AI architecture needs a defined response to uncertainty. It might report a confidence indicator, revert to conventional control logic, request another sensor input, or escalate an ambiguous event for external analysis.
Teams should also identify the model and firmware version running on each deployed device, validate updates before release, and monitor whether real-world performance remains acceptable. In long-lived or safety-relevant equipment, model management is an engineering lifecycle responsibility, not a one-off development task.
When decisions move local, the network changes too
Once a device can interpret its own measurements, the information leaving it changes. A conventional sensor may provide a sequence of measurements for another system to interpret. An intelligent device can instead provide a state, classification, anomaly, or alert.
A camera does not necessarily have to transmit every image simply to tell a wider system that somebody entered a restricted area. Likewise, a machine-monitoring device may not need to export every accelerometer sample when its immediate purpose is to recognise a developing fault.
This raises a second question beyond where a decision should be made: what information needs to leave the device once that decision can be made locally?
The next article in this series looks at how AI-enabled sensors can convert raw vibration, video, and environmental measurements into contextual information at the point of sensing. The final article will consider what this shift means for field-level communications, including the role IO-Link can play as intelligence moves closer to the point of measurement.
Edge AI is an architecture decision
The best Edge AI design is not necessarily the one that performs the most computation locally. It is the one that puts each task where it makes engineering sense.
Immediate sensing and time-critical decisions may belong close to the physical process. More demanding local analysis may belong in an embedded controller or gateway. Fleet-wide comparison, historical analysis, model development, and management may remain in central infrastructure.
Start with the required decision, then select the sensing, processing, network, and lifecycle approach that can deliver a timely and trustworthy outcome within the product’s real constraints.
Explore Farnell’s AI solutions
From development kits and evaluation boards to AI-enabled hardware and engineering resources, Farnell supports engineers in designing and developing Edge AI solutions. Explore resources like DevKit HQ and the Farnell AI Hub to find the tools and information you need to take an AI concept from development to deployment.
Five questions before choosing an Edge AI platform
- What specific decision must the system make, and what happens if it gets that decision wrong?
- How quickly must it respond?
- What power, thermal, and memory budget is available?
- Which functions must continue if connectivity is lost?
- How will the model be validated, versioned, monitored, and securely updated?
