Generic AIoT architecture diagrams are not very useful. They show the same five boxes regardless of whether the product is a coin-cell sensor or a camera on a production line, and the interesting decisions are precisely the ones those diagrams hide.
This guide takes a different approach: three concrete reference architectures for three common product shapes, with the actual constraints that drive each design. Most AIoT products are a variation on one of them.
Battery Sensor With On-Device Classification
Typical use: condition monitoring on equipment with no power supply, asset or environmental sensing, agricultural and livestock applications.
| Layer | Choice | Driven by |
|---|---|---|
| Compute | Low-power MCU, sometimes with a small NPU | Multi-year battery life |
| Model | Kilobyte-scale classifier on extracted features | Peak RAM at the widest layer |
| Connectivity | LoRaWAN, NB-IoT or BLE to a gateway | Range and transmit energy |
| Transmission | Summaries and events only, batched | Radio dominates the energy budget |
| Cloud role | Baselines, fleet comparison, retraining | Device cannot hold history |
The binding constraint is energy, and everything follows from it: features rather than raw signals, cheap triggers gating expensive inference, and transmissions measured in bytes. Model updates must be small enough to deliver over a constrained link, which often means configuration tuning rather than frequent model replacement.
Industrial Line With an Edge Node
Typical use: production monitoring, predictive maintenance across several machines, OEE and quality tracking.
| Layer | Choice | Driven by |
|---|---|---|
| Sensing | Retrofit sensors plus existing PLC tags | Cannot modify certified machine control |
| Edge node | Industrial gateway with real compute | Protocol translation and local correlation |
| Model placement | Edge node, with device-level pre-processing | Must keep working if the uplink fails |
| Connectivity | Wired, outbound-only to the platform | OT security review |
| Cloud role | Cross-site comparison, training, fleet config | Needs data from multiple sites |
The binding constraint is the OT boundary. Read-only, outbound-only data paths and certificate-based identity are what make the design acceptable to the people responsible for plant safety, and retrofitting that later is considerably harder than designing it in.
Vision System at the Point of Decision
Typical use: inspection, counting, safety monitoring, guidance.
| Layer | Choice | Driven by |
|---|---|---|
| Imaging | Controlled lighting, lens and exposure matched to motion | Optics determine accuracy more than the model |
| Compute | SoC with GPU or dedicated accelerator | Real-time inference on image data |
| Model placement | On the device, always | Video is impractical to transmit; privacy |
| Transmission | Results, counts and sampled frames | Bandwidth and privacy |
| Cloud role | Retraining from sampled and disputed frames | Needs labelled examples over time |
The binding constraint is that images must not leave the device, for bandwidth and usually privacy reasons. That forces real compute at the edge and makes the feedback loop harder: the system must capture a bounded sample of frames for retraining without becoming a surveillance product.
What All Three Patterns Share
The deeper architectural reasoning behind these choices is covered in choosing an IoT deployment architecture.
Frequently Asked Questions
Conclusion
There is no single AIoT architecture, but there are three recurring ones, each defined by a different binding constraint: energy, the OT boundary, or keeping images local.
Identify which constraint governs your product and the structure largely follows. Then apply what all three share — decisions near the sensor, conclusions rather than raw data, a retained raw cohort, separate model delivery, a feedback path, and real headroom — and the design will hold as the product grows.