Most IoT systems transmit far more data than anyone will ever look at. A vibration sensor sampling at a few kilohertz produces millions of readings a day, and the overwhelming majority describe a machine behaving exactly as expected. Every one of those readings is paid for twice — once in transmission energy and connectivity cost, once in storage and query cost.
Data reduction is the practice of deciding, as close to the sensor as possible, which information deserves to travel. Done well it cuts operating cost by an order of magnitude, extends battery life, and makes dashboards faster, all without losing anything anyone needed.
This guide covers the techniques that actually work, how to choose between them, and the reduction mistakes that quietly destroy value you cannot recover later.
Why Sending Everything Is the Expensive Default
Transmitting all raw data is the easiest architecture to build, which is why so many systems start there. It becomes untenable for four independent reasons, and usually all four at once.
| Cost | How it scales | When it becomes the constraint |
|---|---|---|
| Radio energy | With payload size and retries | Immediately, on any battery device |
| Connectivity | With volume and device count | At a few hundred devices on metered links |
| Ingestion and storage | With messages per second and retention | At fleet scale, then permanently |
| Query performance | With accumulated history | Roughly a year after launch |
The fourth is the one teams least expect, because it has nothing to do with fleet size. A system with a fixed number of devices still slows down every month as history accumulates, and no amount of query tuning fixes a table that grows without bound.
The cheapest byte in any IoT system is the one the device decided not to send.
Seven Ways to Reduce IoT Data
These are ordered roughly by how much they reduce volume, and by how much judgement each requires.
Sample at the rate the physics requires
The simplest and most frequently missed reduction. Many systems sample far faster than the underlying phenomenon changes, generating volume that carries no additional information.
Set the rate from the fastest change you genuinely need to observe, not from what the sensor supports. A tank level that moves over hours does not need per-second sampling. The one caution is aliasing: if you are sampling a periodic signal, the rate must still respect the frequency content, so reduce deliberately rather than arbitrarily.
Report on change, with a deadband
Instead of reporting every interval, report only when a value moves by more than a defined amount. On stable processes this removes the overwhelming majority of messages.
- Send a periodic heartbeat, so silence is distinguishable from a dead device
- Set the deadband from measurement noise, otherwise noise itself triggers reports
- Always report immediately on threshold crossings, regardless of deadband
Without the heartbeat, this technique creates a dangerous ambiguity: a device that has nothing to report looks identical to one that has failed.
Aggregate over a window
Rather than sending every sample, send summary statistics per window — minimum, maximum, mean, standard deviation, and count. A minute of one-second samples becomes five numbers instead of sixty.
Including minimum and maximum matters more than it appears: an average alone hides the transient spikes that usually carry the diagnostic information. Aggregation that keeps only the mean is the most common way teams accidentally discard the signal they were trying to capture.
Extract features instead of sending signals
For high-rate sensing such as vibration or audio, transmit computed features rather than the waveform: spectral peaks, band energies, RMS, crest factor, kurtosis.
This routinely achieves reductions of several orders of magnitude, and for condition monitoring the features are what the analysis uses anyway. The trade-off is that feature choice is now fixed in firmware, so a question nobody anticipated cannot be answered retrospectively.
Detect events on the device
Move from continuous reporting to event reporting. The device watches continuously and transmits only when something meaningful occurs, with a short window of context around it.
This is the pattern used by vision systems that count rather than stream, and by TinyML classifiers that report a behaviour instead of a raw sensor trace. It is the most powerful form of reduction because the device is deciding what matters, not merely compressing what happened.
Encode and compress efficiently
Independent of what you send, how you encode it matters. Compact binary formats such as CBOR or Protocol Buffers typically halve payload size relative to verbose JSON, and delta encoding of slowly changing values compresses further.
Batching several readings into one message also amortises protocol overhead, which on constrained radios is often a substantial fraction of each transmission.
Adapt the rate to conditions
Reporting frequency does not have to be constant. Increase it when a value approaches a threshold, when an anomaly is suspected, or when a process is active; decrease it when everything is stable.
This gives high resolution exactly when it is informative and near-zero cost the rest of the time — usually the best overall ratio of value to volume, at the price of more firmware complexity.
Choosing the Right Technique
| Situation | Best technique | Typical reduction |
|---|---|---|
| Slow-changing process values | Change-based reporting with heartbeat | Very high on stable processes |
| Continuous monitoring, trends needed | Windowed aggregation with min and max | An order of magnitude |
| High-rate vibration or audio | Feature extraction on device | Several orders of magnitude |
| Video and imaging | On-device inference, send results only | Extreme — images never leave |
| Anything on a metered radio | Binary encoding plus batching | Roughly half, on top of other methods |
| Processes with distinct active and idle phases | Adaptive sampling rate | High, with resolution preserved where it matters |
These combine well. A vibration monitor might sample at full rate, extract features on device, aggregate them into windows, encode in binary, and increase reporting frequency only when a feature drifts — each stage multiplying the saving of the last.
The Reduction Mistakes That Cost You Later
Reduction is irreversible. Data not sent cannot be recovered, so a few decisions deserve genuine care.
Deciding these policies is part of choosing an IoT deployment architecture, because they determine what the platform will ever be able to answer — and they are far cheaper to set before deployment than after.
Frequently Asked Questions
Conclusion
Data reduction is the highest-leverage optimisation available in a connected system, because it reduces cost at every tier at once — radio, connectivity, ingestion, storage, and query — rather than shifting load from one to another.
Sample at the rate the physics demands, report on meaningful change with a heartbeat, aggregate while preserving extremes, extract features for high-rate signals, detect events on the device, encode compactly, and adapt the rate to conditions. Keep raw data from a few units, make the policy configurable, and never invent data to fill a gap. That combination turns an expensive telemetry firehose into a system that sends only what someone will actually use.