Telemetry architecture: collecting sensor data so it is still useful in three years
Why sensor data needs to be designed as a long-term asset, not launched as a one-off pilot that cannot be reused later.
When a company launches its first pilot with sensors or remote equipment monitoring, the focus is almost always on getting something to work. Pull in the first readings, show management, confirm the technology functions. That is reasonable.
The problem appears one or two years later, when it turns out the accumulated data cannot be used the way the team originally intended. The structure is awkward, the history is incomplete, the formats are incompatible with new requirements. The pilot became a production solution by accident, without design.
How telemetry differs from ordinary data
Telemetry is data from physical objects that changes over time: temperature, vibration, pressure, coordinates, speed, status. Its characteristics are that it is continuous, voluminous, and that its value accumulates over time.
A single snapshot of a reading is almost useless. Value appears when you can compare an object's behaviour today with its behaviour three months ago, or find a pattern that precedes failure. This means the storage architecture needs to be designed for time series, not for transactional records.
If telemetry is stored in the same relational database as orders and customer accounts, that database will become a problem in two years. This is a specific instance of a more general pattern: architectural decisions made informally at the start determine how much rework is required later.
Four decisions that need to be made at the start
When designing a telemetry collection system, there are several decisions that are nearly impossible to change later without losing data or rebuilding the entire architecture.
Granularity. How often are readings taken - every second, every minute, every hour? This should be decided based on the task, not "more is better just in case." Excessive granularity creates volumes that are difficult to store and process, without adding analytical value.
Timestamp. Whose time is recorded - the device's or the server's? If the device operates offline and synchronises with a delay, these are different things. Timestamp errors are one of the most common problems in subsequent analysis.
Object context. A reading without context is almost useless. Which sensor, on which piece of equipment, at which location, in which configuration. If equipment is moved or replaced, how is that reflected in the data? Without this, a year from now no one will be able to explain what a specific historical value means.
Schema evolution. What happens when new sensor types are added or the set of metrics changes? If the schema is rigid and does not allow extension, every change in equipment will require a change to the storage structure.
What makes data usable for analysis in three years
Three years from now, the value of telemetry is determined not by volume but by whether you can ask a retrospective question and get an answer.
For that, a few things need to be true. The history must be continuous - gaps and data loss make a time series unreliable. Context must be stored alongside the readings - not in a separate system that may no longer exist. Equipment metadata and configuration history must be preserved with a change log.
And most importantly: someone needs to be responsible for the fact that this data exists and is accessible. Telemetry without an owner degrades quietly.
A practical filter before starting a project
Before launching a telemetry pilot, it is worth asking a few questions:
- What analytical questions do we want to ask two or three years from now?
- What granularity do those questions actually require?
- How will we store object context alongside each reading?
- What happens to the schema when the equipment configuration changes?
- Who will be responsible for the quality of this data?
Answering these questions takes a few hours at the start of a project. Not answering them takes months of rework later.