Process historians: keeping plant time-series data that is still usable years later

Compression discards data on purpose, and the setting is a judgement

Time-series stores typically avoid keeping every sample, recording a point only when the signal deviates from a predicted path by more than a configured amount. That is why decades of plant history fit in a modest space. It also means a deviation smaller than the threshold never existed as far as any future analysis is concerned. Defaults are usually set for storage efficiency rather than for analytical usefulness, and they are applied uniformly across tags of wildly different character. Review the settings on signals that matter for quality or failure investigation, because a threshold chosen for a slow tank level will erase the detail on a fast control loop.

Tags outlive the instruments behind them

A point name persists while the physical reality underneath it changes: a transmitter is replaced with a different type, its range is rescaled, the tapping is moved, the units change during a project. The stored series continues without interruption and now contains a discontinuity that looks like a process event. Anyone analysing it years later has no way to know unless the change was recorded. Keep a change log against the tag itself, covering instrument replacement, range changes, unit changes and relocation. This is unglamorous record-keeping that pays for itself the first time somebody investigates a trend spanning the change.

Retention is a decision about future questions

Retention is usually set by whoever installed the system, based on disk, and revisited when someone needs data that no longer exists. Base it instead on the questions the plant expects to face: warranty and product liability periods, regulatory record requirements in your sector, the interval over which equipment degradation is assessed, and the seasonal cycle you need at least a couple of repetitions of. Tiering helps — full detail for a recent window, reduced resolution for the long tail — provided the reduction is documented. Deleting the raw record is irreversible and the value of old process data is only apparent once you need it.

Naming and metadata decide whether anyone can find anything

A historian with a large tag count and no discipline becomes a system where only one engineer can locate anything. Each point needs a name following a stated convention, a description in plain language, units, an equipment association and an owner. Where an asset model exists that groups points by physical equipment, use it, because it lets someone ask about a machine rather than needing to know the identifiers of its individual measurements. The maintenance of this metadata has to be part of every project's handover, otherwise it is accurate on the day of commissioning and degrades from then on.

Storage is not the application, and the boundary is worth defending

Historians are excellent at recording and retrieving signals against time. They are a poor home for production reporting, scheduling, quality management or anything requiring transactional integrity, and every plant that has built such things inside one has produced something fragile that a single leaver takes with them. Keep the historian as the plant-floor record of what the equipment did, expose it through documented interfaces, and let the manufacturing and analysis applications sit above it. The interface, not the storage engine, is the part worth investing engineering attention in.

Frequently asked questions

Why does historical data look smoother than what the operator saw?

Because compression kept only the points needed to reconstruct the signal within a tolerance, and everything inside that band was discarded. On a signal with meaningful small variation, that removes exactly the detail an investigation wants. Check the deviation setting on the tags that matter and tighten them where the fine structure has value, accepting the extra storage. Also check whether the trend display is averaging over the requested period, which can smooth a signal further without any indication.

How long should plant process data be retained?

Long enough to cover your product liability and warranty exposure, any sector record-keeping obligation, and enough repetitions of the seasonal or campaign cycle to make comparison meaningful. For most manufacturers that is considerably longer than the default configuration allows. Where full-resolution retention is impractical, reduce resolution for older data rather than deleting it, and write down what the reduction does, so that a future analyst understands why old periods contain fewer points than recent ones.

Can a general-purpose database replace a purpose-built historian?

It can, and plenty of plants store time-series data in general database technology successfully. What you take on is the engineering that a purpose-built product provides out of the box: efficient storage of long series, interpolation and aggregation over time ranges, quality flags, and connectors to control systems. The decision usually rests on whether you have the skills to maintain that yourself and whether existing tools your engineers use can query the result without help.