Manufacturing data platforms: giving factory data the context it did not arrive with

Raw signals are nearly worthless without the order behind them

A stream of measurements arriving with an equipment tag and a timestamp can be stored cheaply and analysed almost not at all. Contextualisation adds the associations that make a question answerable: the works order, the product and revision, the material lot, the tool, the shift, the operator, the process step. Those come from the planning and execution systems rather than from the equipment, which means the platform is an integration exercise as much as a storage one. Manufacturers who build storage first and context later usually accumulate a large archive that nobody can use, and then rebuild it.

Two kinds of data that behave differently and need each other

High-frequency readings are appended constantly, queried across time ranges, compressed heavily and rarely updated. Transactional records about orders, batches, defects and materials are comparatively few, change state, and carry relationships. Storing both well in a single technology is awkward, which is why most architectures keep them separately and join on identity and time. The design work that matters is defining those joins precisely: how a time range maps to an order, what happens at a changeover, and how a batch that spans a shift boundary is attributed. Vague answers there produce analysis that cannot be reproduced.

Retention and resolution are decided early and regretted later

Keeping every reading at full frequency forever is expensive; thinning aggressively saves money until an investigation needs the detail that was averaged away. The decision should follow the questions the plant expects to ask, including the ones that arrive from customers after a field failure. A common structure keeps full resolution for a recent window, retains reduced detail for longer, and preserves events and exceptions indefinitely at full fidelity. Whatever is chosen should be written down and reviewed, because retention set by an infrastructure default silently determines what will be possible to investigate.

Ownership of the model has to be internal

The definitions of equipment, process step, product and event, and the way they relate, constitute a description of how the business manufactures. If that description exists only inside an integrator's configuration or a single specialist's knowledge, the platform becomes an asset the company cannot maintain or move. Documented models, version control on configuration, and at least two people internally who understand it are unglamorous requirements that decide whether the platform survives staff turnover and contract changes. Ask during procurement what the exit looks like and what leaves with the people who built it.

The line to the control layer is a security boundary

Pulling data from equipment means creating a path between production control networks and business systems, and that path is exactly what industrial cybersecurity guidance exists to govern. Sensible arrangements favour flows that originate on the control side and push outward, strictly limited protocols, no route by which an analytics user can reach a controller, and segmentation that survives someone plugging a laptop into the wrong port. The technology choices belong with the automation and network engineers; the governance decision, about who may connect what and on whose authority, belongs to plant leadership.

Frequently asked questions

Should the platform sit in the plant or centrally?

Both, usually. Local collection and buffering keeps data safe when the connection to a central site drops, which happens more often than network diagrams suggest, and keeps latency-sensitive uses working during an outage. Central aggregation is where cross-site comparison, longer retention and heavier analysis belong. The split that causes trouble is one where the plant depends on a remote system for something it needs during production, since a network fault then becomes a production stoppage.

Do we need this before we can do anything useful with machine data?

No, and starting with a platform programme is a common way to spend a year without changing anything on the floor. Solving one concrete problem on one line, with whatever collection method is quickest, teaches you what context is genuinely needed and produces a result people can see. The argument for a shared platform strengthens once the third or fourth such project starts rebuilding the same connections, which is the point at which duplicated effort becomes visible.

Who should own it, IT or engineering?

Engineering understands what the signals mean and how the process behaves; IT understands storage, access control, backup and lifecycle. Neither can run it alone. The pattern that works is joint ownership with explicit boundaries: engineering owns the data model and the meaning, IT owns the infrastructure and security posture, and both sign off changes that cross the line. Leaving ownership ambiguous produces a platform that is nobody's priority when it needs upgrading.