Connecting plant systems: files, queues, database access and APIs compared

File transfer: unfashionable, durable, and silent when it dies

A scheduled job writing a delimited file to a shared location remains the most common integration in manufacturing, and there are good reasons for that. It survives brief outages, it is easy to inspect when somebody disputes what was sent, and it needs no cooperation between vendors. Its weaknesses are equally predictable: nothing detects a file that simply stopped arriving, a partially written file gets consumed as though complete, and reprocessing the same file duplicates transactions. Adding a heartbeat that alerts when an expected file is missing, plus a write-then-rename convention, removes most of the risk cheaply.

Queues and the appeal of store-and-forward

Message queuing decouples availability: the sender continues when the receiver is down, and the backlog drains afterwards. In a plant that matters, because production does not pause for a business system restart. The price is operational: somebody has to watch queue depth, handle messages that fail repeatedly rather than letting them block everything behind them, and reason about ordering when two messages about the same order arrive out of sequence. Queues suit high-volume event flows from the floor. They are overhead for a nightly transfer of a price list.

Direct database access: quick to build, expensive to own

Reading or writing another product's tables is the fastest integration to build and the one that ages worst. The schema is internal, so an upgrade can change it without notice and without any obligation to you. Writes bypass whatever validation the application performs, which is how orphaned records and impossible states appear. Support is usually withdrawn for anything touched this way. Where nothing else is on offer, restrict it to reading through a defined view, document it as a known liability, and re-test it as a matter of course at every version change.

Interfaces, and the coupling that arrives with them

A published interface is the right default, with conditions. Synchronous calls create a live dependency: if the responder is slow, the caller waits, and if the caller is an operator scanning a pallet, the line waits with them. Anything on the critical production path needs a timeout, a local fallback and a retry that does not create duplicates when the original call actually succeeded. Version handling matters as much as the design, because an interface that changes shape without notice takes the plant down more reliably than a network fault ever will.

The control-system boundary changes the rules

Between machines and business systems sits a segmentation boundary that exists for safety and security reasons, and integration must respect it rather than tunnel through it. Flows are typically constrained in direction, mediated by a gateway, and subject to change control that follows the plant maintenance window rather than the software release calendar. IEC 62443 is the standards series addressing industrial automation and control system security, published by the International Electrotechnical Commission. The operational rule worth stating plainly: no office system should be able to stop a machine, and no machine should depend on an office system to keep running.

Frequently asked questions

Is point-to-point integration always the wrong answer?

No. For a small number of stable links, direct connections are simpler to build, easier to debug and cheaper than any middleware. The cost appears as the count grows, because each new system multiplies the connections and no single place shows what is flowing where. The practical trigger for introducing a broker is not a specific number of interfaces but the moment nobody can answer which systems consume a given transaction without asking around.

What should we do if a vendor only offers database access?

Treat it as a commercial issue before a technical one. Ask what the supported integration path is, and get the answer in writing, because the response tells you a lot about how the product will age. If direct access is genuinely the only route, confine it to reads, wrap it in a view you control, keep the mapping documented, and budget for revalidation at every upgrade. Also record it on the risk register, since it is a dependency you cannot fix from your side.

How would we know an interface had stopped working?

Only if something is watching, which is the gap in most plants. Silence is the normal appearance of a broken integration: no error, no alert, just an absence of transactions that nobody notices until a stock figure looks wrong. Monitor expected arrivals rather than errors, so a missing file or an idle queue raises an alarm, and give each interface an owner who receives it. Reconciliation counts between the two ends, run on a cycle, catch what monitoring misses.