Wireless on the Plant Floor: Diagnosing Dropouts, Roaming Failures and Interference

intermediate · 14 min

A machine that "lost WiFi" has failed in one of three different ways — interference, a coverage hole, or a roam that outlasted the control protocol's watchdog. This procedure separates them with numbers you can read off the client radio in ten minutes, then fixes each one, and is honest about the traffic that should never leave copper.

ℹ️ Most "wireless dropouts" on a plant floor are not radio failures at all. The radio recovers in 200–400 ms; the control protocol gives up in 100 ms. The link did exactly what it was designed to do — you measured it against a budget it was never told about. Before you touch an antenna, do the timeout arithmetic.

An AGV caught mid-handover between two ceiling APs: the re-association gap — probe, auth, assoc, 802.1X and the 4-way handshake — outlives the I/O watchdog long before signal strength ever becomes the problem.

Tell the three failures apart first

A machine that "lost WiFi" failed in one of three ways. They look identical on the HMI and need different fixes. The discriminator is not where it happens — all three can repeat at the same spot on an AGV route. The discriminator is what the RF and the association state were doing at the moment of the drop.

Interference — the client stays associated, but frames get corrupted and retransmitted. RSSI (received signal strength, in dBm; closer to zero is stronger) looks healthy, the noise floor is elevated, and the retry percentage climbs. The client never left the AP.

Coverage hole — signal runs out and there is nothing else to go to. RSSI from the serving AP walks down past −75 dBm, and no neighbour AP is above the roam threshold at that point. The client drops without ever reassociating.

Roaming failure — a neighbour AP was available and strong, the client attempted the handover, and the handover took longer than the control protocol would wait. You will see an association or reassociation attempt in the AP/controller log at the drop timestamp — that is the tell. A coverage hole has no such attempt.

Read four numbers off the client radio's own status page before opening any configuration screen: RSSI, SNR (signal-to-noise ratio, in dB), noise floor, and retry rate. Then read the disconnect reason code and any (re)association attempts in the AP or wireless controller event log. −55 dBm sitting on a −70 dBm noise floor is interference, not coverage — a healthy noise floor is nearer −92 dBm. No log entry at all usually means the client simply stopped being heard.

2.4 GHz is crowded; 5 GHz is usually right

2.4 GHz gives you three channels that do not overlap: 1, 6 and 11 in North America. Each occupies roughly 22 MHz with centres only 5 MHz apart, inside the 2400–2483.5 MHz ISM band — about 83.5 MHz total. Any other channel overlaps and corrupts your own traffic.

The same band carries microwave ovens and industrial microwave heating and drying equipment (2.45 GHz ISM), Bluetooth, 802.15.4 radios including WirelessHART and ISA100.11a, cordless phones, analogue video senders and some RFID. (RF/dielectric plastic welders are a serious industrial RFI source, but they run at 27.12 MHz — they will not appear in a 2.4 GHz sweep. Do not go looking for them there.)

5 GHz gives you roughly two dozen 20 MHz channels across UNII-1 (36–48), UNII-2A (52–64), UNII-2C (100–144) and UNII-3 (149–165) — about 25 in the US plan. Free-space path loss is higher and non-metallic obstacles attenuate more, so cells are smaller. Indoors that is an advantage: smaller cells mean less co-channel interference. (Metal reflects and blocks at both bands equally — that is not a 5 GHz effect.)

The catch is DFS (Dynamic Frequency Selection). UNII-2A (52–64) and UNII-2C (100–144) are shared with radar. An AP must complete a Channel Availability Check — listening, not transmitting — of 60 seconds before using the channel, extended to 600 seconds for channels in the 5600–5650 MHz weather-radar range, and must vacate within a 10-second channel move time if it detects radar, real or a false positive. That means your AP can change channel mid-shift and every client re-associates. For control traffic, prefer non-DFS channels where your regulatory domain permits.

Do the timeout arithmetic before blaming the radio

This is the AGV problem. A moving client leaving one AP for the next must re-authenticate. With WPA2/WPA3-Enterprise a full EAP re-authentication can take hundreds of milliseconds. Meanwhile:

• EtherNet/IP — CIP defines the connection timeout as RPI (Requested Packet Interval) × a configurable connection timeout multiplier (4, 8, 16 … up to 512). On Rockwell Logix the default is ×4, and for small RPIs Logix raises the multiplier automatically so the resulting timeout is at least 100 ms — a 10 ms RPI therefore gives roughly a 100 ms budget. That floor is Logix behaviour, not a CIP rule. Do not compute it: read the actual Connection Reaction Time Limit from the module's Connection tab, and on a non-Rockwell scanner read whatever that vendor exposes.
• PROFINET — the watchdog is the update time multiplied by the "accepted update cycles without IO data". Siemens defaults that factor to 3 and it is configurable upward. A 4 ms update time at ×3 gives roughly 12 ms.

A 300 ms roam against a 100 ms watchdog drops the connection every lap. Fix it in this order:

1. Confirm the client radio supports 802.11r/k/v before changing anything. Many industrial WLAN modules do not. This step comes first because enabling Fast Transition on an SSID can stop non-FT clients associating at all — older supplicants fail to parse the additional AKM suites in the RSN information element and simply never send an association request. Turning FT on to "see if it helps" can knock every control client off the network at once.
2. Then enable 802.11r (Fast BSS Transition, which folds key establishment into the handover and removes the separate re-authentication), plus 802.11k (neighbour reports) and 802.11v (BSS transition management) so the client knows where to go. If some clients support FT and some do not, stage it on a parallel SSID rather than converting the production one.
3. Then raise the RPI or update time and the watchdog multiplier only as far as needed to exceed the measured roam.

Vendor-specific options exist: Siemens iPCF and iPCF-MC on SCALANCE W hardware poll clients on a coordinated schedule and are specified for handovers in the tens of milliseconds for AGVs, cranes and high-bay shuttles — that is a Siemens feature requiring Siemens APs and Siemens client modules, not a standard one.

⚠ Every change in this section interrupts production traffic. Changing RPI, update time or watchdog values downloads to the controller and drops I/O connections. Changing SSID security settings, the channel plan, or moving APs off DFS channels deauthenticates every associated client — including an AGV in mid-route and every handheld on the floor. Do all of it with the line stopped and outputs de-energised, in a planned window.

Safety and hard real-time I/O

PROFIsafe (V2) is approved for wireless transmission channels including WLAN and Bluetooth via the black channel principle: the safety layer carries its own CRC, consecutive number, codename authentication and watchdog, so it does not care what transported the frame. PI approves the profile against a residual bit error probability up to 1e-2, which is what makes a lossy radio acceptable as a black channel.

That makes wireless safety safe — not available. A lost frame does not produce a dangerous state; it produces a safe state, which on a plant floor means the machine stops. The F-watchdog still has to be set against the measured link latency, and setting it long to stop nuisance trips lengthens your safety reaction time — it is a safety parameter, not a nuisance filter.

So wireless E-stops and wireless teach pendants exist and are certified, but they must go to a safe state on loss of signal — and interference therefore surfaces as nuisance stops. Servo position loops, motion interlocks, drive control at single-digit-millisecond cycles, and safety I/O that must not nuisance-trip belong on copper or fibre.

Wireless is genuinely good for: HMIs and tablets, condition monitoring, barcode and RFID handhelds, AGV supervisory traffic (transport orders, status, telemetry), diagnostics and remote access, and battery-powered process instruments over WirelessHART or ISA100.

On those last: both standards technically support publish periods down to 250–500 ms, but real deployments run at one second or slower because faster rates destroy the multi-year battery life that is the whole point of the technology. Combined with non-deterministic mesh hop latency and retries, that rules them out for closed-loop control regardless of the configured rate.

Antennas, metal and interference

Racking, cable trays, machine frames and steel decking reflect RF. Reflections arrive out of phase and cancel, producing deep fades that shift over a few centimetres. Mount antennas clear of large metal surfaces and use a standoff bracket rather than bolting flat to an enclosure — a metal enclosure acts as ground plane, reflector and shield at once, detuning the antenna and distorting its pattern. Match polarisation between AP and client; a rotated antenna loses signal for no visible reason. Cover aisles from above rather than trying to shoot through racking.

Step-by-step

1. Reproduce and locate. Note exactly where the drop happens and whether the client was moving.
2. At that spot, log RSSI, SNR, noise floor and retry rate from the client radio.
3. Pull the disconnect reason code from the AP or controller event log — and check whether an association or reassociation attempt appears at that timestamp. An attempt means roaming failure; silence means the client was never heard.
4. Sweep the spot with a spectrum analyser (Ekahau Sidekick, MetaGeek Wi-Spy, RF Explorer). Non-WiFi energy confirms interference. A clean sweep does not rule it out — plant interferers are intermittent, so capture across the shift or machine cycle that actually fails before you clear the band.
5. Passive-survey the whole route. Common enterprise and voice design guidance is −67 dBm or better at cell edge with at least 25 dB SNR (noise floor no worse than about −92 dBm).
6. Verify the channel plan — 1/6/11 only on 2.4 GHz, and no two APs on the same channel within earshot of each other. ⚠ Applying a channel change drops all clients: planned window only.
7. Check DFS. If control clients sit on 52–64 or 100–144, look for radar-detect events in the AP log; moving them to non-DFS channels is a planned-window change, not a live one.
8. Measure the actual roam time. Compare it to the configured connection timeout read from the module properties (Connection Reaction Time Limit on Logix) or the PROFINET watchdog — not to a formula.
9. Verify that the control clients' radios support 802.11r/k/v. Only if they all do, enable it on the SSID carrying them — line stopped, because non-FT clients may be unable to associate afterwards. If support is mixed, stage a parallel SSID instead.
10. Only now adjust RPI, update time or watchdog — line stopped, outputs de-energised.
11. Re-run the route ten times with the route cleared of personnel and someone on the E-stop, logging RSSI, retries and roam count each lap.

Never do this

Never widen the watchdog until the nuisance trips stop. This is the expensive mistake, and it is common: an AGV drops every lap, someone stretches the PROFINET watchdog from 12 ms to 2 seconds or bumps the RPI multiplier, the alarms go quiet, and the job is declared closed. You have not fixed the link — you have blinded the controller. The vehicle now runs for up to two seconds on stale process data before anyone notices, and a genuine comms failure that used to fault safely now looks like normal operation. Fix the roam time, then set the watchdog to the smallest value that survives it. The same applies doubly to an F-watchdog, where the number you are stretching is a safety reaction time.

Two more: never raise AP transmit power to fix roaming — a louder AP makes clients cling to it longer and roam later. And never put a hard real-time I/O connection or a safety function on wireless because pulling cable was inconvenient; the cable is cheaper than the incident.