EtherNet/IP — Implicit Connection Timeouts and Faults

intermediate · 30 min

Read an EtherNet/IP I/O drop the way the protocol actually fails: a connection that was refused at the door, or one that was accepted and then starved. After this you can name the cause from the fault code alone, size an RPI without overloading a device, and decide whether IGMP is really your problem.

ℹ️ An EtherNet/IP I/O connection is a contract with a stopwatch. Before you touch a cable, find out which half broke. Either the contract was never signed — the Forward_Open request was rejected, which is a configuration fault that fails identically after every power cycle — or it was signed and then broken, which is code 16#0203, the watchdog expiring because packets stopped arriving. The controller prints which one on the faulted module. The two lead to completely different repairs, and guessing wrong costs you a shift.

Two conversations share one wire

Implicit messaging (CIP class 1) is the cyclic I/O exchange. The scanner (the controller) and the adapter (remote I/O block, drive, valve island) agree once on the terms, then fire small UDP datagrams at each other forever, on UDP port 2222. Nothing is acknowledged. Each end simply watches a clock.

Explicit messaging (CIP class 3, plus unconnected UCMM requests) is request/response: a MSG instruction reading a drive parameter, an HMI polling tags, your laptop browsing the device. It runs on TCP port 44818. TCP acknowledges, retransmits, and hides mild network trouble.

Two details that matter later. The Forward_Open that opens an implicit connection is itself an explicit request — it travels over TCP 44818, and only then does I/O data start flowing on UDP 2222. And EtherNet/IP encapsulation also uses UDP 44818 for ListIdentity / ListServices device discovery, often as a broadcast. A firewall or router boundary that passes TCP 44818 but drops UDP 44818 will break device browsing while messaging still works.

This asymmetry gives you a partial free diagnostic. An HMI can look perfectly healthy while I/O drops, because HMI traffic is explicit and TCP retries through a marginal link — implicit traffic never retries.

But be careful what you conclude from it. If you can still browse the device's web page while its I/O connection is faulted, you have ruled out a hard break: the link is up, the device is powered, and the IP address in your project reaches something. You have not cleared the cable, the switch port, or the physical layer. A marginal link with a climbing FCS error rate, a half/full duplex mismatch, and a switch port whose multicast forwarding has stopped all let a TCP web session through while killing class 1 I/O. Those are exactly the faults the rest of this procedure hunts.

The RPI and the timeout multiplier

RPI is the Requested Packet Interval — how often, in milliseconds, each end sends its half of the I/O data.

A connection does not fault on one late packet. When the connection opens, the two ends settle a connection timeout multiplier in the Forward_Open service. It is a power of two from 4 to 512. The watchdog is:

connection timeout = RPI × multiplier

Do not assume ×4. Delta Motion documents that a ControlLogix opens its EtherNet/IP I/O connections with a timeout of 32 RPIs and does not expose the value to the user — so a 4.0 ms RPI gives a 128 ms watchdog, and a 5.0 ms RPI gives 160 ms. Other scanners expose the setting directly (ABB robot controllers, the Delta RMC itself, which accepts the full 4× to 512× range). Check your own scanner's documentation rather than assuming a number.

The practical consequence: a 16#0203 means the network or the device went silent for several consecutive RPIs, not for one frame. You are hunting a multi-packet gap — a broadcast storm, a spanning-tree reconvergence, a duplex mismatch, a device CPU stall, a brownout — not a single corrupted packet.

Before you touch anything

Three of the steps below are configuration changes, not measurements:

• Changing a module's RPI is a hardware-configuration edit. It requires an offline edit and a download, putting the controller in Program mode, and Studio 5000 raises a dedicated "DANGER — About to change RPI" confirmation for it. At minimum the connection closes and re-establishes.
• Toggling the unicast checkbox, or any edit in the I/O tree, is the same class of change.
• Inhibiting a module deliberately breaks its connection and drives its outputs to their configured fault state.

All three drop I/O to real equipment. Do them with the machine in a safe state, in a planned window, with the people around it told — not while you are poking at a live line. Everything in steps 1, 2, 4, 5 and 6 is read-only and can be done any time.

Sort the fault into a family first

In Studio 5000: I/O Configuration tree → right-click the module → Module Properties → Connection tab. The Module Fault line prints the code and text, for example (Code 16#0203) Connection timed out. To get it onto an HMI, GSV the MODULE object's EntryStatus and FaultCode attributes into tags. For produced/consumed tags, if the first member of the shared UDT is of type CONNECTION_STATUS, the controller maintains its RunMode and ConnectionFaulted bits itself — user code cannot write them.

Sort by mechanism, not by number range.

Family A — never established. The Forward_Open was rejected and the target returned an extended status. Keying mismatch, wrong data size, unconfigured target, no free connections, ownership conflict, no bandwidth, link down. Data never flowed.

Family A splits again, and the split decides where you look:

• Deterministic rejections — 16#0114, 16#0115, 16#0116 (electronic keying), 16#0126, 16#0127, 16#0128 (sizes), 16#0110 (target not configured). These are wrong the same way every power cycle. It is a configuration or a part-number problem. The network is not involved.
• Resource and timing rejections — 16#0100, 16#0113, 16#011A, 16#0302. These depend on what else happens to be connected at that instant, so they can be intermittent. 16#0100 often clears by itself once the target ages out a stale connection. 16#0113 appears the day someone adds an HMI or a cached MSG. Count connections and look at loading, not at keying.
• 16#0800 (network link offline) is a physical-layer rejection. Start at the port.

Family B — established, then died. 16#0203. Data flowed, then stopped. Network, loading, power — or an RPI/multicast configuration the network cannot actually sustain. Usually intermittent, usually correlated with something else in the plant.

16#0204 is its own case: the request got no reply at all. The device is off, at a different IP, unreachable, the CIP path in the configuration is wrong, or TCP 44818 is blocked between you and it.

Step-by-step

1. Read the code and write it down. Connection tab in Studio 5000, or the equivalent diagnostic page in your scanner. Do not skip to hardware. (Read-only.)

1. Read the device's network status LED, and settle duplicate IP here. ODVA's Recommended Functionality for EtherNet/IP Devices (PUB00070) defines it, and most conforming devices follow it — confirm against the device manual, because third-party adapters do deviate. Off = no power or no IP; flashing green = has an IP but no connections established; solid green = at least one connection established; flashing red = one or more connections timed out; solid red = duplicate IP address detected.

If it is solid red, or two devices are flapping together, stop and resolve the address conflict before measuring anything else. A duplicate IP produces exactly the intermittent 16#0203 you are chasing and invalidates every measurement in steps 5 through 8. (Read-only.)

1. Family A, deterministic rejection? Compare the configuration to the device. Check electronic keying (revision, product code, device type), the input/output connection sizes in bytes, and whether the requested RPI is inside the device's supported range. An adapter such as a 1734-AENT POINT I/O will refuse a size that does not match its actual module list. (Correcting this is an offline edit and a download — see the safety note above.)

1. Family A with an "out of connections" or ownership code? Count the connections. If the device's embedded web server is enabled, browse to its IP address — Rockwell EtherNet/IP bridge modules such as the 1756-EN2T serve diagnostic pages showing CIP connection counts against the device's own limit. Note that this web server is a configurable service and is routinely disabled as a security hardening measure; if it is off, use the module's Connections and Module Info tabs in Studio 5000 and the connection limit in the device manual instead. Small embedded devices — a PowerFlex 525 drive, a barcode scanner — support only a handful. Two scanners fighting for the same exclusive-owner connection gives you an ownership conflict; only one owner is allowed, everyone else must use a listen-only connection. (Read-only.)

1. Family B: check the physical layer counters, not the cable. In Studio 5000, go online and open the Port Diagnostics tab on the Ethernet module or controller port; the counters come from the CIP Ethernet Link object (class 16#F6). FCS errors and alignment errors climbing means noise, a bad crimp, or an over-length run. Late collisions mean a duplex mismatch — one end forced, the other auto-negotiating — or an over-length segment. (Read-only. Note that clearing the counters is itself a change; record the values first.)

1. Budget the traffic. An exclusive-owner class 1 connection sends roughly two packets per RPI, one in each direction — about 2000 ÷ RPI(ms) packets per second. Rack-optimised and listen-only connections change that count, and a multicast producer sends one stream regardless of how many consumers subscribe, so count connections rather than devices. Twenty exclusive-owner connections at a 10 ms RPI is a very different network from twenty at 100 ms. Compare the total against the packet-rate capacity in the scanner module's technical data, and against what the field device can actually service. (Read-only.)

1. Slow the RPI down and see if it holds. Set it to something the machine can live with, not the fastest number the dialog accepts. A drive that publishes speed feedback does not need 2 ms. This is an offline edit and a download — machine safe, planned window.

1. If the connection is multicast, check IGMP. On the Connection tab, Studio 5000 has offered a Use Unicast Connection over EtherNet/IP checkbox since RSLogix 5000 v18. Unicast removes the whole multicast question for single-consumer I/O. Multicast remains required for listen-only connections. It is not required for produced tags with several consumers — since v18 a unicast produced tag serves multiple consumers (Max Consumers, 1 to 255), each on its own point-to-point connection — but multicast is far more efficient there, because it is one packet stream instead of one connection per consumer. Changing this checkbox is a download.

Multicast, IGMP snooping, and the querier

Multicast I/O sends one copy that the switch fabric replicates. Devices allocate their group addresses from the administratively scoped 239.192.x.x range, configurable through the CIP TCP/IP Interface Object (class 16#F5).

A plain unmanaged switch floods multicast to every port — every device sees every other device's I/O, and a large cell drowns. (Some modern "unmanaged" industrial switches do run snooping with an automatic querier; check the datasheet rather than assuming.) A managed switch with IGMP snooping listens to join messages and forwards each group only to ports that asked for it. But snooping needs an IGMP querier on that VLAN to periodically re-ask who is still listening. Snooping enabled with no querier is the classic trap: memberships age out, the switch stops forwarding, and every multicast connection times out together with 16#0203. On a Stratix 5700 or any managed industrial switch, enable snooping and designate exactly one querier per VLAN.

Never do this

• Do not reach for "disable IGMP snooping" as the fix for a multicast drop. It works, because it restores forwarding by flooding — and that is the problem. You have told the switch to flood every I/O group to every port, including uplinks to other cells and to IT. The failure comes back later, bigger, on machines nobody was working on, and nobody connects it to your change. Fix the querier, or move that connection to unicast. Flooding is defensible only on a small, isolated cell segment with no uplink, and even then it should be a deliberate decision, not a reflex.
• Do not set the RPI to the minimum the dialog allows. Faster is not safer. It multiplies packets per second on the wire, shortens the watchdog to a window a normal network hiccup will exceed, and can overrun the field device's own processor.
• Do not force speed and duplex on one side only. Force both ends or auto-negotiate both ends. A half-duplex/full-duplex mismatch passes a ping test and destroys implicit I/O.
• Do not clear the fault by inhibiting and un-inhibiting the module and then leave. Inhibiting is not a diagnostic — it breaks the connection deliberately and drives that module's outputs to their configured fault state, on real equipment. It proves nothing and it loses the evidence.
• Do not swap hardware before reading the fault code. A keying or connection-size rejection follows the new part straight out of the box.