I’m no networking expert, but I’ve discussed this several times with my friend and colleague Low Soon Seng, who is a double CCIE, and a Technical Trainer at Nutanix. Thank you Soon Seng, I have learnt a lot from you.
First my pet peeve; that many people mistakenly use the term LACP, to mean Link Aggregation. It’s not the same thing. LACP (Link Aggregation Control Protocol) is simply a handshaking protocol between two connected devices to determine if both ends support Link Aggregation. If yes, enable Link Aggregation, if not, don’t.
At this point, try to be clear about what is Link Aggregation. Among the network vendors, there are so many different vendor specific terms. Which is probably why it has confused so many. As an IT infrastructure professional, be it a system person, and even more so a networking person, you got to be very clear on what these are. I find that this wiki page helps https://en.m.wikipedia.org/wiki/Link_aggregation
I’ll attempt to throw in my variation of it, and what you ought to be aware of.
- Link Aggregation, would probably be the most generic term; and examples of the other synonyms would be Port Aggregation, Port Channeling, Etherchannel, LAG (Link Aggregation Group), Bundling, Port Trunking, etc.
- most networking professionals I’ve met call this the active-active connection. From a system’s engineer, there are other network teaming/bonding techniques which can also be conidered active-active. (More about this next time.)
- Essentially, Link Aggregation is the pairing of two or more direct network connections between two devices in order to achieve increased overall bandwidth and high availability of the connection.
With Link Aggregation, there are two types, static and dynamic. For the systems folks, this is what both types means, and the implications to your configuration on the system end
- Static: this is where the configuration at the switch end has assumed the connections are permanently a bundle, regardless if the other end (the server) is ready.
- Dynamic: this is where the switch will attempt to negotiate with the server using LACP to determine if both sides are capable to allow the network connections to form a bundle.
As you can probably imagine, the dynamic mode is preferred as if negotiation fails, the connections can be configured to have a fall back mode. More often, the preferred fall back is just to let the links be just individual links.
More often than not, due to lack of cross domain knowledge, there are common mistakes made when setting up Link Aggregation between a server and network device. Here are some common dependencies and misunderstanding.
- just like on the server end, the switch ports connected to the server NICs that are to form a bundle must be grouped specifically for the purpose. This adds an extra area of attention where cabling between the server and switch must be spot on. If there is any mistake made in the physical connectivity, there will be issues.
- Going past physical connectivity, you will have to ensure the load balancing algorithm configured on the system end is compatible as well.
A single mistake made in the cabling, and/or the configuration on either end will result in very inconsistent network behavior. Troubleshooting will also not be simple.
Wrapping up, the main point I’d like to drive is awareness of the terminologies, LACP and Link Aggregation are related, but not the same thing. You can have Link Aggregation without LACP being enabled. Configuration wise, there are significant difference between the two. Secondarily, the set up is complex and demands advanced skillsets, as well as mature testing techniques to have a stable set up.
I’ll leave it to a future posts to dive into the details about configuration and testing.