Virtual Chassis Fabric: configuring an AE interface
Connecting other parts of the network to the VCF in a redundant way using Link Aggregation Groups (LAG) is very easy. A LAG can combine several Ethernet interfaces into a single logical link called an Aggregate Ethernet (AE) interface. When you are running a VCF, you’d best spread a LAG across multiple member switches. This way, during an NSSU or during the loss of 1 node, everything in your network will keep on running.
Before we can get into the configuration of the AE interface itself, we need to start by enabling the VCF to support AE interfaces. This is done in the [chassis aggregated-devices] stanza:
set chassis aggregated-devices ethernet device-count 20
This command enables the VCF to support 20 AE interfaces. Without this configuration statement, configured AE interfaces will not be usable.
After enabling the AE interface, we can now move on to the configuration of the interface itself. Let’s create a 20Gbit AE interface that is spread across members 3 and 6:
After preparing the VCF for AE interfaces, let’s assign the physical interfaces to the proper LAG:
delete interfaces xe-3/0/0 set interfaces xe-3/0/0 description to_some_other_switch set interfaces xe-3/0/0 ether-options 802.3ad ae0 delete interfaces xe-6/0/0 set interfaces xe-6/0/0 description to_some_other_switch set interfaces xe-6/0/0 ether-options 802.3ad ae0
Because the interfaces that are part of an AE cannot have other options configured, using the ‘delete interfaces’ command quickly removes all other configuration. The only things that needs to be configured for the physical interface is the ‘ether-options 802.3ad’ statement. This contains a reference to the AE interface that the physical interface will belong to. In our case, this is AE0, since this will be the logical interface we want to start using.
After assigning the physical interfaces to the AE, we can move on to the AE interface itself. Let’s create a trunk interface and make the individual physical interfaces send out LACP messages:
set interfaces ae0 description to_some_other_switch set interfaces ae0 aggregated-ether-options link-speed 10g set interfaces ae0 aggregated-ether-options lacp active set interfaces ae0 unit 0 family ethernet-switching interface-mode trunk set interfaces ae0 unit 0 family ethernet-switching vlan members 100
The above configuration will also work without the ‘aggregated-ether-options lacp’ statement. In that case, the AE would have been a static LAG and there would be no exchange of LACP control packets.
To verify that our LAG is working, we can issue the following command:
play@VCF> show interfaces ae0 extensive Physical interface: ae0, Enabled, Physical link is Up Interface index: 840, SNMP ifIndex: 682, Generation: 663 Description: to_some_other_switch Link-level type: Ethernet, MTU: 1514, Speed: 20Gbps, BPDU Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled, Flow control: Disabled, Minimum links needed: 1, Minimum bandwidth needed: 0 Device flags : Present Running Interface flags: SNMP-Traps Internal: 0x4000 Current address: f0:1c:2d:48:a8:0d, Hardware address: f0:1c:2d:48:a8:0d Last flapped : 2015-03-04 16:38:02 CET (16w4d 21:56 ago) Statistics last cleared: Never Traffic statistics: Input bytes : 1816375890 3624 bps Output bytes : 1499782977 3432 bps Input packets: 17792202 3 pps Output packets: 13917070 3 pps IPv6 transit statistics: Input bytes : 0 Output bytes : 0 Input packets: 0 Output packets: 0 Input errors: Errors: 0, Drops: 0, Framing errors: 0, Runts: 0, Giants: 0, Policed discards: 0, Resource errors: 0 Output errors: Carrier transitions: 2, Errors: 0, Drops: 0, MTU errors: 0, Resource errors: 0 Egress queues: 12 supported, 5 in use Queue counters: Queued packets Transmitted packets Dropped packets 0 best-effort 0 7117509 0 3 fcoe 0 0 0 4 no-loss 0 0 0 7 network-cont 0 6794564 0 8 mcast 0 5008 0 Queue number: Mapped forwarding classes 0 best-effort 3 fcoe 4 no-loss 7 network-control 8 mcast Logical interface ae0.0 (Index 589) (SNMP ifIndex 695) (Generation 252) Flags: SNMP-Traps 0x20024000 Encapsulation: Ethernet-Bridge Statistics Packets pps Bytes bps Bundle: Input : 3353214937388021592 0 5059842287152 0 Output: 5046586572800 0 1184550924416841968 0 Link: xe-3/0/0.0 Input : 0 0 0 0 Output: 0 0 0 0 xe-6/0/0.0 Input : 0 0 0 0 Output: 0 0 0 0 LACP info: Role System System Port Port Port priority identifier priority number key xe-3/0/0.0 Actor 127 f0:1c:2d:48:a7:00 127 12 14 xe-3/0/0.0 Partner 127 2c:21:72:a3:70:40 127 21 11 xe-6/0/0.0 Actor 127 f0:1c:2d:48:a7:00 127 13 14 xe-6/0/0.0 Partner 127 2c:21:72:a3:70:40 127 22 11 LACP Statistics: LACP Rx LACP Tx Unknown Rx Illegal Rx xe-3/0/0.0 3402935 3394454 0 0 xe-6/0/0.0 3402969 3394276 0 0 Marker Statistics: Marker Rx Resp Tx Unknown Rx Illegal Rx xe-3/0/0.0 0 0 0 0 xe-6/0/0.0 0 0 0 0 Protocol eth-switch, MTU: 1514, Generation: 292, Route table: 6 Flags: Trunk-Mode
The more interesting things to look for are the LACP info and the LACP statistics. LACP info displays the ‘Actor’ (local device) and the ‘Partner’ (remote device). The LACP statistics will show you how many LACP hellos have been send and received.
Anyway, this redundant interface is acting as a single 20Gbit link. It will now remain active as a 10Gbit link when a single member fails or whenever you initiate an NSSU (during an NSSU, individual members are rebooted one by one).
Another important thing to realize is that by default, the VCF uses the ‘layer2-payload’ hash mode. This means that IPv4 and IPv6 payload fields are used to hash traffic on the LAG. You can verify the current hash setting by issuing the ‘show forwarding-options enhanced-hash-key’ command. By fiddling about in the [ forwarding-options enhanced-hash-key ] configuration stanza, you can alter the default hashing behavior. The VCF has a lot of options that you can alter.
The links between the individual VCF members (VCP’s) can also be bundled into a LAG. This will happen automatically when same-speed VCP links are configured between two devices. Load-sharing will also be active across these VCP LAGs.