Virtual Chassis Fabric: configuring an AE interface

Connecting other parts of the network to the VCF in a redundant way using Link Aggregation Groups (LAG) is very easy. A LAG can combine several Ethernet interfaces into a single logical link called an Aggregate Ethernet (AE) interface. When you are running a VCF, you’d best spread a LAG across multiple member switches. This way, during an NSSU or during the loss of 1 node, everything in your network will keep on running.

Before we can get into the configuration of the AE interface itself, we need to start by enabling the VCF to support AE interfaces. This is done in the [chassis aggregated-devices] stanza:

set chassis aggregated-devices ethernet device-count 20

This command enables the VCF to support 20 AE interfaces. Without this configuration statement, configured AE interfaces will not be usable.

After enabling the AE interface, we can now move on to the configuration of the interface itself. Let’s create a 20Gbit AE interface that is spread across members 3 and 6:

After preparing the VCF for AE interfaces, let’s assign the physical interfaces to the proper LAG:

delete interfaces xe-3/0/0
set interfaces xe-3/0/0 description to_some_other_switch
set interfaces xe-3/0/0 ether-options 802.3ad ae0

delete interfaces xe-6/0/0
set interfaces xe-6/0/0 description to_some_other_switch
set interfaces xe-6/0/0 ether-options 802.3ad ae0

Because the interfaces that are part of an AE cannot have other options configured, using the ‘delete interfaces’ command quickly removes all other configuration. The only things that needs to be configured for the physical interface is the ‘ether-options 802.3ad’ statement. This contains a reference to the AE interface that the physical interface will belong to. In our case, this is AE0, since this will be the logical interface we want to start using.

After assigning the physical interfaces to the AE, we can move on to the AE interface itself. Let’s create a trunk interface and make the individual physical interfaces send out LACP messages:

set interfaces ae0 description to_some_other_switch
set interfaces ae0 aggregated-ether-options link-speed 10g
set interfaces ae0 aggregated-ether-options lacp active
set interfaces ae0 unit 0 family ethernet-switching interface-mode trunk
set interfaces ae0 unit 0 family ethernet-switching vlan members 100

The above configuration will also work without the ‘aggregated-ether-options lacp’ statement. In that case, the AE would have been a static LAG and there would be no exchange of LACP control packets.

To verify that our LAG is working, we can issue the following command:

play@VCF> show interfaces ae0 extensive
Physical interface: ae0, Enabled, Physical link is Up
  Interface index: 840, SNMP ifIndex: 682, Generation: 663
  Description: to_some_other_switch
  Link-level type: Ethernet, MTU: 1514, Speed: 20Gbps, BPDU Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled, Flow control: Disabled, Minimum links needed: 1, Minimum bandwidth needed: 0
  Device flags   : Present Running
  Interface flags: SNMP-Traps Internal: 0x4000
  Current address: f0:1c:2d:48:a8:0d, Hardware address: f0:1c:2d:48:a8:0d
  Last flapped   : 2015-03-04 16:38:02 CET (16w4d 21:56 ago)
  Statistics last cleared: Never
  Traffic statistics:
   Input  bytes  :           1816375890                 3624 bps
   Output bytes  :           1499782977                 3432 bps
   Input  packets:             17792202                    3 pps
   Output packets:             13917070                    3 pps
   IPv6 transit statistics:
    Input  bytes  :                   0
    Output bytes  :                   0
    Input  packets:                   0
    Output packets:                   0
  Input errors:
    Errors: 0, Drops: 0, Framing errors: 0, Runts: 0, Giants: 0, Policed discards: 0, Resource errors: 0
  Output errors:
    Carrier transitions: 2, Errors: 0, Drops: 0, MTU errors: 0, Resource errors: 0
  Egress queues: 12 supported, 5 in use
  Queue counters:       Queued packets  Transmitted packets      Dropped packets
    0 best-effort                    0              7117509                    0
    3 fcoe                           0                    0                    0
    4 no-loss                        0                    0                    0
    7 network-cont                   0              6794564                    0
    8 mcast                          0                 5008                    0
  Queue number:         Mapped forwarding classes
    0                   best-effort
    3                   fcoe
    4                   no-loss
    7                   network-control
    8                   mcast

  Logical interface ae0.0 (Index 589) (SNMP ifIndex 695) (Generation 252)
    Flags: SNMP-Traps 0x20024000 Encapsulation: Ethernet-Bridge
    Statistics        Packets        pps         Bytes          bps
        Input : 3353214937388021592          0 5059842287152            0
        Output: 5046586572800          0 1184550924416841968            0
        Input :             0          0             0            0
        Output:             0          0             0            0
        Input :             0          0             0            0
        Output:             0          0             0            0
    LACP info:        Role     System             System      Port    Port  Port
                             priority          identifier  priority  number   key
      xe-3/0/0.0     Actor        127  f0:1c:2d:48:a7:00       127      12    14
      xe-3/0/0.0   Partner        127  2c:21:72:a3:70:40       127      21    11
      xe-6/0/0.0     Actor        127  f0:1c:2d:48:a7:00       127      13    14
      xe-6/0/0.0   Partner        127  2c:21:72:a3:70:40       127      22    11
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx
      xe-3/0/0.0           3402935     3394454            0            0
      xe-6/0/0.0           3402969     3394276            0            0
    Marker Statistics:   Marker Rx     Resp Tx   Unknown Rx   Illegal Rx
      xe-3/0/0.0                 0           0            0            0
      xe-6/0/0.0                 0           0            0            0
    Protocol eth-switch, MTU: 1514, Generation: 292, Route table: 6
      Flags: Trunk-Mode

The more interesting things to look for are the LACP info and the LACP statistics. LACP info displays the ‘Actor’ (local device) and the ‘Partner’ (remote device). The LACP statistics will show you how many LACP hellos have been send and received.

Anyway, this redundant interface is acting as a single 20Gbit link. It will now remain active as a 10Gbit link when a single member fails or whenever you initiate an NSSU (during an NSSU, individual members are rebooted one by one).

Another important thing to realize is that by default, the VCF uses the ‘layer2-payload’ hash mode. This means that IPv4 and IPv6 payload fields are used to hash traffic on the LAG. You can verify the current hash setting by issuing the ‘show forwarding-options enhanced-hash-key’ command. By fiddling about in the [ forwarding-options enhanced-hash-key ] configuration stanza, you can alter the default hashing behavior. The VCF has a lot of options that you can alter.

The links between the individual VCF members (VCP’s) can also be bundled into a LAG. This will happen automatically when same-speed VCP links are configured between two devices. Load-sharing will also be active across these VCP LAGs.