QFX5100 Layer 3 VPN

For future use cases, I was interested in knowing how a Layer 3 VPN (L3VPN) would behave on a QFX5100. For this reason, I created the following lab;



In this lab, Aether is the QFX5100 that will connect the Gaius CPE to the rest of the L3VPN. Aether, running IS-IS, will be configured with RSVP-signaled LSP’s towards all the other PE routers. The QFX (and the other PE’s) will be configured to exchange routing information with a route-reflector. I’ll wrap up with the configuration of a routing-instance and verify the connectivity in the L3VPN.

Let’s start with the configuration of the interfaces that connect the QFX to the rest of the network;



These interfaces need to be enabled for IS-IS, RSVP and MPLS. We'll begin with the configuration of the interfaces and IS-IS;

set interfaces ge-0/0/5 unit 0 family ethernet-switching interface-mode trunk
set interfaces ge-0/0/5 unit 0 family ethernet-switching vlan members 1250

set interfaces ge-0/0/6 unit 0 family ethernet-switching interface-mode trunk
set interfaces ge-0/0/6 unit 0 family ethernet-switching vlan members 1250

set interfaces lo0 unit 0 family inet address 1.1.1.254/32
set interfaces lo0 unit 0 family iso address 49.0010.0010.0100.1254.00

set interfaces irb unit 1250 family inet address 9.0.0.2/30
set interfaces irb unit 1250 family iso
set interfaces irb unit 1251 family inet address 9.0.0.6/30
set interfaces irb unit 1251 family iso

set protocols isis traffic-engineering family inet shortcuts
set protocols isis level 2 authentication-key "$9$VIbgaZGi.fzDiOREcvM"
set protocols isis level 2 authentication-type md5
set protocols isis level 2 wide-metrics-only
set protocols isis interface irb.1250 point-to-point
set protocols isis interface irb.1250 level 1 disable
set protocols isis interface irb.1251 point-to-point
set protocols isis interface irb.1251 level 1 disable
set protocols isis interface lo0.0 level 1 disable

set vlans VLAN_1250 vlan-id 1250
set vlans VLAN_1250 l3-interface irb.1250
set vlans VLAN_1251 vlan-id 1251
set vlans VLAN_1251 l3-interface irb.1251

To verify that the QFX is now adjacent to two routers, we can issue the following command:

play@Aether> show isis adjacency
Interface             System         L State        Hold (secs) SNPA
irb.1250              MX480-TEST-RE0-Romulus 2 Up            26
irb.1251              MX480-TEST-RE0-Caligula 2 Up           20

Let’s move on to the MPLS part and enable RSVP signaled LSPs from the QFX towards all the PE’s. To make this happen, we need to work our way through two other configuration stanza’s. These are RSVP and MPLS.

In the RSVP stanza, we can configure the IS-IS enabled interfaces for RSVP as follows:

set protocols rsvp interface irb.1250 authentication-key "$9$1R/ElM8LNYgJcyMX-boaP5QFCuKvL-dsBINbwYGU"
set protocols rsvp interface irb.1250 link-protection
set protocols rsvp interface irb.1251 authentication-key "$9$1R/ElM8LNYgJcyMX-boaP5QFCuKvL-dsBINbwYGU"
set protocols rsvp interface irb.1251 link-protection

The configuration above will enable the interfaces for RSVP and ensure that protocol exchanges are authenticated. Link-protection is enabled on all routers in the lab. This is not required, but I wanted to show you that the QFX supports this as well.

After enabling RSVP, we can move over to the MPLS part of the configuration. First thing will be enabling the interfaces for MPLS:

set interfaces irb unit 1250 family mpls
set interfaces irb unit 1251 family mpls

set protocols mpls interface irb.1250
set protocols mpls interface irb.1251

Proceeding in the mpls stanza, we need to configure the LSPs towards the three PE routers:



set protocols mpls label-switched-path to_Commodus to 1.1.1.4
set protocols mpls label-switched-path to_Commodus link-protection
set protocols mpls label-switched-path to_Mars to 1.1.1.20
set protocols mpls label-switched-path to_Mars link-protection
set protocols mpls label-switched-path to_Tiberius to 1.1.1.9
set protocols mpls label-switched-path to_Tiberius node-link-protection

Initially, I tried to configure a primary and a secondary standby path for all LSP’s. Since ‘weird’ stuff started to happen, I settled for an LSP with link-protection. Let’s see if our configuration brought us what we wanted, LSPs with link-protection;

play@Aether> show mpls lsp ingress
Ingress LSP: 3 sessions
To              From            State Rt P     ActivePath       LSPname
1.1.1.4         1.1.1.254       Up     0 *                      to_Commodus
1.1.1.9         1.1.1.254       Up     0 *                      to_Tiberius
1.1.1.20        1.1.1.254       Up     0 *                      to_Mars
Total 3 displayed, Up 3, Down 0

play@Aether> show mpls lsp name to_Mars extensive
Ingress LSP: 3 sessions

1.1.1.20
  From: 1.1.1.254, State: Up, ActiveRoute: 0, LSPname: to_Mars
  ActivePath:  (primary)
  Link protection desired
  LSPtype: Static Configured, Penultimate hop popping
  LoadBalance: Random
  Encoding type: Packet, Switching type: Packet, GPID: IPv4
 *Primary                    State: Up
    Priorities: 7 0
    SmartOptimizeTimer: 180
    Computed ERO (S [L] denotes strict [loose] hops): (CSPF metric: 30)
 9.0.0.5 S 2.0.0.17 S 6.0.0.5 S
    Received RRO (ProtectionFlag 1=Available 2=InUse 4=B/W 8=Node 10=SoftPreempt 20=Node-ID):
          1.1.1.8(flag=0x21) 9.0.0.5(flag=1 Label=301024) 1.1.1.12(flag=0x20) 2.0.0.17(Label=300816) 1.1.1.20(flag=0x20) 6.0.0.5(Label=3)
    6 Jul  3 08:15:19.537 Link-protection Up
    5 Jul  3 08:15:11.179 Selected as active path
    4 Jul  3 08:15:11.177 Record Route:  1.1.1.8(flag=0x21) 9.0.0.5(flag=1 Label=301024) 1.1.1.12(flag=0x20) 2.0.0.17(Label=300816) 1.1.1.20(flag=0x20) 6.0.0.5(Label=3)
    3 Jul  3 08:15:11.177 Up
    2 Jul  3 08:15:10.534 Originate Call
    1 Jul  3 08:15:10.534 CSPF: computation result accepted  9.0.0.5 2.0.0.17 6.0.0.5
  Created: Fri Jul  3 08:15:09 2015
Total 1 displayed, Up 1, Down 0

Egress LSP: 12 sessions
Total 0 displayed, Up 0, Down 0

Transit LSP: 0 sessions
Total 0 displayed, Up 0, Down 0

Next up is enabling BGP on the QFX. RSVP is taking care of the label exchange for the individual LSPs between the routers. BGP will be used to exchange VPN labels as well as VPN IPv4 address information:



Enabling BGP on the QFX:

set routing-options autonomous-system 1
set protocols bgp local-address 1.1.1.254
set protocols bgp out-delay 2
set protocols bgp log-updown
set protocols bgp group rr type internal
set protocols bgp group rr family inet-vpn unicast
set protocols bgp group rr authentication-key "$9$JsDiqTQ3n/ABIKWLNws"
set protocols bgp group rr peer-as 1
set protocols bgp group rr neighbor 1.1.1.1 description Aurelius_rr

To verify that the session is up, we can issue the following command:

play@Aether> show bgp summary
Groups: 1 Peers: 1 Down peers: 0
Table          Tot Paths  Act Paths Suppressed    History Damp State    Pending
bgp.l3vpn.0
                       0          0          0          0          0          0
Peer                     AS      InPkt     OutPkt    OutQ   Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped
1.1.1.1                   1         45         45       0       0       17:53 Establ
  bgp.l3vpn.0: 0/0/0/0

The session is up and Aether is ready to exchange L3VPN prefix information. We have only one last thing left to configure; the routing-instance.



The routing-instance on the QFX can be configured as follows:

set interfaces ge-2/0/11 unit 0 family ethernet-switching interface-mode access
set interfaces ge-2/0/11 unit 0 family ethernet-switching vlan members 1550

set interfaces lo0 unit 4000 family inet address 1.1.1.254/32
set interfaces irb unit 1550 family inet address 2.0.0.2/30

set routing-instances ipvpn instance-type vrf
set routing-instances ipvpn interface irb.1550
set routing-instances ipvpn interface lo0.4000
set routing-instances ipvpn route-distinguisher 1:1
set routing-instances ipvpn vrf-target target:1:1
set routing-instances ipvpn vrf-table-label
set routing-instances ipvpn routing-options static route 10.0.0.3/32 next-hop 2.0.0.1

set vlans VLAN_1550 vlan-id 1550
set vlans VLAN_1550 l3-interface irb.1550

This configuration will enable IP connectivity between the loopback of the CPE and the rest of the L3VPN.



From the CPE, can verify this with a simple ping:

play@MX480-TEST-RE0:Gaius> ping source 10.0.0.3 rapid 1.1.1.4
PING 1.1.1.4 (1.1.1.4): 56 data bytes
!!!!!
--- 1.1.1.4 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.739/3.034/11.687/4.330 ms

Let’s look at the more interesting verification commands on the QFX. First, let’s check the prefix information that was exchanged with the route-reflector. We’ll focus on the prefix information that was send to and received from the Tiberius PE (1.1.1.9) in this example:

play@Aether> show route advertising-protocol bgp 1.1.1.1 detail table ipvpn.inet.0

ipvpn.inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
* 1.1.1.254/32 (1 entry, 1 announced)
 BGP group rr type Internal
     Route Distinguisher: 1:1
     VPN Label: 16
     Nexthop: Self
     Flags: Nexthop Change
     Localpref: 100
     AS path: [1] I
     Communities: target:1:1

* 2.0.0.0/30 (1 entry, 1 announced)
 BGP group rr type Internal
     Route Distinguisher: 1:1
     VPN Label: 16
     Nexthop: Self
     Flags: Nexthop Change
     Localpref: 100
     AS path: [1] I
     Communities: target:1:1

* 10.0.0.3/32 (1 entry, 1 announced)
 BGP group rr type Internal
     Route Distinguisher: 1:1
     VPN Label: 16
     Nexthop: Self
     Flags: Nexthop Change
     Localpref: 100
     AS path: [1] I
     Communities: target:1:1

{master:0}
play@Aether> show route receive-protocol bgp 1.1.1.1 detail table ipvpn.inet.0 1.1.1.9

ipvpn.inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
* 1.1.1.9/32 (1 entry, 1 announced)
     Import Accepted
     Route Distinguisher: 1:1
     VPN Label: 16
     Nexthop: 1.1.1.9
     Localpref: 100
     AS path: I (Originator)
     Cluster list:  1.1.1.1
     Originator ID: 1.1.1.9
     Communities: target:1:1

Labels and VPN IPv4 prefix information was exchanged as expected. To see what label Aether will use to reach the Tiberius PE, we can issue the following command:

play@Aether> show route protocol rsvp 1.1.1.9 detail

inet.0: 39 destinations, 39 routes (37 active, 0 holddown, 2 hidden)

inet.3: 5 destinations, 8 routes (5 active, 0 holddown, 0 hidden)
1.1.1.9/32 (2 entries, 1 announced)
        State: <FlashAll>
        *RSVP   Preference: 7/1
                Next hop type: Router
                Address: 0x9390304
                Next-hop reference count: 3
                Next hop: 9.0.0.5 via irb.1251 weight 0x1, selected
                Label-switched-path to_Tiberius
                Label operation: Push 300928
                Label TTL action: prop-ttl
                Session Id: 0x3
                Next hop: 9.0.0.1 via irb.1250 weight 0x8001
                Label-switched-path Bypass->9.0.0.5->2.0.0.58
                Label operation: Push 300816, Push 300960(top)
                Label TTL action: prop-ttl, prop-ttl(top)
                Session Id: 0x2
                State: <Active Int>
                Local AS:     1
                Age: 1:39:14    Metric: 20
                Validation State: unverified
                Task: RSVP
                Announcement bits (1): 1-Resolve tree 1
                AS path: I

A quick look into the forwarding table of the QFX reveals the following:



play@Aether> show route forwarding-table vpn ipvpn destination 1.1.1.9
Routing table: ipvpn.inet
Internet:
Destination        Type RtRef Next hop           Type Index NhRef Netif
1.1.1.9/32         user     0                    indr 131075     2
                              9.0.0.5           Push 16, Push 300928(top)  1981     1 ge-0/0/6.0

Juniper is continuously adding and improving MPLS features for the QFX. For instance, added around Junos OS 13 or so was support for running MPLS on vlan interfaces. This is allowing for a lot more flexibility. For other scenario’s, the QFX also supports Layer 2 circuits. There is no BGP signaled VPLS, though EVPN over VXLAN seems to be on the roadmap.

5-7-2015