JNCIS-SP: BGP lab.
The JNCI-SP BGP lab looks as follows:
The lab setup.
This BGP lab is set up in such a way that it simulates (to a certain extent) a small ISP environment. The ISP will run OSPF in its core network. OSPF will make sure all the ISPs routers will have connectivity and that all loopback addresses are reachable. By using this OSPF network, R6 and R7 will run IBGP between their loopback IP addresses. There is an Internet exchange present, LAB-IX. Like normal IX-es, LAB-IX is a flat layer 2 environment that companies can use to establish BGP neighbor relationships. On this LAB-IX, our ISP will be peering with two different companies. The LAB-IX will also be connected to both core routers. There are also 2 customers present. Customer 1 is a multihomed customer that is using a private AS. Customer 2 will have its own AS and will advertise is own PI.
What this post will cover:
• the basic configuration and concepts of BGP
• EBGP and IBGP
• policies and manipulation of path attributes
• some troubleshooting and verification commands will be woven throughout this article
I'll start out by confguring OSPF and IBGP for the core. Afterwards, I will proceed to connect the core to LAB-IX to exchange routes with the peering partners. Then we will configure 2 customers. Customer 1 is going for the multihomed scenario and customer 2 is bringing a public AS and a PI. I'll wrap things up with some basic troubleshooting and verification commands.
OSPF and IBGP for the core.
The type of BGP neighbor relationship used will be IBGP. Normally, in a service provider network, you'll find that every rotuer has an IBGP session with several Route-Reflektors. Since route-reflection is not in the JNCIS-SP exam I'll skip that. Also, since we only have two routers, it is useless anyway.
A route-reflektor is not bound by the rule that prohibits IBGP from advertising IBGP routes to other IBGP peers. In this scenario however, we only have two IBGP routers.
First, before we start configuring the IBGP neighbor relationship, let's enable OSPF. After configuring R6, I performed the same configuration on R7 ;
The ospf neighbor relationship is up and there is connectivity between R6 and R7's loopback IPs. The next thing is to configure a basic BGP neighbor relationship between the routers.
I have configure R6 allready. This router is allready trying to form a neighbor relationship with R7. However, since R7 isn't configured yet, the state is active;
Besides the IP addresses, R6 and R7 have the exact same configuration BGP-wise. To configure R7 and verify that the BGP state, let's perform the following;
Configurationwise, the following was done;
• R7 was given an autonomous system under [routing-options]
• R7 was configured to use it's loopback IP address as the source address for the TCP session upon which any BGP neighbor relationship will be build
• the group 'IBGP' was created and we set the neighbor type to 'internal', meaning all neighbors must be in the same AS
• the R6 router was configured to be the neighbor
A few seconds after the commit, the command 'show bgp neighbor' was issued. This command will tell you just about anything you need to know about the session parameters and status. I highlighted some interesting things;
• the first line shows the neighbors IP and AS as well as this routers source IP (for this session) and AS
• the state of the neighbor is shown, currently it is 'Established', which means we are up and running
• something worth noting for people familiar with other vendors, the preference of IBGP and EBGP in Juniper is the same; its 170
• the NLRIs advertised by both routers are shown. Since BGP is multi-protocol, this can varry. In our case, its just IPv4 unicast addresses
• last marking shows the number of routes exchanged, which is currently 0
The number of routes is 0. BGP will only start advertising routes after a policy is configured telling BGP to do so or after the router has an EBGP session through which it is learning routes.
If R7 would have and IBGP session through which routes were learned, these routes would be advertised to R6. So let's go ahead an do just that
Connecting to LAB-IX.
An Internet Exchange is a place where service providers setup their routers and connect to each other in order to exchange their prefixes. It is becoming more and more coming for big companies to join the service providers on these IX-es. For instance, on the AMS-IX website, you can see the connected networks. Currently, among the connected websites are several large companies such as Facebook and Google.
The AMS-IX is a neutral and independent Internet Exchange. Anyone can become a member, get a public IP address and ask other members to establish a peering with them.
You can send an e-mail to the mailing list asking all the peers in one e-mail to connect with your network. The mailing list is also used for planned work and outages. The funnies request I received from a new AMS-IX member was the following:
People of Earth:
we are MANDA, ASxxx and we are peering with IP addresses x.x.x.x and xxxx:xxxx:x::x on AMS-IX.
To facilitate our network growth and the unrestricted flow of cat videos we will upgrade our AMS-IX connection today between 11:00 and 11:30 CET. You will see our BGP sessions flap once during this maintenance window.
Sorry for the short notice and any inconvenience this may cause you or your cat videos,
Being able to does not mean that every partner will want to peer with you if you setup shop though. If companies do not want to peer with you, and in general because not everyone in the world is on a single IX, you will probably need a connection to a company offering IP transits.
IP transits are ususally the bigger networking companies (Cogent, KPN, etc.) that have connectivity to all IX-es and they can handle all the traffic for which you will not have a destination route. At least, they should be able to.
Let's proceed with the lab and configure an EBGP session. Let's assume we have received an e-mail wanting to peer with us. Usually, they will supply you with an IP address, an AS and a list of their prefixes. In our case, the company send us the peering address 192.168.1.1, AS3 and prefix 188.8.131.52/24. We replied with 'Yes of course', our IP address, our AS, our prefixes and we suggested to use the password 'XXX'
The steps on R7 will be the following:
• configure an EBGP group with import and export policies (this way you can use them for all IX members
• configure the neighbor using the information exchanged via mail
Let's start with the EBGP neighbor relationship first.
We can see the peer up almost immediately. The EBGP peer R5 allready has policies in place so before configuring our own policies, let's examine the prefix they are advertising.
We can see that the prefix they own is allready in our routing table. The next-hop is the neighbors address used in the BGP session as source-address. The second command issued reveals all the routes that the peer has sent to us. In this case, they sent use only 1 prefix.
At the bottom, we see that R7 is allready advertising the prefix towards R6. Since the route learned was learned through IBGP, R7 can advertise this to IBGP neighbor R6.
If R6 would have an IBGP neighbor as well, R6 would not tell that neighbor about 184.108.40.206/24.
Learning the route towards their prefix is not enough to establish connectivity though, they need a route back to our routes as well. We are able to ping their prefix from R7 using the prefix active on the LAB-IX segment, but that's all.
We will need to create and apply export policy to advertise our range (220.127.116.11/24 and 18.104.22.168/8) before we can send packets to and fro. Let's do that next.
In the output above, we created and applied a policy that redistributes all static routes. We applied it to the group since we want to advertise the prefix to peers we will create in the future as well.
You can notice two things from this output:
• the next-hop IP address is the address that R7 uses as source address to peer with (this is always the case with EBGP)
• R7 inserted the AS after advertising the route (upon issuing 'show route advertising-protocol bgp 192.168.1.1' on R7, the AS is not there yet)
So we can ping to 22.214.171.124/32 using R7's loopback address and R7 is advertising the 126.96.36.199/24 prefix to R6. All is well right?
Well, not exactly. When we go to R6, we can see that the route is hidden:
When a route is advertised across an EBGP neighbor relationship, the next-hop of the route is changed. In this case, R7 changes the next-hop to 192.168.1.1, which is the address used as source address for the EBGP session by R5. This next-hop address, 192.168.1.1, is not in R6's routing table. So when R6 receives a route that has 192.168.1.1 as a next-hop, the route becomes hidden.
As can be observed in the previous output, by configuring a policy on R7, we can change the next-hop of the route to the IP address of R7. This way, R6 does not need to have the IP address of R5 in its routing table before it is able to reach the prefixes advertised by R5.
Let’s take a look on R6.
In the output above, we can see the following things happening:
• the 188.8.131.52/24 route is received from R7 and the next-hop is R7’s loopback IP
• there is connectivity to 184.108.40.206 from R6’s loopback IP address
• the route is installed in the forwarding table with an ‘Indirect’ next-hop type
That last part is interesting. In the BGP table, the route to 220.127.116.11/24 has a protocol next hop that is defined by the BGP path attribute next-hop. This protocol next-hop is used by Junos to do a recursive lookup in the routing table.
In our scenario, the routing table has the 18.104.22.168 installed in the forwarding table because the route was learned via OSPF. Hence, through the recursive lookup, the forwarding-table entry of the next-hop to 22.214.171.124 (which is learned via OSPF and has the value of 126.96.36.199) will be used to forward all traffic towards 188.8.131.52.
On R6, we then have to configure the same thing as we did on R7. Afterwards, we have connectivity between the prefixes of AS1 and AS2. Observe the following:
This shows there is connectivity. There is one important thing to note though. Check the Advertised prefixes. We are sending 3 prefixes to R8, even though we only have 2 prefixes ourselves. We did not implement any restrictions on our export-policy, so what is happing now is the following. We are;
• advertising our own prefixes by redistributing them into BGP
• learning two external prefixes from two different peers
• re-advertising the prefixes we learned to our peers as well
This means we have become sore of an IP-transit. This can be observed by performing a traceroute on R8:
Of course, we do not want to transit traffic for other service providers. If we would have our own IP-transit, and we would advertise the full internet routing table to our peers, they could start to use us as an IP transit. This would only be the case if the peer has no filtering in place. A far more likely scenario is that the peer will shut the BGP-session towards our AS when we send too many prefixes, or prefixes not belonging to our AS.
Anyway, let’s solve this by creating a prefix-list which prohibits the advertisement of routes not belonging to our AS. Let’s redo our policy in the following way:
By changing our policy in this way, we will:
• announce our prefixes if they are present in the routing table and the prefix-list
• no longer announce routes learned from other peers
After performing the same change on R6, we will not be used as an IP-transit and we will only have to update a prefix-list when we start announcing other PI’s besides the current ones. Please not that this is just one of many security measures you should take, import policy and prefix-limit amongst others. Besides that, this is not exactly a decent export policy either. Much more is possible. Please don’t go and soil any IX based on this example lab but use this lab only as an example.
Also note that the Internet feed we are offering in this lab is rather limited. I just checked my companies live IP transits to see how many IPv4 prefixes they are sending. Currently, 13-8-2014, they are sending the following amount of routes:
• Telia: 498778
• KPN: 500618
• Cogent: 497790
• Atom: 465174
Besides these IP transits, there are also hundreds of direct peers with whom we currently exchange routing information. This is quite a lot of information to process, so in a real world scenario, make sure you have some routers that are up to the task.
After taking care of the IX-part, we now have connectivity to offer to other companies as well. So let’s connect the first customer.
The customer does not have an AS or a PI. They want to create a multihomed scenario with an IP address from the AS1 range. In order to achieve this, we will perform the following:
• create EBGP sessions with the customer routers
• advertise a default route to the customers routers
• ensure that the customer will only be allowed to advertise the IP address that the customer was allocated
• create an import policy to ensure R9 is the primary router for outbound traffic (from AS1 point-of view)
• create an export policy to ensure R9 receives a default route that is more interesting than the one advertised to R2
First, we create a static default route that will enable us to redistribute the default route to the customer.
Second, we have to create two export policies. In these export policies, we restrict route advertisement to the default route only. Another thing we will do is alter the MED. The MED (Multi-Exit Discriminator) is a BGP path attribute used to influence route-selection in a downstream AS. The lowest MED will win. R6 will be the primary router, advertising the default route with a MED of 75. R7 will be the secondary router, advertising the default route with a MED of 150.
Third, we will created two import policy. This policy will reject all prefixes except for the one allocated to the customer. This policy will also set the local preference. With the local preference attribute, you can determine the best exit path to a prefix. This is AS-wide and the highest will win. R6 will advertise the default route with a local preference of 150 and R2 will advertise the default route with a local preference of 75.
Let's start with the policy and BGP configuration on R6 after creating the static default route;
Afterwards, we can observe the following:
So, everything checks out on R9 and R6. We are learning the customer route with the right local preference and we are sending the default route with the MED value of 75. So, let's move on the customers secondary site. The configuration on R7 will be as follows:
The end result of the configuration is displayed below:
• learning the route from the secondary location, but installed the route learned from R6. This is due to the higher preference.
• sending the MED with a value of 150. If the customer respects the MED (the customer can ignore it or rewrite it), the customer will use the default route towards R6.
Normally, as-override would also be used. As-override will make the ISP routes swap the private AS for their own AS. Since this is out-of-scope, maybe I'll dig into it some more when I'm preparing for JNCIP.
Customer 2 has become a member of the RIPE NCC. RIPE is one of five Regional Internet Registries (RIR).
RIPE NCC stands for Réseaux IP Européens Network Coördination Centre.
One of the things they do, or the most important thing they do, is that they provide global Internet resources and related services. These include IPv4, IPv6 and AS-numbers.
Customer 2 contacted this organization and requested to become a LIR, Local Internet Registry. They received their own public AS (AS4) and they have received their own /22 prefix (184.108.40.206/22). They then contacted us for an Internet-feed. They will also create a multihomed scenario, but the second Internet feed will be a BGP session with another company. For them, it will look something like this:
Anyway, the only things we will need to do are the following:
• create an EBGP neighbor relationship with this customer
• create and apply import and export policies for this customer
• make sure the customers prefix is announced to all our peers
Let’s move over to R7 and perform our side of the configuration for this customer;
We are receiving the customers prefix and we are advertising the default route. If they have their policies and routing setup in order, we should have connectivity. Let’s check it out:
As far as connectivity between AS1 and AS4 is concerned, it seems to be ok.
However, the policies we initially created for the BGP sessions with the LAB-IX routers are not sufficient anymore. They do not allow us, AS1, to advertise prefixes other than our own yet.
So let’s alter the policy on both R6 and R7 and extend it with a term that will allow us to advertise prefixes from other customers as well.
This can be done in the following way:
If we take a look on the peering partners R5 router, we can see that they learned the prefix via our AS.
As soon as the customer would connect via another upstream ISP, the peering partner would probably see two routes towards the same prefix.
If the peering partner would prefer one ISP over the other, they could attempt to influence the return traffic through the use of AS-path prepending. What they could do is change the number of times their AS is prepended to the ISP they would like to be secondary. That ISP would learn and advertise a route with a much longer AS path marking the route less attractive. I said attempt, because the AS path attribute is not the first thing BGP looks at. Other upstream operators could manipulate their return path via local preference and send traffic to AS4 via the secondary connection.
There is a second thing customer 2 would have to do in order to make sure that one of the ISP’s is primary. That would be changing the local preference of the default route they are learning. The primary ISP’s default route should have a higher local preference than the secondary ISP’s default route.
Finally, let’s verify the connectivity from our other peering partner R8;
Well, as far as JNCIS-SP BGP is concerned that’s about it I think. This complemented with the theory of course. Real world ISP scenario’s tend to be a bit more complex though. Anyway, hopefully this gives anyone reading/trying it a bit more insight as far as the general principles of BGP are concerned.