Cart
Welcome back everybody to another edition of IPexpert’s techtorial series! Today we will be looking at a topic that seems to be scaring some folks out there with regards to the R&S v4.0 blueprint – OSPF Sham Links. Now – Up until v4.0 this was pretty much always considered strictly a CCIE SP related topic. But, that was then and this is now. I’m not saying to definitely expect it, but I am saying it is likely “fair game” for the new exam, and it would probably nice if you at least get the general concept down. Let’s take a look at our base topology than shall we?
Alright, that doesn’t look so bad does it? Well…not if you already have read my MPLS L3 VPN blog heh… So what we have going on here is a simple MPLS L3 VPN setup. We have a customer, we’ll call them “Customer A” and they have two remote sites connected together via the services provider MPLS cloud here. Additionally, they have a direct “backdoor link” configured between them for backup purposes. The PE/CE routing protocol of choice here is OSPF.
To start things off, let’s disable the backdoor links and just see how things work normally. Also notice we are running the SAME OSPF process-ID on both PE routers for the customer VRF and that both customer sites are running in area 0 on all links. All the MPLS and redistribution has already been done here.
R1:
R1(config-router)#int fa0/1 R1(config-if)#shut
R7:
R7(config-router)#int fa0/1 R7(config-if)#shut
R2:
R2#sh ip proto vrf CustA | i Routing Protocol Routing Protocol is "ospf 2" Routing Protocol is "bgp 55"
R4:
R4(config)#do sh ip proto vrf CustA | i Routing Protocol Routing Protocol is "ospf 2" Routing Protocol is "bgp 55"
OK. So what do our routes look like on R1 and R7?
R1:
R1#sh ip ospf neigh Neighbor ID Pri State Dead Time Address Interface 10.0.12.2 1 FULL/BDR 00:00:38 10.0.12.2 FastEthernet0/0 R1#sh ip route ospf 7.0.0.0/32 is subnetted, 1 subnets O IA 7.7.7.7 [110/3] via 10.0.12.2, 00:04:35, FastEthernet0/0 10.0.0.0/8 is variably subnetted, 12 subnets, 2 masks O IA 10.0.47.0/24 [110/2] via 10.0.12.2, 00:04:35, FastEthernet0/0 O IA 10.0.204.7/32 [110/3] via 10.0.12.2, 00:04:35, FastEthernet0/0 O IA 10.0.201.7/32 [110/3] via 10.0.12.2, 00:04:35, FastEthernet0/0 O IA 10.0.200.7/32 [110/3] via 10.0.12.2, 00:04:35, FastEthernet0/0 O IA 10.0.203.7/32 [110/3] via 10.0.12.2, 00:04:35, FastEthernet0/0 O IA 10.0.202.7/32 [110/3] via 10.0.12.2, 00:04:35, FastEthernet0/0
R7:
R7(config-if)#do sh ip route ospf 1.0.0.0/32 is subnetted, 1 subnets O IA 1.1.1.1 [110/3] via 10.0.47.4, 00:04:25, FastEthernet0/0 10.0.0.0/8 is variably subnetted, 12 subnets, 2 masks O IA 10.0.12.0/24 [110/2] via 10.0.47.4, 00:04:25, FastEthernet0/0 O IA 10.0.104.1/32 [110/3] via 10.0.47.4, 00:04:25, FastEthernet0/0 O IA 10.0.103.1/32 [110/3] via 10.0.47.4, 00:04:25, FastEthernet0/0 O IA 10.0.102.1/32 [110/3] via 10.0.47.4, 00:04:25, FastEthernet0/0 O IA 10.0.101.1/32 [110/3] via 10.0.47.4, 00:04:25, FastEthernet0/0 O IA 10.0.100.1/32 [110/3] via 10.0.47.4, 00:04:25, FastEthernet0/0
Alright, wait a tick…the site-to-site routes are showing up as O IA INTER-area routes via type-3 summary LSAs. But…aren’t I in the same area, area 0? Well…”sort of” By default, if our OSPF domain-ID matches on both sides and we are transporting routes over MPLS we will get inter-area routes here. The domain-ID is the OSPF process ID by default, but can be changed. If they are not the same, we will see the routes come in as external O E2 routes. Let’s see that in action.
R4:
R4(config-router)#router ospf 2 R4(config-router)#domain-id 0.0.0.3
R7:
R7(config-if)#do sh ip route ospf 1.0.0.0/32 is subnetted, 1 subnets O E2 1.1.1.1 [110/2] via 10.0.47.4, 00:00:17, FastEthernet0/0 10.0.0.0/8 is variably subnetted, 12 subnets, 2 masks O E2 10.0.12.0/24 [110/1] via 10.0.47.4, 00:00:17, FastEthernet0/0 O E2 10.0.104.1/32 [110/2] via 10.0.47.4, 00:00:17, FastEthernet0/0 O E2 10.0.103.1/32 [110/2] via 10.0.47.4, 00:00:17, FastEthernet0/0 O E2 10.0.102.1/32 [110/2] via 10.0.47.4, 00:00:17, FastEthernet0/0 O E2 10.0.101.1/32 [110/2] via 10.0.47.4, 00:00:17, FastEthernet0/0 O E2 10.0.100.1/32 [110/2] via 10.0.47.4, 00:00:17, FastEthernet0/0
Wow, that is pretty cool! Let’s change it back now…
R4:
R4(config-router)#no domain-id 0.0.0.3
R7:
R7(config-if)#do sh ip route ospf 1.0.0.0/32 is subnetted, 1 subnets O IA 1.1.1.1 [110/3] via 10.0.47.4, 00:00:54, FastEthernet0/0 10.0.0.0/8 is variably subnetted, 12 subnets, 2 masks O IA 10.0.12.0/24 [110/2] via 10.0.47.4, 00:00:54, FastEthernet0/0 O IA 10.0.104.1/32 [110/3] via 10.0.47.4, 00:00:54, FastEthernet0/0 O IA 10.0.103.1/32 [110/3] via 10.0.47.4, 00:00:54, FastEthernet0/0 O IA 10.0.102.1/32 [110/3] via 10.0.47.4, 00:00:54, FastEthernet0/0 O IA 10.0.101.1/32 [110/3] via 10.0.47.4, 00:00:54, FastEthernet0/0 O IA 10.0.100.1/32 [110/3] via 10.0.47.4, 00:00:54, FastEthernet0/0
Alright…well that is neat and pretty cool and all, but we still don’t know what a sham-link is at this point! Let’s bring up that backup link. Notice that right now the MPLS path is the ONLY path we know traffic through.
R1:
R1(config)#int fa0/1 R1(config-if)#no shut
R7:
R7(config-if)#int fa0/1 R7(config-if)#no shut R7(config-if)#do sh ip ospf neigh Neighbor ID Pri State Dead Time Address Interface 10.0.104.1 1 FULL/DR 00:00:36 10.0.17.1 FastEthernet0/1 10.0.47.4 1 FULL/BDR 00:00:36 10.0.47.4 FastEthernet0/0 R7(config-if)#do sh ip route ospf 1.0.0.0/32 is subnetted, 1 subnets O 1.1.1.1 [110/2] via 10.0.17.1, 00:00:26, FastEthernet0/1 10.0.0.0/8 is variably subnetted, 13 subnets, 2 masks O 10.0.12.0/24 [110/2] via 10.0.17.1, 00:00:26, FastEthernet0/1 O 10.0.104.1/32 [110/2] via 10.0.17.1, 00:00:26, FastEthernet0/1 O 10.0.103.1/32 [110/2] via 10.0.17.1, 00:00:26, FastEthernet0/1 O 10.0.102.1/32 [110/2] via 10.0.17.1, 00:00:26, FastEthernet0/1 O 10.0.101.1/32 [110/2] via 10.0.17.1, 00:00:26, FastEthernet0/1 O 10.0.100.1/32 [110/2] via 10.0.17.1, 00:00:26, FastEthernet0/1
Hmmm…interesting! When I brought up the backup link over VLAN 17, my routes completely changed. NOW my best path is an INTRA-area route via VLAN 17. Why? Well, OSPF always preferes intra-area routes to inter-area routes. Even if we tweaked our OSPF metric all day long to give precedence to the MPLS route, we are screwed at this point due to this rule. This is a rule of OSPF we can NOT change…but can we manipulate other things? What if we could make the routes coming in from the MPLS intra-area routes as well? Enter the sham-link. The sham link is essentially an OSPF virtual-link that goes through the service provider cloud to connect two areas together. So basically, we can create a special virtual-link between our PE routers called a sham-link that connects our two area 0’s together. The routes over the sham link will then be considered inter-area routes. Let’s do it! For this to work, we need to do a few things:
- A /32 loopback address on each PE router. This loopback has to be in the customer VRF and can NOT be directly advertised into OSPF.
- Advertise these loopbacks into MP-BGP as vpnv4 routes. This is how the PE routers will learn about the endpoints of the sham-link.
- Configure the sham-link under the OSPF process on the PE routers
R2:
R2(config)#int lo22 R2(config-if)#ip vrf forwarding CustA R2(config-if)#ip add 22.22.22.22 255.255.255.255 R2(config-if)#router bgp 55 R2(config-router)#address-family ipv4 vrf CustA R2(config-router-af)#network 22.22.22.22 mask 255.255.255.255 R2(config-router-af)#router ospf 2 R2(config-router)#area 0 sham-link 22.22.22.22 44.44.44.44
R4:
R4(config-router)#int lo44 R4(config-if)#ip vrf forwarding CustA R4(config-if)#ip add 44.44.44.44 255.255.255.255 R4(config-if)#router bgp 55 R4(config-router)#address-family ipv4 vrf CustA R4(config-router-af)#network 44.44.44.44 mask 255.255.255.255 R4(config-router-af)#router ospf 2 R4(config-router)#area 0 sham-link 44.44.44.44 22.22.22.22 R4(config-router)#do sh ip ospf neigh Neighbor ID Pri State Dead Time Address Interface 6.6.6.6 0 FULL/ - 00:01:35 100.100.100.6 Serial0/0/0 10.0.12.2 0 FULL/ - - 22.22.22.22 OSPF_SL1 10.0.204.7 1 FULL/DR 00:00:31 10.0.47.7 FastEthernet0/0
Rock N’ Roll!!! Notice the sham-link came up! (OSPF_SL1) So, to recap we added a loopback on both ends. We advertised that loopback into MP-BGP in the CustA VRF. We then created a sham-link that connected our two area 0’s together. NOW, let’s see those routes.
R7:
R7(config-if)#do sh ip route ospf 1.0.0.0/32 is subnetted, 1 subnets O 1.1.1.1 [110/2] via 10.0.17.1, 00:12:20, FastEthernet0/1 22.0.0.0/32 is subnetted, 1 subnets O E2 22.22.22.22 [110/1] via 10.0.47.4, 00:04:23, FastEthernet0/0 10.0.0.0/8 is variably subnetted, 13 subnets, 2 masks O 10.0.12.0/24 [110/2] via 10.0.17.1, 00:12:20, FastEthernet0/1 O 10.0.104.1/32 [110/2] via 10.0.17.1, 00:12:20, FastEthernet0/1 O 10.0.103.1/32 [110/2] via 10.0.17.1, 00:12:20, FastEthernet0/1 O 10.0.102.1/32 [110/2] via 10.0.17.1, 00:12:20, FastEthernet0/1 O 10.0.101.1/32 [110/2] via 10.0.17.1, 00:12:20, FastEthernet0/1 O 10.0.100.1/32 [110/2] via 10.0.17.1, 00:12:20, FastEthernet0/1 44.0.0.0/32 is subnetted, 1 subnets O E2 44.44.44.44 [110/1] via 10.0.47.4, 00:02:48, FastEthernet0/0
Boooooo, hisssssss, stink! We still have the same problem. Why? Well, despite the fact that we now have intra-area routes through the MPLS, the cost over the directly connected FastEthernet interface still beats us (darn FastEthernet cost of 1!!!) Oh well, this is easily fixed…
R1:
R1(config-if)#int fa0/1 R1(config-if)#ip ospf cost 10
R7:
R7(config-if)#int fa0/1 R7(config-if)#ip ospf cost 10 R7(config-if)#do sh ip route ospf 1.0.0.0/32 is subnetted, 1 subnets O 1.1.1.1 [110/4] via 10.0.47.4, 00:00:19, FastEthernet0/0 22.0.0.0/32 is subnetted, 1 subnets O E2 22.22.22.22 [110/1] via 10.0.47.4, 00:06:32, FastEthernet0/0 10.0.0.0/8 is variably subnetted, 13 subnets, 2 masks O 10.0.12.0/24 [110/3] via 10.0.47.4, 00:00:19, FastEthernet0/0 O 10.0.104.1/32 [110/4] via 10.0.47.4, 00:00:19, FastEthernet0/0 O 10.0.103.1/32 [110/4] via 10.0.47.4, 00:00:19, FastEthernet0/0 O 10.0.102.1/32 [110/4] via 10.0.47.4, 00:00:19, FastEthernet0/0 O 10.0.101.1/32 [110/4] via 10.0.47.4, 00:00:19, FastEthernet0/0 O 10.0.100.1/32 [110/4] via 10.0.47.4, 00:00:19, FastEthernet0/0 44.0.0.0/32 is subnetted, 1 subnets O E2 44.44.44.44 [110/1] via 10.0.47.4, 00:04:57, FastEthernet0/0
R1:
R1(config-if)#do sh ip route ospf 22.0.0.0/32 is subnetted, 1 subnets O E2 22.22.22.22 [110/1] via 10.0.12.2, 00:07:38, FastEthernet0/0 7.0.0.0/32 is subnetted, 1 subnets O 7.7.7.7 [110/4] via 10.0.12.2, 00:00:16, FastEthernet0/0 10.0.0.0/8 is variably subnetted, 13 subnets, 2 masks O 10.0.47.0/24 [110/3] via 10.0.12.2, 00:00:16, FastEthernet0/0 O 10.0.204.7/32 [110/4] via 10.0.12.2, 00:00:16, FastEthernet0/0 O 10.0.201.7/32 [110/4] via 10.0.12.2, 00:00:16, FastEthernet0/0 O 10.0.200.7/32 [110/4] via 10.0.12.2, 00:00:16, FastEthernet0/0 O 10.0.203.7/32 [110/4] via 10.0.12.2, 00:00:16, FastEthernet0/0 O 10.0.202.7/32 [110/4] via 10.0.12.2, 00:00:16, FastEthernet0/0 44.0.0.0/32 is subnetted, 1 subnets O E2 44.44.44.44 [110/1] via 10.0.12.2, 00:05:43, FastEthernet0/0
Rock N’ Roll…Notice the best path is now via the MPLS and they are INTRA-Area routes! There is also an option when we create the sham-link to set the cost of the sham-link. In this case, it wouldn’t really help use since our total path cost out VLAN 17 is like 2 :P So there you have it…that’s not so bad now is it?
–
Joe Astorino
CCIE #24347 (R&S)
Great, well explained!!!
Thanks Joe, really like your style!
please help put back the diagram
Hi ,
how we can load balancing the OSPF routes between the backdoor link and PE-CE link
Make the paths be equal cost.
the most important part at all is:
R4(config-router)#do sh ip ospf neigh
10.0.12.2 0 FULL/- 22.22.22.22 OSPF_SL1
what if this neighborship doesn’t comp up even if the sham link is up?
In that case, you need to go into the “troubleshooting mode” and figure out why it didn’t come up. A very quick and non-comprehensive checklist:
1. Are your SL Loopbacks advertised in MP-BGP? (they must be)
2. Are your SL Loopbacks advertised in OSPF? (they shouldn’t be)
3. Are your SL Loopbacks reachable by ping? (they should be)
4. Is there ned-to-end LSP between PE routers? (there must be)
thanx for ur fast reply
Step 3 is failed:
I have a doubt in mpls lables, how can i check this?
=======================================
this is a little brief from my lab:
ip cef
mpls label protocol ldp
interface FastEthernet0/0
ip address 10.12.1.2 255.255.255.252
mpls label protocol ldp
tag-switching ip
======================================
R2#sh ip route vrf AB
!
B 11.11.11.11 [200/0] via 1.1.1.1, 00:13:03
R2#ping vrf AB 11.11.11.11
…..
Success rate is 0 percent (0/5)
=======================================
R2#sh ip bgp vpnv4 vrf AB
*>i11.11.11.11/32 1.1.1.1 0 100 0 i
ping debugging message:
IP: s=22.22.22.22 (local), d=11.11.11.11 (FastEthernet0/0), len 76, encapsulation failed
Encapsulation failed usually points to a L2 problem. Did you ping inside VRF – “ping vrf AB 11.11.11.11 so loX”?
By the way – a much more suitable place to ask a generic troubleshooting question like this would be our Online Study List. You may want to consider subscribing to R&S one and asking a question there.
-Marko.
R2#ping vrf AB 11.11.11.11 source loopback 22
Sending 5, 100-byte ICMP Echos to 11.11.11.11, timeout is 2 seconds:
Packet sent with a source address of 22.22.22.22
…..
Success rate is 0 percent (0/5)
it doesn’t work.
I tried everything, when I discover it, I’ll let you know.
Is there a return route? Is your MPLS working (show mpls ldp discovery) on all your PE and P devices? Do they all have the unsummarized routes for PE Loopbacks used for MP-BGP peering?
As I said before – a much better place for this discussion is OSL. You will probably get faster responses from other students, too. I highly recommend you join the list!
-Marko.
LoL it worked with the Last trial ever:
“Do they all have the unsummarized routes for PE Loopbacks used for MP-BGP peering?”
the answer was “NO”
R1(config)#no ip route 2.0.0.0 255.0.0.0 10.12.1.2
R1(config)#ip route 2.2.2.2 255.255.255.255 10.12.1.2
so please tell me why MP-BGP peering needs exact loopback route not summarized?
Thank you ^_^
Answer is right here on this blog. We have at least two articles that deal with this issue:
http://blog.ipexpert.com/2012/01/10/mpls-l3vpn-and-summarized-loopback-routes/
http://blog.ipexpert.com/2010/05/31/next-hop-in-mpls-vpns/
I put the whole Lab on my blog if you want to review it.
http://getcisco.blogspot.com/
thank you a lot Marko.