What happened to CCIE Datacenter?

Everybody thought that new Data Center CCIE will be announced at Cisco Live 2010. Unfortunately (or not), that didn’t happen. In the mostly rumored BRKCCIE-1001 on the final day nothing was announced although we had a great time with Antonella Corno, who is the program manager for CCIE Storage. The final word she had to say about it was: “It will be there sometime, but not now and not tomorrow”. She said that the software is still undergoing major overhauls, especially for UCS. It wouldn’t be a good idea to pick a software version and keep that in the blueprint for 2 years. It’s simply not ‘mature’ yet. I personally think that is a very good argument, as I’ve seen pretty much every UCS version from 1.0.1 to 1.3 and a LOT has changed in these versions.

The next major part of the CCIE Datacenter would be Nexus 2000, 5000 and 7000. NX-OS has been on the market now for about 1.5 to 2 years and you can see they are still adding major features, although with the introduction of OTV and FabricPath this has been completed now for a while. OTV and FabricPath are so new that this is also impossible to fit in a CCIE track as the first release of software was a few weeks ago and FabricPath will be out in September.

Again when everything flattens out and ‘matures’ than the CCIE Datacenter will be released.

CCIE Storage Reading

There are not a lot of resources to study from for CCIE Storage. There are 2 Cisco Press titles.
The first one is Storage Area Network Fundamentals. Which gives you an introduction to SAN technologies. It’s a bit dated (2002), but not much has changed in SCSI and FC standards over the last couple years (except for 8G, 10G FC and FCoE of course). I personally haven’t read this book, but it seems to be a good introduction.

The second is Storage Networking Protocol Fundamentals. This book goes really deep into the different standards and their respective OSI layers. It’s a good book if you know the basics as it really goes deep and can be very confusing if you start with it.

The third is one of the best texts that has even been written on Fibre Channel. The Fibre Channel Bench Reference Guide is a tough one, but describes everything very good! It’s not an easy read, but try and see if you can find the things you need to know and things you don’t need to know (don’t try to remember the 8B/10B encoding and especially not the reasons why you need disparity).

The chapter on SW_ILS communication (basically the enabling of ISLs or inter-switch-links) is the best and was really helpful for me to understand this.

The last recommendation I can give is the IBM Redbook on FICON. There are multiple IBM redbooks on describing FICON, but this one helps as it describes the implementation on Cisco’s MDS switches and gives you a great example of an IOCP file (IBM mainframe configuration files), that you will need to be able to read for the CCIE Storage lab exam.

FICON Native Implementation and Reference Guide: http://www.redbooks.ibm.com/abstracts/sg246266.html
Cisco FICON Basic Implementation: http://www.redbooks.ibm.com/abstracts/redp4392.html

Besides that you will need some real-life experience or some lab time on a couple MDS switches.

CCIE Storage Lab Preparation

To prepare for the CCIE Storage lab is a little more difficult in comparison to other, more popular, CCIE tracks. You need your own rack of 2 or preferably 3 MDS switches, a couple servers with dual HBA’s, dual-attached JBOD’s and preferably a Brocade and/or McData switch for interoperability testing.

The hardware you need can be of various sources, eBay is a good one to start with. One downside of the MDS switches is that they work with a license model. There are several licenses that can be bought for MDS switches. You need the following licenses to study for all topics:

• ENTERPRISE_PKG
  • This license is required for a LOT of ‘advanced’ features, like read-only zoning, zone-based QoS, IPsec and many other things.
• SAN_EXTN_OVER_IP
  • This is the so-called FCIP license. You need this just for extending a SAN over IP and the only technology currently available for that is FCIP (Fibre Channel over IP). Be aware that you can only build 3 tunnels per GigE port on the MDS switches!
• MAINFRAME
  • This license is required for the FICON feature. FICON is a technology used by IBM mainframes and it’s another ULP (Upper Level Protocol) for Layer 1 and Layer 2 Fibre Channel standards. Usually the only ULP used is FCP (Fibre Channel Protocol). FICON has a few own specifics, especially about semi-automatically allocated port numbers versus standard FCID addresses.
• FM_SERVER_PKG
  • Fabric Manager is the management tool provided by Cisco to manage an entire fabric of MDS switches (multiple connected FC switches are called a fabric). This software can be installed in a stand-alone fashion in which you can manager 1 single fabric. This is perfect for studies, as this doesn’t consume a license, but of course, that’s too good to be true as there is 1 exception, which is that you don’t have access to the web client of Fabric Manager. Therefore you need to install the server-based version that uses a PostgreSQL database and runs as a Windows service. This server based version can manage multiple fabrics, but also supports the web client in which you can create several reports and can run performance monitoring, which is definitely a blueprint topic to study. Unfortunately this server version does consume a license!

Please be aware of the grace period that is available to you when you get a new switch. By default every switch gets 120 days ‘trial’ period for every feature in the box (exception is the Fabric Manager license, that doesn’t know this period).

Hardware

The MDS portfolio knows many different switches. The top notch is the MDS9500 chassis based switch that knows just like the Catalyst 6500 a large amount of modules to fit in its slots and as the hardware got new features, new ‘Generations’ of modules were released.

There are Generation 1, Generation 2 and Generation 3 linecards. You need a Supervisor 2 and NX-OS 4.1 to support Gen 3, so you will not find these in the lab as that is based on SAN-OS 3.2. Generation 2 linecards support 4Gbps FC and know oversubscription limits. Each Generation 2 linecard comes with 4 port-groups each having 12.8Gbps to share, besides that you have to deal with a maximum 4:1 oversubscription rate on all Gen 2 modules except the 48 port module that knows a 5:1 oversubscription limit (this limit can be disabled, but prepare for any combination in the lab) and the 12 port module which has no oversubscription at all (4 port groups = 3 ports per group, 3×4=12Gbps max traffic per port group).

The Generation 2 linecards are very expensive, as are the 9500 chassis and they take a lot of space and power, so are not really suited for a rack at home.

The switches you will want to get are in the most perfect situation MDS9222i switches. These switches have an on-board MPS18+4 module, meaning you have 18 1/2/4Gbps FC ports and 4 GigabitEthernet ports for IP features. Besides that, this particular module supports all the advanced storage features/services that Cisco has built. The 18+4 linecard is a Generation 2 card, so it does know some oversubscription limits. One downside is that this switch is relatively new and is expensive.

The best option from my opinion would be a couple MDS9216i (or MDS9216A) switches. This is a small switch (only 3 RU), has a built-in 14 port 1/2Gbps FC module and has a spare slot in which you can fit a generation 1 MDS9500 module. You will want to have a module in there that has GigabitEthernet ports, in which you have 3 options. First are the IPS-4 and IPS-8 module that have 4 and 8 GigE ports respectively. They support Ethernet port-channeling, Network Simulator and a lot of the IP features, except IPsec VPN’s. The third option is the MPS14/2 linecard which as 14 1/2Gbps FC connections and 2 GigabitEthernet connections. This card supports all IP features, including IPsec VPN’s, except port-channeling and Network Simulator. Please be aware that this switch will not support any Generation 2 linecards, so you can’t practice oversubscription, you can practice Storage Services with the 9032-SSM module. This is another expensive (generation 1) module that supports all the advanced storage features. Please don’t think you really need this card, it’s a nice to have, as there are quite some features to be tested, but this won’t be a core-topic on the lab! 2, preferably 3 MDS9216i switches with at least 2 IPS4/8 or 14/2 cards would suffice all your study needs, but be sure to practice for the oversubscription and storage services by reading documentation.

Documentation

As of labs, try doing everything that is on the blueprint for CCIE Storage. If you master all the topics you can go to the lab and FLY through it. The advantage of the blueprint is that it really tells you what to do and mentions actual technologies and features that you can try.

The MDS Cookbook (3.x) is a fantastic way to start your preparation as it runs you through labs all the way using CLI and GUI. By going over it you should have a good overview of all the basic technologies.

After that the MDS Configuration Guide is the resource you want to be comfortable with. Don’t try to remember the full 1600 page document, but know where to find the stuff you need and use it as a resource for your lab work.

I hope you’ve enjoyed this brief overview of the preparation for CCIE Storage and hope you will enjoy the studying for this awesome track.

–
Rick Mur, CCIE3 #21946 (Routing & Switching / Service Provider / Storage)
Sr. Support Engineer – IPexpert, Inc.
URL: http://www.ipexpert.com/

Print Friendly

Therefore it’s good to expand thoughts and knowledge and I bought a stack of books related to Storage and Cisco Datacenter solutions. Besides that it’s a little personal thing to see if I can combine the current studies for Security and also try to pursue the CCIE Storage as well. It’s a difficult choice, since there are quite some rumors going on the internet, that suspect a change to CCIE Datacenter and that Cisco would drop having a CCIE solely based on the MDS switches. With the current Nexus portfolio, it really removes the need for certain type of MDS switches and position the MDS line only in the Fibre Channel core of a network.

So I personally think it’s inevitable for Cisco to change the Storage track to cover a bigger aspect of the datacenter and include the LAN switch environment and include some servers in the form of a UCS set-up.

To not get surprised I put my studies not solely to Storage and Fibre Channel, but also take effort to learn more on UCS and Nexus. I did a little Datacenter Networking certification, but that doesn’t really cover anything on a CCIE level. I did have the privilege of doing a couple projects that included the installation and configuration of Nexus switches and I even had the privilege of implementing the first UCS system in the Netherlands.

So how do you start studying for something that is not even announced?

I made a small list of things that I expect Cisco to put in the Datacenter lab. Just to remind you, I do NOT have any contact with Cisco and I just created this based on rumors and logical thinking.

MDS 9000

o MDS switches will remain in the lab, same as Fibre Channel and probably a 3rd party FC switch like a Brocade

o They will be positioned as the Storage core network and might have an access MDS to connect some storage systems

Nexus 7000

o Definitely in there as a LAN core

o L2/L3 device

o Separate VDC’s to virtualize the network

Nexus 5000

o Positioned at the Network and even Storage access layer

o This is THE FCoE switch that Cisco has

o L2 debice with uplink to Nexus 7000 core

o Fibre Channel device with uplink to MDS core

o 10G servers attached

UCS 5100

o The 8-slot generic blade chassis

o UCS B200 blades

o UCS Fabric Extenders (switches inside the chassis)

UCS 6120

o Interconnect device to connect UCS Fabric Extenders

o This is the same chassis as the Nexus 5000, but it’s NOT a switch

o Contains the management software for the whole UCS system

o Has a master, slave set-up possibility

Fibre Channel storage

o Some NetApp or EMC Fibre Channel or maybe FCoE storage cluster

o Might be a MetroCluster set-up, so traffic replicates over the MDS switches

Other vendor blade system

o They might throw in a blade chassis from another vendor like HP including some Catalyst blade switches

Well I think that should cover the CCIE Datacenter track. An important thing is of course, would Cisco go so far to implement this? Since this is clearly something way to expensive for home users, rack rental providers and even for Cisco’s CCIE team. This would seem to me as the perfect fit to test for any Datacenter product. The problem that Cisco introduced with the Nexus line is that they are very tightly positioned. If you look at router features than lower platforms can perform about any feature that the higher platforms can do as well. This is different with the Nexus portfolio, you can only use them for the goal that they are positioned for.

Either case I will certainly update you with all the discoveries I do in this area. Keep checking back for more posts. Next one will be the booklist.

Print Friendly

Show running-config

At first it’s not that special of course to display the current running-config of the box, but when you issue a question mark there are tons of options.

Rack1ASA1# sh run ?

aaa Show AAA configuration information

aaa-server Show aaa-server configuration information

access-group Show access group(s)

access-list Show configured access control elements

alias Show configured overlapping addresses with dual

NAT

all Show current operating configuration including

defaults

arp Show configured ARP entries, ARP timeout

asdm Show ASDM configuration

auth-prompt Show configured authentication challenge,

reject and acceptance prompts

auto-update Show Auto Update configuration

banner Show configured login/session banners

boot Show boot configuration information

class Show class configuration

class-map Show class-map configuration

client-update Show global client-update configuration

information

clock Show clock configuration

command-alias Show configured command aliases

compression Show compression global configuration

console Show console idle timeout

crypto Show crypto configuration

ctl-file Show configured CTL file instances

<— More —>

It’s possible to view EVERY configuration ‘snippet’ with a show run. For example, displaying access-list configuration:

Rack1ASA1(config)# sh run access-list

access-list OUTSIDE_IN extended permit tcp any host 136.1.122.12 eq telnet

access-list OUTSIDE_IN extended permit tcp 150.1.2.0 255.255.255.0 host 136.1.122.12 eq www

access-list INSIDE_IN extended permit icmp any any echo

Rack1ASA1(config)#

Or class-maps:

Rack1ASA1(config-cmap)# sh run class-map

!

class-map TEST

match port tcp eq www

class-map inspection_default

match default-inspection-traffic

!

Rack1ASA1(config-cmap)#

Or NAT configuration:

Rack1ASA1(config)# sh run global

global (outside) 1 136.1.122.100-136.1.122.200

Rack1ASA1(config)# sh run nat

nat (inside) 1 0.0.0.0 0.0.0.0

Rack1ASA1(config)#

No ‘do’

Another thing to notice is that the ‘do’ command is NOT used on the ASA CLI. It doesn’t matter if you are in enabled mode or in configuration mode, the do command does not exist and ALL commands can be executed from configuration mode.

Rack1ASA1(config)# do sh ip

^

ERROR: % Invalid input detected at ‘^’ marker.

Rack1ASA1(config)# sh ip

System IP Addresses:

Interface Name IP address Subnet mask Method

Ethernet0/0 outside 136.1.122.12 255.255.255.0 manual

Ethernet0/1 inside 136.1.121.12 255.255.255.0 manual

Ethernet0/2 dmz 10.0.0.12 255.255.255.0 manual

Context sensitive help

The context sensitive help is also quite informational on the ASA. It first differentiates the config and exec mode commands (since they are executable from config mode)

Rack1ASA1(config-if)# crypto ?

configure mode commands/options:

ca Certification authority

dynamic-map Configure a dynamic crypto map

ipsec Configure transform-set, IPSec SA lifetime, and fragmentation

isakmp Configure ISAKMP

key Long term key operations

map Configure a crypto map

exec mode commands/options:

ca Execute Certification Authority Commands

Rack1ASA1(config-if)# crypto

Auto completion

Sometimes values are automatically available in the context help. For example interface ‘nameif’’s are possible for autocompletion:

Rack1ASA1(config)# access-group OUTSIDE in int ?

configure mode commands/options:

Current available interface(s):

dmz Name of interface Ethernet0/2

inside Name of interface Ethernet0/1

outside Name of interface Ethernet0/0

Rack1ASA1(config)# access-group OUTSIDE in int

When tab is pressed and enough characters are entered, the ASA will auto-complete the interface name. Also the question mark immediately gives a great overview of which physical interface is assigned which nameif.

Remove configuration

Sometimes a simple ‘no’ is not enough for the ASA to remove configuration. It’s impossible to remove an entire access-list with a simple no. In IOS this would work by just entering ‘no ip access-list extended <NAME>’. Since the ASA CLI also supports inserting and removing lines from an ACL without using numbering like in IOS, it requires you to enter the entire ACE (Access List Entry) before it actually removes it from the configuration.

For removing entire parts of configuration, Cisco introduced the ‘clear configure’ command on the ASA CLI. This command has the same logic as the ‘show run’ as it can remove entire configuration snippets with it, so for example all NAT config and a specific ACL.

Rack1ASA1(config)# clear configure global

Rack1ASA1(config)# clear configure access-list TELNET

Rack1ASA1(config)# sh run global

Rack1ASA1(config)# sh run access-list TELNET

ERROR: access-list <TELNET> does not exist

Rack1ASA1(config)#

This is a very handy command when doing labs, as it might be affective to redo the entire NAT setup again or remove an entire ACL to rename it for example.

VPN setup

Since the CCIE lab offers very few time left to really dive in the documentation the ASA CLI has a little cheat sheet for VPN configuration built-in! This is a really cool feature!

With the command ‘vpnsetup’ in configuration mode you have several options to see different kinds of VPN set-ups.

Rack1ASA1(config)# vpnsetup ?

configure mode commands/options:

ipsec-remote-access Display IPSec Remote Access Configuration Commands

l2tp-remote-access Display L2TP/IPSec Configuration Commands

site-to-site Display IPSec Site-to-Site Configuration Commands

ssl-remote-access Display SSL Remote Access Configuration Commands

Rack1ASA1(config)# vpnsetup

To check the site-to-site required steps enter the following commands and the ASA demonstrates all the required steps to take for a successful site-to-site VPN implementation

Rack1ASA1(config)# vpnsetup site-to-site ?

configure mode commands/options:

steps Display VPN Setup Commands

Rack1ASA1(config)# vpnsetup site-to-site steps ?

configure mode commands/options:

<cr>

Rack1ASA1(config)# vpnsetup site-to-site steps

Steps to configure a site-to-site IKE/IPSec connection with examples:

1. Configure Interfaces

interface GigabitEthernet0/0

ip address 10.10.4.200 255.255.255.0

nameif outside

no shutdown

interface GigabitEthernet0/1

ip address 192.168.0.20 255.255.255.0

nameif inside

no shutdown

2. Configure ISAKMP policy

crypto isakmp policy 10

authentication pre-share

encryption aes

hash sha

3. Configure transform-set

crypto ipsec transform-set myset esp-aes esp-sha-hmac

4. Configure ACL

access-list L2LAccessList extended permit ip 192.168.0.0 255.255.255.0 192.168.50.0 255.255.255.0

5. Configure Tunnel group

tunnel-group 10.20.20.1 type ipsec-l2l

tunnel-group 10.20.20.1 ipsec-attributes

pre-shared-key P@rtn3rNetw0rk

6. Configure crypto map and attach to interface

crypto map mymap 10 match address L2LAccessList

crypto map mymap 10 set peer 10.10.4.108

crypto map mymap 10 set transform-set myset

crypto map mymap 10 set reverse-route

crypto map mymap interface outside

7. Enable isakmp on interface

crypto isakmp enable outside

Rack1ASA1(config)#

The only tiny thing that’s forgotten in this example are ‘security-level’s under the interface, though the inside and outside keywords automatically assign a security-level of 100 and 0 respectively.  So when there is no time to check the documentation, you can always rely on a little built-in cheat sheet provided by our friends at Cisco.

If you have any other cool features of the ASA CLI that you want to share? Please feel free to comment on this blog post!

Rick Mur
CCIE2 #21946 (R&S / Service Provider)
Sr. Support Engineer – IPexpert, Inc.
URL: http://www.IPexpert.com

Print Friendly

Why?

Etherchanneling is used in networks to have both redundant links and a double (or more) bandwidth link to another Catalyst switch. It also helps in forming a loop-free path and overcomes the issue of having 1 link in blocking (as you would when using Spanning Tree). On some router platforms it’s also possible to bundle links, but most times this doesn’t really work and you could run into problems while using it on different IOS versions.

It’s possible to bundle up to 8 active links in 1 etherchannel. After that it’s more cost effective to have 1 higher bandwidth link (8x1G is almost 10G). This bundled link is treated as 1 virtual-link within the switch and practically any layer 2 or layer 3 feature is possible. You can use it as access port, trunk, Q-in-Q or regular routed (layer 3) port. The only thing that is NOT possible is using it as Private VLAN host or promiscuous port or use the channel as SPAN destination interface (can be done with Remote-SPAN).

The command used to configure a etherchannel is channel-group # mode <mode> the number can by anything between 1 and 48. 48 is the hardware limit of the number of port-channels in the Catalyst 3560 and 3750 (it’s no use of configuring every port in a separate channel of course). The modes are explained later.

Different vendors use a LOT of different naming conventions for these features. Some vendors call it a trunk, aggregate, NIC team, MLT, bonding and many more. Cisco uses the term ‘portchannel’ and ‘etherchannel’, which both mean the exact same thing and can be used interchangeably. Other terms that Cisco tends to use are FEC, GEC and MEC. The first is Fast EtherChannel, meaning that the links that are used to bundle are FastEthernet interfaces. GEC means the same thing, but with GigabitEthernet interfaces. The MEC is much more interesting. This is the abbreviation for Multi Chassis EtherChannel. This technology is used within VSS systems for the Catalyst 6500’s. Normally etherchanneling is used on a SINGLE device on 1 end. With VSS you get 2 systems on 1 end and therefore it’s both loadsharing AND box-redundant.

3 protocols?

There are a couple ways of managing etherchannels. Cisco developed the proprietary PAgP protocol (Port AGgregation Protocol). This protocol negotiates the status of links with the other end. If there is no negotiation possible than the links are not bundled and treated as single links. There are 2 ‘modes’ while configuring an etherchannel for PAgP.

First is desirable, this is a mode where the device actively is sending PAgP packets. This side of the link is desperate in bundling it’s links and asks the other end to participate.

Second is auto, this is the most confusing mode in my opinion. It suggests that the switch will ‘automatically’ negotiate the bundle, but in fact, this mode is the passive variant of PAgP. It WAITS on the other end to send packets and then replies to it and bundles the links. Having both ends on auto would not work!

The second protocol is LACP, which is IEEE 802.3ad and the industry standard for bundling links. This protocol has the same function as PAgP, but has some little details that are different. First of all it’s a standard and therefore compatible on a lot of systems (servers) and other vendor switches. Second is that LACP knows a difference between active and standby links. The first 8 links in the bundle are active and working, the other 8 (making a total of 16) are hot-standby and not handling any traffic. That way you could build in extra redundancy if you are relying on all the bandwidth or prevent unequal loadbalancing (more on that later). To decide which links are participating in the bundle at all times there should be a LACP port-priority configured. The higher the number, the more likely the link will be used as active link.

LACP also knows 2 different modes. First is active, this mode is comparable to desirable for PAgP as it also actively tries to negotiate a bundled link and is asking the other end to participate.

Second is passive, which is the same as auto in PAgP but has a much more clear keyword as it does what you expect. It remains passive, but if the other end is requesting bundling, it complies and bundles the link.

The third way of bundling links is without any protocol that negotiates the state the links. This mode is known as on and is putting the interface in a bundle no matter what. As long as the interface is up it’s participating in the bundle. This could be the solution when the other end is not supporting any protocol or is not compliant with IEEE 802.3ad. I would always use this mode as last resort if it were not possible to use negotiation especially when you are not administering the other end of the link!

With the use of these protocols you overcome quite some issues that you could run into. First is an important requirement for bundles. Both sides of the link should have the same speed and duplex settings. This is checked in the negotiation. Second is that when active optical equipment is used (xWDM), than the interface is not going down when somewhere in the path the fiber is cut. When using a negotiation protocol that error is detected and the link is taken out of the bundle.

Loadbalancing

The second features besides redundancy is loadbalancing or loadsharing between the links in the bundle. This is an important topic as it’s NOT negotiated when using PAgP or LACP, but the balancing needs to be the same on both ends for a correctly functioning link.

On switches it’s possible to configure the way of balancing the traffic. Depending on the hardware you have several options are possible. On the Catalyst 3560 used in the CCIE labs it’s possible to configure balancing on the following values:

• Source MAC (default)

• Destination MAC

• Source+Destination MAC

• Source IP

• Destination IP

• Source+Destination IP

On the higher platforms like the 6500 there are a lot more ways of doing loadbalancing on Layer 2 to 4 information and various combinations of that.

You must take this in mind when designing an etherchannel. For example, when using a layer 3 channel, there would be NO loadbalancing with the default setting as the source mac address is always the link itself.

Etherchannel loadbalancing is configured with the command port-channel loadbalance <type> the types are discussed previously.

The balancing also needs to be the same on both ends of the link otherwise some strange traffic patterns are seen and this could cause problems.

A 50/50 load balancing is very hard to achieve and you’ll see that the best balancing is 70/30 on layer 3 information. When using layer 4 information like a combination of src+dst IP and src+dst TCP/UDP port, than you’ll get a lot close to this 50/50 scheme. Unfortunately this is not possible on the Cat 3560/3750.

When determining the ‘perfect’ way of balancing, it’s important to choose the right amount of links in the bundle! From each pair of values (src, dst, mac/ip) a hash is calculated and all those values are divided over so-called ‘buckets’. There are 8 buckets (for 8 possible links) no matter what amount of links are used. Each bucket is served in order, there is no way of giving priority

When using 2 links the buckets are divided as 4:4. When using 3 links, it’s divided as 3:3:2. That way there is unequal load balancing, that way link #3 is not used as much as link 1 and 2. The only way you get evenly divided loadbalancing when using 2,4 or 8 links. You’ll get 4:4, 2:2:2:2 and 1:1:1:1:1:1:1:1 allocations of the buckets respectively, which is completely even. This makes it interesting to use the hot-standby feature of LACP or making sure there are always that amount of links active in the bundle (by using other mechanisms).

I hope I was able to give an impression and the considerations that need to be done when implementing etherchannels in your network and also when preparing for the CCIE lab.

Warm Regards,

Rick Mur
CCIE2 #21946 (R&S / Service Provider)
Sr. Support Engineer – IPexpert, Inc.
URL: http://www.IPexpert.com

Print Friendly

This was the first CCIE written I did in the ‘new-style’. The written exam used to consist of 100 questions and you got 1 point per correctly answered question. The passing score was somewhere between 70 and 80, which was in my opinion a great way to score an exam.

Now Cisco uses the same style as for all other exams. The score is between the 300 and 1000 and the amount of questions between 90 and 110.

The passing score was quite low in my opinion, on my exam it was 699, so a pretty do-able job.

Blueprint

The current blueprint for the CCIE Security Lab is at version 3.0, where all the technologies have been updated to the latest and greatest. An important difference is that the blueprint for the written exam hasn’t changed since the introduction of the 3.0 and is still at version 2.0. This means that officially the PIX and VPN3000 are still part of the written. Although the chance that you’ll get a question about it is very small.

Another thing is that I think that the CCIE Security Written is the written exam which is the least aligned with the lab exam. DMVPN and GETVPN for example are not on the blueprint and are not tested (maybe at a very basic level).

There are also very few topology questions and a lot of pure theoretical questions. A large focus lies on Encryption, IPsec VPNs, the Cisco software tools, AAA and standards.

IPsec VPNs

The questions asked about IPsec are from all aspects and can be theoretical up to configuration examples. Most important is to know which port and protocol numbers IPsec uses in the different phases. You could run into some drag-n-drop question where you are asked to put the correct IP protocol and UDP port numbers.

Just a quick reminder:

IKE = IP protocol 50

AH = IP protocol 51

IKE = UDP 500

IKE NAT-T = UDP 4500

Or a drag-n-drop about which description aligns with which Phase and protocol.

IKE Main/Aggressive Mode = Phase 1

X-auth = Phase 1.5

IPsec Quick Mode = Phase 2

So be familiar with the various ways of configuring IPsec and all the required protocols and procedures.

Encryption

A lot of questions were related to the different encryption protocols and all the aspects and differences between Hash, Symmetric and Asymmetric algorithms.

Reminder:

DES/3DES and AES = Symmetric

RSA, RC4 = Asymmetric

MD5, SHA1 = Hash

Be familiar with the performance differences between the different variations and where and how they are used.

AAA

The differences between RADIUS and TACACS+ are also very important to understand, same with the implementation of SSH on the Cisco IOS platform. For the 2 AAA protocols it’s important to remember the packets than go back and forth between the device and authentication server. Screenshots of Cisco Secure ACS and asking what that configuration would do also belong to the possibility.

Software Tools

The software tools are not very much tested on the lab exam, that’s why they are thoroughly tested on the written. You could run into a lot of questions about Cisco Secure Agent, Cisco Trust Agent, Cisco SecureDesktop, Cisco Security Manager (CSM), SDM and ASDM. All are tested on the written. You don’t have to know every little detail, but mainly what they do, where they are used and how they get installed or perhaps automatically downloaded.

Standards

This is the most pesky thing about the written is that Cisco wants you to know a LOT of RFC’s and ISO standards. According to the Cisco CCIE Security Written Blueprint the standards you need to know are:

Security General

A. Policies – Security Policy Best Practices

B. Information Security Standards (ISO 17799, ISO 27001, BS7799)

C. Standards Bodies

D. Common RFCs (e.g. RFC1918, RFC2827, RFC2401)

E. BCP 38

F. Attacks, Vulnerabilities and Common Exploits – recon, scan, priv escalation, penetration, cleanup, backdoor

G. Security Audit & Validation

H. Risk Assessment

I. Change Management Process

J. Incident Response Framework

K. Computer Security Forensics

Knowing which addresses belong to RFC1918 (10/8, 172.16/12 and 192.168/16) and what RFC2827 describes (Source IP address spoofing). Besides that you could run into questions like: What does ISO 27001 describe?

Be familiar with all the rules, RFC’s and standards what they describe, since it’s not really usable knowledge, still it never hurts to learn anything new.

Misc

Other small topics that are not tested on the lab exam like Multicast and IPv6 could be tested on the written, but also on a very basic level. It would suffice to have a basic understanding of what multicast is and what multicast features the ASA has and what security features on Multicast IOS has.

I hope I was able to give a basic overview of what is tested in the CCIE Security Written exam and you will be able to have an idea on how the exam is put together. Good luck and enjoy all your studies!!

Rick Mur

CCIE2 #21946 (R&S, Service provider)

Sr. Support Engineer — IPexpert, Inc.

Print Friendly

I took the exam in San Jose, CA. Mainly because I was in the US a couple weeks for studying and vacation. During my studies there I felt really good about my level and really had the feeling I was in ‘the zone’. It was in the time that you could still reschedule your exam 30 days before instead of the current 90 days.  With that 30 day limit I was just within reach of rescheduling my exam to about 5 days after I finished the bootcamp I did.

I took a couple days off and did some really touristic stuff and really had a good time in San Francisco. After a good day in the Cisco Building C in San Jose I went home with a very neutral feeling. At 11.30pm I got the e-mail. YES I passed, #21946.

Now a lot changed. I got a very nice raise at my employer but it really took some effort. Mainly due to my age. I had to fight for it, but after a few meetings with the HR managers I got the salary I wanted and the growth plan I wanted.

The attitude I got from my co-workers really changed a lot! They really saw me as a senior engineer now and came to me with lot’s of questions and wanted me to verify about everything. Teaching basic classes about their own network and more stuff. I really enjoyed that feeling and it really felt the CCIE paid a lot more than just money!

After staying a couple months with my current employer I got this really nice opportunity to work for a company which is very close to my home. My previous work was at least a 1 to 2 hour drive and the new job would be a 5 minute drive.

I went there for a chat and that meeting only took about 45 minutes. Mainly because they knew that I had my CV and I just had 2 basic technical questions and I had to draw a basic LAN design which I just implemented. Afterwards the guy who interviewed me said: I just felt you were a great guy, you don’t need to ask a lot of questions, the feeling was immediately great and you really fitted in the group! Which in my opinion is about the best compliment I got so far.

So after debating a long time I chose to switch jobs! This was a big thing which I considered a long time and the day before I went on vacation I had my exit meeting with the HR manager, which he totally didn’t expect.

During my vacation I started working on the CCIE Service Provider big time! I bought a couple books and bought some training materials including videos. I read all the books and watched over 40 hours of video and came back with a VERY good understanding of the Service Provider topics.

After coming back I attended Cisco Networkers in Barcelona. It was my first time and I enjoyed it so much! The sessions that Cisco gives are just the BEST. The people that are giving the lectures are very good and a lot of them are ‘distinguished’ engineers.

I took the CCIE Service Provider techtorial which I really liked. We got a practice lab which was really comparable to the real lab. I could do it quite fast and didn’t have to much trouble with it.

After another month I joined my new employer. I changed from an outsourcing company to an integrator, which is a really different way of working. Instead of working on 1 project for 1 customer I’m now working on 10 projects for 10 customers. This means a lot in terms of skills you need to have, as I’m now really expected to advice the customer and defend my solutions.

I’m now so much entertained by the challenges I get, which are much more difficult than I got with my previous employer.

After another few months of studying and doing a lot of full-scale labs I passed my CCIE Service Provider also on the first attempt on May 19^th. I was so happy that I was able to pass 2 CCIE’s within a year and all before my 22^nd birthday.

After passing the SP exam, again a lot changed! Not in terms of how my colleagues saw me, but I got approached by a number of people that asked me to coach them and I was also approached to be doing CCIE lab development.

So another thing that can change your life. Now people approach you internationally and ask how you did it and what they have to do, to be at the same level. It’s such an awesome experience!

My last major change was that I joined IPexpert as Sr. Support Engineer, to assist in supporting the CCIE Community wherever I can and also co-author some of the BEST workbooks that are available. I really enjoy making the labs for students and it’s also a huge addition to keep your own knowledge up to date.

This is a big thing for CCIE’s. Once you got your digits, your knowledge just somehow erases from your memory for a big part. At least that’s what I experienced after a couple months. I just wasn’t at that high level anymore. It takes a lot of effort to keep up to date with everything and remain at CCIE level! I spend a lot of time in reading the Cisco Documentation and even doing R&S labs again, just to keep my knowledge ‘active’. People expect you to have a certain vast amount of knowledge and you have to keep proving it. The CCIE pays off, but it keeps taking time and effort! At least that’s my opinion if you want to be one of the best. The only way to reach and maintain your level if by constantly keep improving yourself, don’t loose focus and keep looking.

The downside of all the things you are doing and all the thousands of hours you spend on it, is that people could become jealous. It’s also something I experienced. Suddenly I was ‘selfish’ and ‘arrogant’. People even start forum topics just to bash at you. It’s something you have to deal with I guess. I like it, since some of the replies actually make sense and people start defending you. That’s such a great example of what a fantastic community this is.

So after a little more than a year of being a CCIE I switched jobs, I earned another CCIE, I joined the best CCIE training company to support the great community I got to be part of, I met dozens of CCIEs and I can say that my social community grew enormously. It’s been such a great experience and it still is. I keep discovering new things and I still learn so much every day! I now see the CCIE not as an end-goal, it’s a milestone on a path to become an ‘expert’ (or ‘guru’ as some people might call it). I hope I will reach that level one day, but on the other hand it’s hard to tell that, as you keep learning.

Keep looking, keep learning, stay hungry, don’t settle!

Rick Mur

CCIE2 #21946 (R&S / Service Provider)

Sr. Support Engineer – IPexpert, Inc.

URL: http://www.IPexpert.com

Print Friendly

The basic principle behind it, is that you, as an enterprise have multiple uplinks or even routers to the internet or SP backbone network where all your branch offices connect. Now in the modern networks you can’t rely on the line-protocol of an interface anymore, thus meaning you have to make sure you have an end-to-end check if your networks are still reachable. IGP’s are designed that always the path with the lowest cost and therefore the highest bandwidth is preferred. If you happen to have experience in the field on this, you might know that ISP’s always oversubscribe their links and therefore the fastest uplink is definitely not always the best. Due to congestion, maintenance or problems in the SP network you get bad performance in your own network. The redundant uplink is therefore not used as the metrics determined that the path is not as fast as the primary link.

PfR is build to overcome this problem and by doing several measurements the ‘best’ path is chosen (read not always the fastest). You can for example determine that VoIP and Video traffic should use another SP uplink as the bandwidth is not that much, you agreed several QoS policies on that link and traffic will be much more assured to arrive than on the fast non-QoS link.

Another technique we know that can do this is MPLS Traffic Engineering. It has less built-in features, but the basic idea is the same, though TE is designed to work within one network and even more specific in one area.

PfR has thousands of possibilities and hundreds of commands. If you want it could become extremely complex if you use several measurements and differentiate several links to use for different purposes. For the R&S lab exam I would expect that only the basic principle is tested and you have to be familiar with the technology. We are implementing this in a couple of our Vol.2 labs, so stay tuned for updates on that!

For the rest of all the geeky features: Know where to look it up! There is even a separate configuration guide available for PfR:

http://www.cisco.com/en/US/docs/ios/oer/configuration/guide/12_4t/oer_12_4t_book.html

PfR uses a strict ‘phased’ implementation so you are always certain to use the best values for your network and it gives you a good insight in the preferred working method. The image below is taken from the Cisco IOS Optimized Edge Routing Configuration Guide

Figure 1

During this series we will use these phases to explain how PfR works.

In this first article we will define all the steps necessary for getting PfR to work and set-up the necessary components and define interfaces.

The topology shows that we have 3 connections to our branch office (R9), 2 are transparent links, we don’t know how or what kind of Service Provider equipment is in the path and we have a third connection, trough an ISP with PPP connections were we are sure that it’s only 1 SP network.

Figure 2

PfR is designed to have 1 device that manages the entire setup; this includes pushing IGP or BGP information into the network and informing the other routers what to do.  This router is called the Master Controller and this is the single location where you configure the policies after which they are pushed to the other routers.

The other routers that carry uplinks to the SP networks are called Border Routers. These devices are very easy to setup, as they only need information on where the MC is located. It’s possible to combine these roles in one device, which could be useful in smaller networks.

First we configure all necessary connectivity configurations and do a quick check if our layer 2 is working.

R2(config-if)#do ping 100.1.124.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 100.1.124.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/4/16 ms
R2(config-if)#do ping 100.1.124.4
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 100.1.124.4, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms
R2(config-if)#do ping 100.1.29.9 
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 100.1.29.9, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms
R2(config-if)#do ping 100.1.26.6
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 100.1.26.6, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 16/17/20 ms
R2(config-if)#
R2(config-if)#do sh ip int brie | ex unas
Interface                  IP-Address      OK? Method Status                Protocol
Serial0/1/0.1              100.1.26.2      YES manual up                    up
FastEthernet1/0            100.1.124.2     YES manual up                    up
FastEthernet1/1            100.1.29.2      YES manual up                    up
R6(config-subif)#do sh ip int brie | ex unas
Interface                  IP-Address      OK? Method Status                Protocol
Serial0/1/0.1              100.1.26.6      YES manual up                    up
Multilink1                 100.1.69.6      YES manual up                    up
R6(config-subif)#do ping 100.1.69.9
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 100.1.69.9, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms
R6(config-subif)#do sh ip route  con
100.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
C       100.1.69.0/24 is directly connected, Multilink1
C       100.1.69.9/32 is directly connected, Multilink1
C       100.1.26.0/24 is directly connected, Serial0/1/0.1
R6(config-subif)#

Next we configure the ‘border routers‘, we start with these, as the configuration is very easy. To ensure no rogue devices will screw up our PfR setup, it’s possible to configure authentication. We connect over the loopback addresses and therefore it’s necessary to specify the interface were we want to source the communication from.

R2 & R4
!
key chain OER_AUTH
key 1
key-string IPEXPERT
!
oer border
local Loopback0
master 100.1.1.1 key-chain OER_AUTH
R4(config)#do sh oer border
OER BR 100.1.4.4 ACTIVE, MC 100.1.1.1 UP/DOWN: DOWN
Conn Status: CONNECT FAILED
OER Netflow Status: ENABLED, PORT: 3949
Version: 2.2  MC Version: 0.0
Exits

As you can see, not really intelligent stuff on the border routers, really everything is controlled from the master. When the session to the master is not ‘up’ yet, the border doesn’t know anything. Not even which interfaces are participating.

Last is to configure the ‘master controller‘.

Besides configuring the borders in this network, you also need to specify which interfaces are used for communication with the SP and which are for internal use. These are interface names on the border routers and not on the master.

R1
!
key chain OER_AUTH
key 1
key-string IPEXPERT
!
oer master
logging
!
border 100.1.2.2 key-chain OER_AUTH
interface FastEthernet1/0 internal
interface FastEthernet1/1 external
interface Serial0/1/0.1 external
!
border 100.1.4.4 key-chain OER_AUTH
interface FastEthernet0/0 internal
interface FastEthernet0/1 external

After configuring the master we see the neighbor sessions coming up, to see this information it’s necessary to enable ‘logging’ under the OER process.

*Sep  6 13:01:26.113: %OER_MC-5-NOTICE: BR 100.1.2.2 UP
*Sep  6 13:01:26.125: %OER_MC-5-NOTICE: BR 100.1.2.2 IF Se0/1/0.1 UP
*Sep  6 13:01:26.165: %OER_MC-5-NOTICE: BR 100.1.2.2 IF Fa1/1 UP
*Sep  6 13:01:26.165: %OER_MC-5-NOTICE: BR 100.1.2.2 IF Fa1/0 UP
*Sep  6 13:01:26.165: %OER_MC-5-NOTICE: BR 100.1.2.2 Active
*Sep  6 13:01:26.165: %OER_MC-5-NOTICE: MC Active
*Sep  6 13:01:26.213: %OER_MC-5-NOTICE: BR 100.1.4.4 UP
*Sep  6 13:01:26.229: %OER_MC-5-NOTICE: BR 100.1.4.4 IF Fa0/1 UP
*Sep  6 13:01:26.257: %OER_MC-5-NOTICE: BR 100.1.4.4 IF Fa0/0 UP
*Sep  6 13:01:26.257: %OER_MC-5-NOTICE: BR 100.1.4.4 Active
R1#sh oer master bord det
Border           Status   UP/DOWN             AuthFail  Version
100.1.4.4        ACTIVE   UP       00:01:00          0  2.2
Fa0/1           EXTERNAL UP
Fa0/0           INTERNAL UP
External            Capacity      Max BW   BW Used    Load Status          Exit Id
Interface            (kbps)       (kbps)    (kbps)    (%)
---------           --------      ------   ------- ------- ------           ------
Fa0/1           Tx    100000       75000         0       0 UP                    6
Rx                100000         0       0
--------------------------------------------------------------------------------
Border           Status   UP/DOWN             AuthFail  Version
100.1.2.2        ACTIVE   UP       00:01:00          0  2.2
Se0/1/0.1       EXTERNAL UP
Fa1/1           EXTERNAL UP
Fa1/0           INTERNAL UP
External            Capacity      Max BW   BW Used    Load Status          Exit Id
Interface            (kbps)       (kbps)    (kbps)    (%)
---------           --------      ------   ------- ------- ------           ------
Se0/1/0.1       Tx      1544        1158         0       0 UP                    5
Rx                  1544         0       0
Fa1/1           Tx    100000       75000         0       0 UP                    4
Rx                100000         0       0

The master controller knows about the interfaces, their role and even capacity on how much bandwidth is available and currently used (depended on how timers are set).

R2(config-router)#do sh oer border
OER BR 100.1.2.2 ACTIVE, MC 100.1.1.1 UP/DOWN: UP 00:13:55,
Auth Failures: 0
Conn Status: SUCCESS
OER Netflow Status: ENABLED, PORT: 3949
Version: 2.2  MC Version: 2.2
Exits
Se0/1/0.1       EXTERNAL
Fa1/0           INTERNAL
Fa1/1           EXTERNAL
R2(config-router)#
PL-Pod123#4
[Resuming connection 4 to r4 ... ]
*Sep  6
R4(config-router)#do sh oer border
OER BR 100.1.4.4 ACTIVE, MC 100.1.1.1 UP/DOWN: UP 00:04:37,
Auth Failures: 0
Conn Status: SUCCESS
OER Netflow Status: ENABLED, PORT: 3949
Version: 2.2  MC Version: 2.2
Exits
Fa0/0           INTERNAL
Fa0/1           EXTERNAL
R4(config-router)#

Now the master has been configured, the border routers know which interfaces they should monitor and NetFlow is automatically enabled to share the utilization status with the master controller.

Now the PfR setup is done we can start with the cool stuff. In the next part the profile phase will commence.

Stay tuned!

Rick Mur

CCIE2 #21946 (R&S / Service Provider)

Sr. Support Engineer – IPexpert, Inc.

Print Friendly

Another great option is that it’s possible to reserve a specific amount of bandwidth throughout the entire SP network. This way you can ensure the path from A to B has a dedicated amount of bandwidth reserved on the link. With even more advanced features like Fast Re-Route (FRR) it’s possible to have 2 different paths leading to the same destination and when the primary fails, the secondary path will take over, without the end-nodes even noticing it. Due to the used IOS versions on the SP lab it’s not possible to configure FRR as this is only supported in 12.0S and not in 12.2S (the version used on the 7200 routers in the SP lab), therefore I will not focus on this technology.

Quite some people get confused that MPLS TE only uses a label-stack of 1 throughout the network as many people think it has 2 (one IGP label and one TE label). This is not the case, MPLS TE only uses a single label and therefore it’s not required to configure LDP/TDP when using TE.  Using the RSVP protocol a specific label is propagated throughout the SP network, which is the label the tunnel will be using when forwarding traffic. Due to this, it’s not necessary for the MPLS network to know the source and destination prefix.

This simplifies the forwarding and saves a lot of resources on core routers.

One important requirement of MPLS TE is that it requires a link-state protocol to be able to work. When figuring out where to send the RSVP packet to the router does a SPF calculation and to be able to do that it needs an entire overview of the network. Protocols like RIP, EIGRP and BGP can’t deliver that, only (single level) IS-IS and (single area) OSPF can. Still it’s possible to create inter-area TE tunnels. This is what we will investigate in this article.

Below is the diagram that is used for this, 2 OSPF areas are connected to the backbone area. We first will create 2 normal intra-area TE tunnels and finally an inter-area tunnel between the 2 172.16.x.x prefixes. On all 3 tunnels we will use a different method for getting traffic in the tunnel.

Another thing to get confused by is that when creating a tunnel only traffic originating from that router will go through the TE tunnel, all return traffic will take the normal routed or (LDP) labeled path. You need to configure a tunnel on both ends to have all the traffic tunneled through the network. That way it’s also possible to create asymmetric traffic, as the return traffic can take an entirely different path depending on the tunnel configuration.

Please note that you need an ISR like the 1841, 2811 on our ProctorLabs.com racks running 12.4T or a 7200 running 12.0S or 12.2S to test this as the inter-area tunnels won’t work on a lower platform or other IOS version.

First we enable OSPF on all routes within the respective area and check for full reach-ability.

R2(config-router)#do sh ip route ospf
172.16.0.0/32 is subnetted, 2 subnets
O IA    172.16.8.1 [110/69] via 150.1.24.4, 00:00:27, FastEthernet1/0
150.1.0.0/16 is variably subnetted, 11 subnets, 2 masks
O IA    150.1.8.8/32 [110/69] via 150.1.24.4, 00:00:27, FastEthernet1/0
O IA    150.1.7.7/32 [110/68] via 150.1.24.4, 00:02:01, FastEthernet1/0
O IA    150.1.6.6/32 [110/67] via 150.1.24.4, 00:03:19, FastEthernet1/0
O IA    150.1.5.5/32 [110/3] via 150.1.24.4, 00:04:02, FastEthernet1/0
O       150.1.4.4/32 [110/2] via 150.1.24.4, 00:04:02, FastEthernet1/0
O IA    150.1.56.0/24 [110/66] via 150.1.24.4, 00:03:29, FastEthernet1/0
O       150.1.45.0/24 [110/2] via 150.1.24.4, 00:04:02, FastEthernet1/0
O IA    150.1.67.0/24 [110/67] via 150.1.24.4, 00:01:56, FastEthernet1/0
O IA    150.1.78.0/24 [110/68] via 150.1.24.4, 00:00:22, FastEthernet1/0

We do a little end-to-end verification

R2(config-router)#do ping 172.16.8.1 sour lo1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.8.1, timeout is 2 seconds:
Packet sent with a source address of 172.16.2.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 32/32/36 ms

R2(config-router)#do trace 172.16.8.1
Type escape sequence to abort.
Tracing the route to 172.16.8.1
1 150.1.24.4 0 msec 0 msec 0 msec
2 150.1.45.5 4 msec 0 msec 0 msec
3 150.1.56.6 16 msec 16 msec 16 msec
4 150.1.67.7 16 msec 16 msec 16 msec
5 150.1.78.8 20 msec *  16 msec
R2(config-router)#

Connectivity is established end-to-end across all 3 OSPF areas.

Now it’s time to enable MPLS Traffic Engineering and RSVP on the routing protocol and on all interfaces that we want to be participating. Notice that we do NOT enable ‘mpls ip’ on the interfaces.

The configuration of the most crucial routers is shown:

R5

mpls traffic-eng tunnels
!
interface FastEthernet0/1
mpls traffic-eng tunnels
ip rsvp bandwidth
!
interface Serial0/1.1
mpls traffic-eng tunnels
ip rsvp bandwidth
!
router ospf 1
mpls traffic-eng router-id loopback0
mpls traffic-eng area 0
mpls traffic-eng area 24
!

R6

mpls traffic-eng tunnels
!
interface FastEthernet0/1
mpls traffic-eng tunnels
ip rsvp bandwidth
!
interface Serial0/1.1
mpls traffic-eng tunnels
ip rsvp bandwidth
!
router ospf 1
mpls traffic-eng router-id loopback0
mpls traffic-end area 0
mpls traffic-eng area 24
!

After configuring all the routers with the correct TE configuration, we can verify by using the ‘show mpls traffic-eng topology’ and see how every interface is learned (within the specified areas) and what bandwidth reservations have been made on which priority.

The first 2 lines show that this router is active for TE in 2 areas. This is to see there is a clear separation for how this technology works within areas.

The bandwidth that could be allocated depends on the configuration of the ‘ip rsvp bandwidth’ command. By default this command will use all possible bandwidth, which is of course 75% of the true bandwidth (the ‘max-reserved-bandwidth’ command could change this).

R6#sh mpls traf topo
My_System_id: 150.1.6.6 (ospf 1  area 0)
My_System_id: 150.1.6.6 (ospf 1  area 78)
Signalling error holddown: 10 sec Global Link Generation 10
IGP Id: 150.1.5.5, MPLS TE Id:150.1.5.5 Router Node  (ospf 1  area 0)
link[0]: Point-to-Point, Nbr IGP Id: 150.1.6.6, nbr_node_id:6, gen:1
frag_id 11, Intf Address:150.1.56.5, Nbr Intf Address:150.1.56.6
TE metric:64, IGP metric:64, attribute flags:0x0
SRLGs: None
physical_bw: 1544 (kbps), max_reservable_bw_global: 1158 (kbps)
max_reservable_bw_sub: 0 (kbps)
Global Pool       Sub Pool
Total Allocated   Reservable        Reservable
BW (kbps)         BW (kbps)         BW (kbps)
---------------   -----------       ----------
bw[0]:            0             1158                0
bw[1]:            0             1158                0
bw[2]:            0             1158                0
bw[3]:            0             1158                0
bw[4]:            0             1158                0
bw[5]:            0             1158                0
bw[6]:            0             1158                0
bw[7]:            0             1158                0
IGP Id: 150.1.8.8, MPLS TE Id:150.1.8.8 Router Node  (ospf 1  area 78)
link[0]: Broadcast, DR: 150.1.78.7, nbr_node_id:3, gen:5
frag_id 1, Intf Address:150.1.78.8
TE metric:1, IGP metric:1, attribute flags:0x0
SRLGs: None
physical_bw: 100000 (kbps), max_reservable_bw_global: 75000 (kbps)
max_reservable_bw_sub: 0 (kbps)
Global Pool       Sub Pool
Total Allocated   Reservable        Reservable
BW (kbps)         BW (kbps)         BW (kbps)
---------------   -----------       ----------
bw[0]:            0            75000                0
bw[1]:            0            75000                0
bw[2]:            0            75000                0
bw[3]:            0            75000                0
bw[4]:            0            75000                0
bw[5]:            0            75000                0
bw[6]:            0            75000                0
bw[7]:            0            75000                0
IGP Id: 150.1.67.7, Network Node  (ospf 1  area 78)
link[0]: Broadcast, Nbr IGP Id: 150.1.7.7, nbr_node_id:4, gen:2
link[1]: Broadcast, Nbr IGP Id: 150.1.6.6, nbr_node_id:7, gen:2
(output omitted)

The information of the above output is a combination of the topology information received from the IGP and the flows that RSVP has received and allocated bandwidth.

With the enabling of TE on OSPF, the database now receives a totally new LSA type. TE link information is not carried in Type 1 LSA’s as it can be enabled or disabled per interface. The so-called ‘opaque’-LSA is used for carrying this information. There are 3 types of opaque LSA’s. One for area scope, one for AS scope and one for link scope. You can gues which type is used for TE. It’s the area scope LSA.

If we issue a ‘show ip ospf database opaque-area’ we see the following:

R6#sh ip ospf data opaque-area
OSPF Router with ID (150.1.6.6) (Process ID 1)
Type-10 Opaque Link Area Link States (Area 0)
LS age: 1182
Options: (No TOS-capability, DC)
LS Type: Opaque Area Link
Link State ID: 1.0.0.11
Opaque Type: 1
Opaque ID: 11
Advertising Router: 150.1.5.5
LS Seq Number: 80000003
Checksum: 0xBC7A
Length: 132
Fragment number : 11
Link connected to Point-to-Point network
Link ID : 150.1.6.6
Interface Address : 150.1.56.5
Neighbor Address : 150.1.56.6
Admin Metric : 64
Maximum bandwidth : 193000
Maximum reservable bandwidth : 144750
Number of Priority : 8
Priority 0 : 144750      Priority 1 : 144750
Priority 2 : 144750      Priority 3 : 144750
Priority 4 : 144750      Priority 5 : 144750
Priority 6 : 144750      Priority 7 : 144750
Affinity Bit : 0x0
IGP Metric : 64
Number of Links : 1
Type-10 Opaque Link Area Link States (Area 78)
LS age: 841
Options: (No TOS-capability, DC)
LS Type: Opaque Area Link
Link State ID: 1.0.0.0
Opaque Type: 1
Opaque ID: 0
Advertising Router: 150.1.6.6
LS Seq Number: 80000001
Checksum: 0x4B9F
Length: 28
Fragment number : 0
MPLS TE router ID : 150.1.6.6
Number of Links : 0
(output omitted)

Now it’s time to get the tunnels up and running.

First we configure the intra-area tunnel between R2 and R5. Remember we need to configure a tunnel on both ends of the link. We use interface tunnel25 for this.

R2

interface Tunnel25
ip unnumbered Loopback0
tunnel destination 150.1.5.5
tunnel mode mpls traffic-eng
tunnel mpls traffic-eng path-option 1 dynamic

R5

interface Tunnel25
ip unnumbered Loopback0
tunnel destination 150.1.2.2
tunnel mode mpls traffic-eng
tunnel mpls traffic-eng path-option 1 dynamic

If all works out then we should see:

*Aug 25 06:30:45.175: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel25, changed state to up
R5(config-if)#do sh mpls traf tun
Name: R5_t25                              (Tunnel25) Destination: 150.1.2.2
Status:
Admin: up         Oper: up     Path: valid       Signalling: connected
path option 1, type dynamic (Basis for Setup, path weight 2)
Config Parameters:
Bandwidth: 0        kbps (Global)  Priority: 7  7   Affinity: 0x0/0xFFFF
Metric Type: TE (default)
AutoRoute:  disabled  LockDown: disabled  Loadshare: 0        bw-based
auto-bw: disabled
Active Path Option Parameters:
State: dynamic path option 1 is active
BandwidthOverride: disabled  LockDown: disabled  Verbatim: disabled
InLabel  :  -
OutLabel : FastEthernet0/1, 16
RSVP Signalling Info:
Src 150.1.5.5, Dst 150.1.2.2, Tun_Id 25, Tun_Instance 3
RSVP Path Info:
My Address: 150.1.45.5
Explicit Route: 150.1.45.4 150.1.24.4 150.1.24.2 150.1.2.2
Record   Route:   NONE
Tspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
RSVP Resv Info:
Record   Route:   NONE
Fspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
History:
Tunnel:
Time since created: 2 minutes, 42 seconds
Time since path change: 1 minutes, 12 seconds
Number of LSP IDs (Tun_Instances) used: 3
Current LSP:
Uptime: 1 minutes, 12 seconds
R5(config-if)#

The tunnel is up, but now we need to route traffic over that link.

R5(config-if)#do sh ip route | in 150.1.2.2
O       150.1.2.2/32 [110/3] via 150.1.45.4, 03:13:04, FastEthernet0/1

The routing table still tells us that traffic for R2′s loopback interface is still routed using the normal path. As you see, no MPLS label attached.

R5(config-if)#do trace 150.1.2.2
Type escape sequence to abort.
Tracing the route to 150.1.2.2
1 150.1.45.4 0 msec 0 msec 4 msec
2 150.1.24.2 0 msec *  0 msec
R5(config-if)#

Somehow we need to get the traffic into the tunnel.

There are 2 ways of doing that. The first of which is called ‘autoroute announce’. When enabling this on the tunnel interface, the interface will be treated as part of the IGP and therefore it becomes an option to choose. Please keep in mind the metric/cost value as other prefixes can go over that tunnel as well if the cost is less than the normal routed path.

After the command is added to the tunnel configuration, the next-hop is no longer a FastEthernet connection, but the Tunnel interface:

R5(config-if)#do sh ip route | in 150.1.2.2
O       172.16.2.1 [110/3] via 150.1.2.2, 00:00:00, Tunnel25
O       150.1.2.2/32 [110/3] via 150.1.2.2, 00:00:00, Tunnel25

Like previously said, more prefixes could prefer the tunne interface as the cost might be lower than the normal routed path. If we check connectivity we now see a label attached:

R5(config-if)#do trace 150.1.2.2
Type escape sequence to abort.
Tracing the route to 150.1.2.2
1 150.1.45.4 [MPLS: Label 16 Exp 0] 0 msec 4 msec 0 msec
2 150.1.24.2 0 msec *  0 msec
R5(config-if)#

On the ‘transit’ router R4 we see that for this tunnel ID (notice that it’s not a prefix), a label is received which is popped and then transmitted on FastEthernet0/0

R4#sh mpls for
Local  Outgoing      Prefix            Bytes Label   Outgoing   Next Hop
Label  Label or VC   or Tunnel Id      Switched      interface
16     Pop Label     150.1.5.5 25 [3]  252           Fa0/0      150.1.24.2

Now this side of the network has been configured properly. On the other side we want to have more influence in the path the tunnel takes to reach it’s destination. In this topology it’s not really exciting, but in real-life networks the power of TE tunnels is that you can specify an explicit path, where every hop is defined. That way you can defer from your IGP. Still the link-state information is required, as the path needs to be validated before the tunnel comes up.

R6

interface Tunnel68
ip unnumbered Loopback0
tunnel destination 150.1.8.8
tunnel mode mpls traffic-eng
tunnel mpls traffic-eng path-option 1 explicit name R6R8
!
ip explicit-path name R6R8 enable
next-address 150.1.67.7
next-address 150.1.78.8
next-address 150.1.8.8
!

ip route 150.1.8.8 255.255.255.255 Tunnel68

The second way of getting traffic into a MPLS TE tunnel is by static routing. The tunnel interface is just a regular interface on the router, so it’s possible to assign static routes to that interface. The downside of this compared to the ‘autoroute announce’ is that it possibly creates a black hole when the tunnel doesn’t properly go down. This is the preferred method if you can’t have any other prefixes choose this path to a tunnel and you want to explicitly control which traffic to which prefixes can go through the tunnel.

A third way to get traffic into the tunnel is PBR (Policy Based Routing), but we are already influencing the routing decision and MPLS TE was designed to create a centralized way of creating ‘PBR-like-paths’ throughout the entire network.

After both tunnel interfaces are created on R6 and R8, the traffic is routed over an explicitly configured path over R7 as transit:

R7#sh mpls for
Local  Outgoing      Prefix            Bytes Label   Outgoing   Next Hop
Label  Label or VC   or Tunnel Id      Switched      interface
16     Pop Label     150.1.8.8 68 [8]  126           Fa0/1      150.1.67.6
17     Pop Label     150.1.6.6 78 [5]  686           Fa0/0      150.1.78.8

Now the interesting part starts as we are building a tunnel over places that the originating router has absolutely no control. This is why the standard had to be ‘stretched’ and the control can be ‘delegated’ to an ABR. Since the router doesn’t have an overview of the other areas you can’t use a ‘dynamic’ path here. You have to use explicit paths until the respective ABR’s, these are called with a ‘loose’ next-hop, so they have to figure out what they think is the best next-hop for the traffic.

To ensure tunnel28 works we have to specify an explicit-path and we only specify the ABR’s. Those routers will find the correct path and the tunnel comes up. For a long time this technique was only available in the 12.0S and 12.2S images. Since 12.4T Advanced Enterprise Services this also works for the ISR routers. If it’s not supported you will receive a log message when you try to enable MPLS TE for a second area.

R2

interface Tunnel28
ip unnumbered Loopback0
tunnel destination 150.1.8.8
tunnel mode mpls traffic-eng
tunnel mpls traffic-eng path-option 1 explicit name R2R8
!
ip explicit-path name R2R8
next-address loose 150.1.5.5
next-address loose 150.1.6.6
next-address loose 150.1.8.8
!
ip route 172.16.8.1 255.255.255.255 Tunnel28

The ‘loose’ keyword is the one that does the trick. If nothing is specified ‘strict’ is used.

R8(cfg-ip-expl-path)#next ?
WORD    Enter IP address (A.B.C.D)
loose   Target address is loose
strict  Target address is strict
R8(cfg-ip-expl-path)#next

Now if we ping the 172.16.8.1 prefix, the traffic is send over an end-to-end label switched path.

R2(config)#do trace 172.16.8.1
Type escape sequence to abort.
Tracing the route to 172.16.8.1
1 150.1.24.4 [MPLS: Label 18 Exp 0] 52 msec 52 msec 52 msec
2 150.1.45.5 [MPLS: Label 16 Exp 0] 52 msec 52 msec 52 msec
3 150.1.56.6 [MPLS: Label 16 Exp 0] 36 msec 36 msec 36 msec
4 150.1.67.7 [MPLS: Label 18 Exp 0] 32 msec 36 msec 36 msec
5 150.1.78.8 20 msec *  16 msec
R2(config)#

As you can see the PATH information (RSVP) is only available for the area in which the router resides (area 24).

R2(config)#do sh mpls traf tun tun28
Name: R2_t28                              (Tunnel28) Destination: 150.1.8.8
Status:
Admin: up         Oper: up     Path: valid       Signalling: connected
path option 1, type explicit R2R8 (Basis for Setup, path weight 2)
Config Parameters:
Bandwidth: 0        kbps (Global)  Priority: 7  7   Affinity: 0x0/0xFFFF
Metric Type: TE (default)
AutoRoute:  disabled  LockDown: disabled  Loadshare: 0        bw-based
auto-bw: disabled
Active Path Option Parameters:
State: explicit path option 1 is active
BandwidthOverride: disabled  LockDown: disabled  Verbatim: disabled
InLabel  :  -
OutLabel : FastEthernet1/0, 18
RSVP Signalling Info:
Src 150.1.2.2, Dst 150.1.8.8, Tun_Id 28, Tun_Instance 10
RSVP Path Info:
My Address: 150.1.24.2
Explicit Route: 150.1.24.4 150.1.45.4 150.1.45.5 150.1.5.5
150.1.6.6* 150.1.8.8*
Record   Route:
Tspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
RSVP Resv Info:
Record   Route:  150.1.45.4 150.1.56.5 150.1.67.6 150.1.78.7
150.1.78.8
Fspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
Shortest Unconstrained Path Info:
Path Weight: UNKNOWN
Explicit Route:  UNKNOWN
History:
Tunnel:
Time since created: 2 hours, 25 minutes
Time since path change: 2 hours, 22 minutes
Number of LSP IDs (Tun_Instances) used: 10
Current LSP:
Uptime: 2 hours, 22 minutes

On the first ABR, the path information for area 0 is known. It also remembers (record route) which routers have been crossed. This information is used for finding the way back.

R5#sh mpls traf tun name R2_t28
LSP Tunnel R2_t28 is signalled, connection is up
InLabel  : FastEthernet0/1, 16
OutLabel : Serial0/1/0.1, 16
RSVP Signalling Info:
Src 150.1.2.2, Dst 150.1.8.8, Tun_Id 28, Tun_Instance 10
RSVP Path Info:
My Address: 150.1.5.5
Explicit Route: 150.1.56.6 150.1.6.6 150.1.8.8*
Record   Route:  150.1.45.4 150.1.24.2
Tspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
RSVP Resv Info:
Record   Route:  150.1.67.6 150.1.78.7 150.1.78.8
Fspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits

The last ABR knows the path to the destination, so there the PATH info shows information about area 78.

R6#sh mpls traf tun name R2_t28
LSP Tunnel R2_t28 is signalled, connection is up
InLabel  : Serial0/1/0.1, 16
OutLabel : FastEthernet0/1, 18
RSVP Signalling Info:
Src 150.1.2.2, Dst 150.1.8.8, Tun_Id 28, Tun_Instance 10
RSVP Path Info:
My Address: 150.1.6.6
Explicit Route: 150.1.67.7 150.1.78.7 150.1.78.8 150.1.8.8
Record   Route:  150.1.56.5 150.1.45.4 150.1.24.2
Tspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
RSVP Resv Info:
Record   Route:  150.1.78.7 150.1.78.8
Fspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits

To verify the traffic is truly label switched, we remove the 172.16.x.x subnets from OSPF.

R4#sh ip route 172.16.2.1
% Network not in table

But R2 and R8 can still ping each other’s interfaces, as this is end-to-end label switched now.

R2(config-router)#do ping 172.16.8.1 sour lo1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.8.1, timeout is 2 seconds:
Packet sent with a source address of 172.16.2.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 32/34/36 ms
R2(config-router)#

At first it seems an impossible achievement, since the local area behavior of this fantastic technology, but with a little tweaks and by delegating control to other routers it’s still possible to build inter-area MPLS Traffic Engineering tunnels.

Thanks!

Rick Mur

CCIE2 #21946 (R&S / Service Provider)

Sr. Support Engineer – IPexpert, Inc.

Print Friendly

I’d like to kick-off with some very basic stuff. Since people are scared of the new topics like MPLS and L3VPN on the blueprint. I think it’s a good idea to show you how easy it is to get MPLS up and running through LDP and BGP and configure some specific LDP features.

The topology is that we have 2 sites separated by a Frame-Relay cloud. In each site OSPF area 0 runs as IGP and BGP is used to advertise all the prefixes to the other site. The prefixes in the 172.16.x.x range are not allowed to be directly configured under OSPF.  The goal of this task is to get an end-to-end label switched path (LSP) between 172.16.1.1 and 172.16.2.1.

First OSPF is configured on all routers and enabled on the R1-R3, R3-R4, R5-R6 and R6-R2 segments. After all OSPF neighbors are configured and have a state of FULL we verify full reachability.

R1:

R1#ping 150.1.34.4
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 150.1.34.4, timeout is 2 seconds:\
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms

R2:

R2#ping 150.1.56.5
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 150.1.56.5, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms

In both the sites we have achieved connectivity. Now we want to set-up LDP local within the site. To enable MPLS is one of the easiest things to do. There are some things to pay attention to.

LDP or TDP?

When ‘tag-switching’ was introduced by Cisco they created their own label distribution protocol called TDP. After tag switching became standardized they renamed it to MPLS and TDP was replaced with LDP. Most features of TDP are exactly the same in LDP with some added advantages like MD5 authentication.  Since this is not explicitly stated on the blueprint it’s possible they could ask for both. The difference is not that hard. The only feature which is not in TDP is authentication and TDP uses TCP port 711 instead of 646 for LDP. Therefore this article will focus only on LDP.

Different IOS, different label protocol

During the migration from TDP to LDP, Cisco changed the default protocol quite some times. Therefore it’s not really easy to say which protocol will be the default. Now all the routers in the lab are ISR’s and running 12.4T, you might say that it’s almost certain that LDP is the default.

I personally always configure the global command ‘mpls label protocol ldp’ to be sure the right protocol is used on all interfaces, please be careful when configuring this, since the same command could be on an interface as well. The one configured on the interface is leading.

What rests is enabling MPLS on all interfaces within the site. This is done with the single command: ‘mpls ip’. That’s it!

But wait..now we enabled MPLS on all interfaces, but the LDP neighbors are not coming up. Why is that? After issuing ‘debug mpls transport events’, we see the following:

R6:

R6#debug mpls ldp transport events

Aug 21 09:46:14.292: ldp: Send ldp hello; Ethernet0/1, src/dst 150.1.56.6/224.0.0.2, inst_id 0
Aug 21 09:46:15.844: ldp: Rcvd ldp hello; Ethernet0/0, from 150.1.26.2 (150.1.2.2:0), intf_id 0, opt 0xC
Aug 21 09:46:15.844: ldp: ldp Hello from 150.1.26.2 (150.1.2.2:0) to 224.0.0.2, opt 0xC
Aug 21 09:46:15.844: ldp: local idb = Ethernet0/0, holdtime = 15000, peer 150.1.26.2 holdtime = 15000
Aug 21 09:46:15.848: ldp: Link intvl min cnt = 2, intvl = 5000, idb = Ethernet0/0
Aug 21 09:46:15.848: ldp: Opening ldp conn; adj 0x65A44E84, 150.1.6.6 <-> 150.1.2.2; with normal priority
Aug 21 09:46:15.848: ldp: Found adj 0x65A44E84 for 150.1.2.2 (Hello xport addr opt)
Aug 21 09:46:15.848: ldp: MD5 setup for neighbor 150.1.2.2; password changed to [nil]Aug 21 09:46:15.848: ldp: No route to peer 150.1.2.2; set LDP_CTX_HANDLE_ROUTEUP

The LDP multicast hello is received, but there is no route to the loopback address of the other router. LDP uses the router-id as destination IP address for the TCP connection.

After all loopback interfaces are advertised in OSPF, all LDP neighbors are up and running.

R2:

R2(config)#router ospf 1
R2(config-router)#network 150.1.2.2 0.0.0.0 area 0

13:14:14: %LDP-5-NBRCHG: LDP Neighbor 150.1.6.6:0 is UP

To ensure that one specific loopback interfaces is chosen when setting up LDP neighbors, it’s highly recommended to use the command ‘mpls ldp router-id loopback0′. That way you are certain that you won’t break anything if you configure another loopback interface with a higher IP address, which is not advertised in an IGP and it could take a long time before you discover it.

Now all prefixes within the IGP network are receiving a label, which is advertised to the neighbor. That way within the site all prefixes are ‘label switched’!

R2:

R2#sh mpls forwarding-table
Local  Outgoing      Prefix            Bytes Label   Outgoing   Next Hop

Label  Label or VC   or Tunnel Id      Switched      interface
16     Pop Label     150.1.6.6/32      0             Fa1/0      150.1.26.6
17     Pop Label     150.1.56.0/24     0             Fa1/0      150.1.26.6
18     18            150.1.5.5/32      0             Fa1/0      150.1.26.6

Let’s see if it really does what it promises:

R2:

R2#trace 150.1.5.5
Type escape sequence to abort.
Tracing the route to 150.1.5.5
1 150.1.26.6 [MPLS: Label 18 Exp 0] 4 msec 4 msec 0 msec
2 150.1.56.5 4 msec *  0 msec

The traffic from R2 to the loopback of R5 is getting label switched over R6. R6 pops the label as it is the last hop to the destined prefix. This technique is called Penultimate Hop Popping or PHP.

It saves an extra look-up in the MPLS and IP forwarding table. The for last hop knows the prefix is directly connected to the next router, if it should only swap the label R5 had to do a MPLS lookup and had seen it needs to strip the label and then a IP forwarding lookup needs to be done to know the final destination interface. When this label is stripped on the router before, R5 only needs to do an IP forwarding lookup, saving time and resources.

Now label switching within the site is done, we need to get inter-site label switching enabled.

LDP will not work on this interface as LDP only uses the IGP learned prefixes for allocating and advertising labels and only BGP is enabled between R4 and R5. We will be using the BGP way of advertising labels.

First we set-up a peering between R4 and R5 and perform a mutual redistribution between OSPF and BGP on both sites. To enable allocating and advertising labels for BGP is quite easy to configure. Just add ‘neighbor x.x.x.x send-label’ to the configuration. With a ‘show ip bgp labels’ you can see all labels.

R5:

R5(config-if)#do sh ip bgp labels

Network          Next Hop         In Label/Out Label
150.1.1.1/32     150.1.45.4       21/18
150.1.2.2/32     150.1.56.6       18/nolabel
150.1.3.3/32     150.1.45.4       22/19
150.1.5.5/32     0.0.0.0          imp-null/nolabel
150.1.6.6/32     150.1.56.6       19/nolabel
150.1.13.0/24    150.1.45.4       23/17
150.1.26.0/24    150.1.56.6       17/nolabel
150.1.34.0/24    150.1.45.4       24/imp-null
150.1.45.0/24    150.1.45.4       imp-null/imp-null
150.1.45.0/24    0.0.0.0          imp-null/nolabel
150.1.45.4/32    0.0.0.0          16/nolabel
150.1.45.5/32    150.1.45.4       nolabel/16
150.1.56.0/24    0.0.0.0          imp-null/nolabel
172.16.1.1/32    150.1.45.4       25/20

Now connectivity is established between R1 and R2:

R1:

R1#ping 150.1.2.2 sour lo0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 150.1.2.2, timeout is 2 seconds:
Packet sent with a source address of 150.1.1.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 60/61/64 ms

When using traceroute you can see an end-to-end label switched path is established and we comply with the goal of this task:

R1

R1#trace 150.1.2.2
Type escape sequence to abort.
Tracing the route to 150.1.2.2
1 150.1.13.3 [MPLS: Label 24 Exp 0] 96 msec 96 msec 96 msec
2 150.1.34.4 [MPLS: Label 29 Exp 0] 152 msec 100 msec 100 msec
3 150.1.45.5 [MPLS: Label 18 Exp 0] 64 msec 64 msec 64 msec
4 150.1.56.6 [MPLS: Label 24 Exp 0] 64 msec 64 msec 64 msec
5 150.1.26.2 32 msec *  40 msec

We issue a ‘redistribute connected subnets’ on both R1 and R2 to make sure that the 172.16.x.1 prefix is also advertised. After trying to ping the prefix, we see it’s not working. Why not?

R5:

R5#sh ip route 172.16.2.1
Routing entry for 172.16.2.1/32
Known via "ospf 1", distance 110, metric 20, type extern 2, forward metric 20
Last update from 150.1.56.6 on Ethernet0/1, 00:00:00 ago
Routing Descriptor Blocks:
* 150.1.56.6, from 150.1.2.2, 00:00:00 ago, via Ethernet0/1
Route metric is 20, traffic share count is 1

It’s known on R5, bot not on R1

R1:

R1#sh ip route 172.16.2.1
% Subnet not in table

Apparently it’s not getting advertised over the different sites. It’s also not visible in the BGP table on R5.

R5:

R5(config-router)#do sh ip bgp
BGP table version is 33, local router ID is 150.1.5.5
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

Network          Next Hop            Metric LocPrf Weight Path
*> 150.1.1.1/32     150.1.45.4              21             0 200 ?
*> 150.1.2.2/32     150.1.56.6              21         32768 ?
*> 150.1.3.3/32     150.1.45.4              11             0 200 ?
*> 150.1.5.5/32     0.0.0.0                  0         32768 ?
*> 150.1.6.6/32     150.1.56.6              11         32768 ?
*> 150.1.13.0/24    150.1.45.4              20             0 200 ?
*> 150.1.26.0/24    150.1.56.6              20         32768 ?
*> 150.1.34.0/24    150.1.45.4               0             0 200 ?
*  150.1.45.0/24    150.1.45.4               0             0 200 ?
*>                  0.0.0.0                  0         32768 ?
*> 150.1.45.4/32    0.0.0.0                  0         32768 ?
r> 150.1.45.5/32    150.1.45.4               0             0 200 ?
*> 150.1.56.0/24    0.0.0.0                  0         32768 ?

This problem has to do with the default redistribution behavior of OSPF. If you issue a ‘redistribute ospf 1′ on R4 and R5 only the INTERNAL routes are getting advertised. If you want the external routes getting redistributed as well you have to change the command to ‘redistribute ospf 1 match internal external’.

After we change this on both R4 and R5. The 172.16.x.1 prefixes are getting redistributed and now it’s known on R1 and R2:

R1:

R1#sh ip route 172.16.2.1
Routing entry for 172.16.2.1/32
Known via "ospf 1", distance 110, metric 1
Tag 100, type extern 2, forward metric 11
Last update from 150.1.13.3 on FastEthernet1/0, 00:00:15 ago
Routing Descriptor Blocks:
* 150.1.13.3, from 150.1.4.4, 00:00:15 ago, via FastEthernet1/0
Route metric is 1, traffic share count is 1
Route tag 100

And the subnets can reach each other through an end-to-end LSP (label switched path):

R1:

R1#ping 172.16.2.1 sour lo1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.2.1, timeout is 2 seconds:
Packet sent with a source address of 172.16.1.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 60/60/64 ms

R2:

R2#ping 172.16.1.1 sour lo1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.1.1, timeout is 2 seconds:
Packet sent with a source address of 172.16.2.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 60/61/64 ms

R2#trace 172.16.1.1
Type escape sequence to abort.
Tracing the route to 172.16.1.1
1 150.1.26.6 [MPLS: Label 23 Exp 0] 96 msec 96 msec 96 msec
2 150.1.56.5 [MPLS: Label 25 Exp 0] 144 msec 100 msec 100 msec
3 150.1.45.4 [MPLS: Label 20 Exp 0] 64 msec 64 msec 68 msec
4 150.1.34.3 [MPLS: Label 17 Exp 0] 64 msec 64 msec 64 msec
5 150.1.13.1 36 msec *  32 msec

It seems like nothing has changed (ping still functions as normal), but in fact we changed the entire way of getting packets through a router! It’s not voodoo, just MPLS. When we setup a continuous ping on R1 to R2 and issue a ‘debug ip packet detail’ we see only management traffic for OSPF and LDP:

R3

R3#debug ip pack det
IP packet debugging is on (detailed)

Aug 21 11:14:49.952: IP: s=150.1.34.4 (Ethernet0/1), d=224.0.0.5, len 80, rcvd 0, proto=89
Aug 21 11:14:50.428: IP: s=150.1.34.3 (local), d=224.0.0.5 (Ethernet0/1), len 80, sending broad/multicast, proto=89
Aug 21 11:14:50.508: IP: s=150.1.13.3 (local), d=224.0.0.2 (Ethernet0/0), len 62, sending broad/multicast
Aug 21 11:14:50.508:     UDP src=646, dst=646
Aug 21 11:14:50.740: IP: s=150.1.13.1 (Ethernet0/0), d=224.0.0.2, len 62, rcvd 0
Aug 21 11:14:50.740:     UDP src=646, dst=646
Aug 21 11:14:50.740: IP: s=150.1.34.4 (Ethernet0/1), d=224.0.0.2, len 62, rcvd 0
Aug 21 11:14:50.744:     UDP src=646, dst=646

But if you issue a ‘debug mpls packets’, you do see a lot of traffic going through:

R3:

R3#debug mpls pack
MPLS packet debugging is on

Aug 21 12:03:54.384: MPLS: Et0/0: recvd: CoS=0, TTL=255, Label(s)=18
Aug 21 12:03:54.384: MPLS: Et0/1: xmit: CoS=0, TTL=254, Label(s)=21
Aug 21 12:03:54.444: MPLS: Et0/1: recvd: CoS=0, TTL=252, Label(s)=17
Aug 21 12:03:54.448: MPLS: Et0/0: xmit: (no label)
Aug 21 12:03:54.448: MPLS: Et0/0: recvd: CoS=0, TTL=255, Label(s)=18
Aug 21 12:03:54.448: MPLS: Et0/1: xmit: CoS=0, TTL=254, Label(s)=21
Aug 21 12:03:54.508: MPLS: Et0/1: recvd: CoS=0, TTL=252, Label(s)=17
Aug 21 12:03:54.512: MPLS: Et0/0: xmit: (no label)

You see the ICMP echo coming in on e0/0 with label 18, swapped with label 21, TTL decreased by 1 and transmitted on e0/1. If we check this in the ‘show mpls forwarding-table’ we see this is the correct behavior:

R3:

R3#sh mpls for | in ^18
18     21          172.16.2.1/32     1212568    Et0/1      150.1.34.4

Than the ICMP echo-reply it coming in with label 17 on e0/1, the label is stripped (PHP) and transmitted on e0/0.  Which also complies with the MPLS forwarding table.

R3:

R3#sh mpls for | in ^17
17     Pop tag     172.16.1.1/32     1173228    Et0/0      150.1.13.1

This is the key thing what we have trying to accomplish. No IP forwarding is done, just MPLS, not a single routing lookup is done in this process.

When the prefix and label are learned through BGP, it’s not shown in the ‘show mpls forwarding-table’. If you want to troubleshoot it, then the ‘show ip cef’ will help you with that, remind you that CEF is  mandatory for MPLS to work. If we want to know which label is imposed on R4 to send the packet to R5:

R4:

R4#sh ip cef 172.16.2.1
172.16.2.1/32, version 53, epoch 0, cached adjacency to Serial0/0.1
0 packets, 0 bytes
tag information set
local tag: 21
fast tag rewrite with Se0/0.1, point2point, tags imposed: {20}
via 150.1.45.5, 0 dependencies, recursive
next hop 150.1.45.5, Serial0/0.1 via 150.1.45.5/32
valid cached adjacency
tag rewrite with Se0/0.1, point2point, tags imposed: {20}

The ‘local tag’ is the tag that is advertised and packets destined for this prefix should have label 21 set on the packet. That way CEF knows that it matches this entry and it should swap it with label 20 (in the ‘tag imposed’ section) and send out on Serial0/0.1.

I hope I could give you an introduction to MPLS and show the basics of the technology and the basic ideas behind it. The next article will be based on the same topology, but then without any LDP and solely based on Traffic Engineering tunnels.

–
Thanks!!!
Rick Mur
CCIE2 #21946 (R&S / Service Provider)
Sr. Support Engineer - IPexpert, Inc.

Print Friendly