MPLS Traffic Engineering

  1. TE
    1. IOS CLI
  2. TE IGP
    1. IS-IS
    2. OSPF
      1. IOS CLI
    3. IGP flooding
      1. IOS CLI
    4. IGP cost
      1. IOS CLI
  3. TE tunnel attributes
    1. IOS CLI
  4. RSVP TE
    1. IOS CLI
  5. Fast reroute (FRR)
    1. IOS CLI
    2. IOS CLI
    3. IOS CLI
  6. Forwarding onto MPLS TE
    1. IOS CLI
  7. P-router = tail/head
  8. Autobandwidth
    1. IOS CLI
  9. Auto-tunnel
    1. IOS CLI
      1. Mesh
  10. Inter-area TE
  11. Path protection
    1. IOS CLI
  12. AToM & TE
    1. IOS CLI
  13. P2MP TE
    1. IOS CLI

TE

  • dynamically takes into account changes of path attributes (MPLS VPN SLA)
  • source-based routing
  • link resources:
    • max 32 resources per link
    • flood: regular and on changes
  • RRR – routing with resource reservation
  • PCALC:
    1. if paths are equal, select the one with the largest minimum BW (links along the path, ~EIGRP)
    2. then select path through minimal number of hops
    3. IOS specific choice
  • CSPF – constrained SPF, PCALC; always selects a single path
  • link TE attributes:
    • attribute flags, 32 bits, user flags
    • SRLG: shared risk link group, fate-sharing links
    • max reservable subpool BW, used by DiffServ-aware tunnels
    • TE metric: equal to IGP cost by default
    • max BW: equal to value configured for the link
    • max reservable BW: available to TE
    • unreserved BW: (max reservable BW) – (reserved BW), BW that is available to other tunnels
    • administrative group

IOS CLI

; enable TE globally
(config)# mpls traffic-eng tunnels
; enable TE on the link
(config-if)# mpls traffic-eng tunnels

; max reservable BW, mandatory for BW-based calculations
(config-if)# ip rsvp bandwidth <kbps>

; 32 bits, flag bitmask
(config-if)# mpls traffic-eng attribute-flags <value>

; TE metric
(config-if)# mpls traffic-eng administrative-weight <num>

TE IGP

IS-IS

  • RFC 5305, extended IS reachability TLV sub-TLVs
  • sub-TLVs:
    • 3 = administrative group
    • 6 = IPv4 interface address
    • 8 = IPv4 neighbour address
    • 9 = max link BW
    • 10 = max reservable link BW
    • 11 = unreserved BW
    • 18 = TE metric

OSPF

  • expansion, RFC 2370:
    • opaque LSA:
      • LSA 9:
        • link-scope
        • not used by TE
      • LSA 10:
        • area-scope
        • carries TLVs
      • LSA 11:
        • AS-scope (~LSA5)
        • not passed to stub areas
    • O-bit:
      • in Hello, DBD, LSA
      • can receive/send opaque LSA
    • LSID: Opaque type (0x01 for TE) + Opaque ID (3 bytes)
  • TLV:
    • Router Address TLV: TE router ID (~ OSPF RID)
    • Link TLV:
      • link type
      • link ID
      • local interface IP
      • remote interface IP
      • TE metric
      • maximum BW: bytes/sec
      • max reservable BW: bytes/sec
      • unreserved BW: 32 bytes (4 bytes per priority 0-7), bytes/sec
      • administrative group

IOS CLI

; required for LSA generation
(config-router)# mpls traffic-eng router-id <intf>

(config-router)# mpls traffic-eng area <num>
; metric for CSPF, TE by default
(config)# mpls traffic-eng path-selection metric [igp | te]
; on the tunnel
(config-if)# tunnel mpls traffic-eng path-selection metric [igp | te]
; bandwidth in kilobytes/sec
# show ip ospf database opaque-area

IGP flooding

  • triggers:
    • link status change
    • configuration change
    • periodic flood (3 mins by default)
    • change of reserved BW that exceeds threshold
    • after unsuccessful attempt to establish a tunnel (cause probably the info is out of date)
  • small changes of reserved BW do not trigger flood (do not exceed threshold)
  • thresholds for reserved BW are denser to full link utilization (15%, 30$, … 98%, 99%, 100%)

IOS CLI

; TE LSA flood period, 3 mins default
(config)# traffic-eng link-management timers periodic-flooding <sec>
; separate thresholds for inscreasing/decreasing reserved BW
(config-if)# mpls traffic-eng flooding thresholds <up | down> <num> ...

IGP cost

  • autoroute announce: IGP cost from head to tail – minimal! (shortest unconstrained path info)
  • if a better IGP path found – rebuild if possible (reoptimization)
  • if prefix is connected to tail, only TE tunnel is used even if it’s worse than IP path
  • if load-balancing is done via TE tunnels to the same LSR (not necessarily to the same IP), then unequal load-balancing is proportional to reserved BW (costs are equal)

IOS CLI

; sets metric for tunnel, caution
(config-if)# tunnel mpls traffic-eng autoroute metric <absolute | relative> <cost>

TE tunnel attributes

  • attributes:
    1. tunnel destination: MPLS TE RID of LHR
    2. desired BW
    3. affinity: requirement for attribute-flags
    4. setup priority:
      • preempt priority
      • ≥ hold
      • lower = better
    5. hold priority:
      • resists setup priority
      • lower = better
    6. reoptimisation: switching over to a better path
    7. path-options: explicit (hop: TE RID or IP), dynamic (CSPF)
  • reoptimization trigger:
    1. periodic
    2. event-driven, disabled by default
    3. manual
  • avaialble BW is decreased for priotities n ≥ m when m reserves BW

IOS CLI

; 60 mins by default
(config)# mpls traffic-eng reoptimize timers frequency <sec>

; enable event-driven reoptimization
(config)# mpls traffic-eng reoptimize events link-up

(config)# ip explicit-path [identifier <num> | name <NAME>]

; strict by default
(cfg-ip-expl-path)# next-address [strict | loose] <IP>

(cfg-ip-expl-path)# exclude-address <IP>
; val = 0x0, mask = 0xFFFF by default
(config-if)# tunnel mpls traffic-eng affinity <value> [mask <mask>] 

; global – common TE tunnels, sub-pool – DiffServ-aware
(config-if)# tunnel mpls traffic-eng bandwidth [sub-pool | global] <BW>

(config-if)# tunnel mpls traffic-eng priority <setup> [<hold>]

; lower priority = better, lockdown = disable reoptimization
(config-if)# tunnel mpls traffic-eng path-option <priority> [dynamic | explicit] [identifier <num> | name <NAME>] [lockdown]
# mpls traffic-eng reoptimize [tunnel <intf>]
# mpls traffic-eng tunnels tunnel <intf> [protection]

RSVP TE

  • resource reservation protocol
  • sets up tunnel, transports labels (DoD), does not require LDP
  • objects:
    1. ERO:
      • explicit route object
      • list of IP addresses of LSR along LSP
    2. RRO:
      • record route object
      • optional
      • real list of nodes and labels
    3. Label Request:
      • tunnel establishment request
    4. Label:
      • transports label for tunnel between peer LSRs
    5. Sender Tspec:
      • bytes/s
      • BW requirement for the tunnel
    6. Session:
      • IP egress LSR, tunnel ID, extended tunnel ID
    7. Session Attributes:
      • setup/hold priorities, flags, affinity
  • messages:
    1. Path:
      • head → end
      • Label Request object, ERO, RRO
      • temporarily reserves resources
    2. Resv:
      • end → head
      • along Path packetroute
      • Label object, RRO
      • confirms upstream reservation
    3. PathTear:
      • head → end
      • deletes tunnel
    4. ResvTear:
      • end → head
      • response to PathTear
    5. PathErr
      • → end
      • error (e.g. link towards next-hop not accessible)
      • LSP recalc required, tear down old path
    6. ResvErr:
      • → end
    7. ResvConfirm
  • tunnel is established only after receiveing Resv
  • LSR deletes its address from ERO on receiving Path, sends further according to ERO
  • local label is generated only after Resv with outgoing label is received (ordered control)
  • IOS considers explicit-null as implicit-null
  • Shared Explicit Style (SE):
    1. old TE tunnel is torn down only after establishing a new tunnel
    2. no double booking on links by new and old tunnels (tunnel rerouting, BW requirement change)
  • unused labels are deleted (conservative label retention)
  • TE is not routed through TE

IOS CLI

; hidden command, disabled by default, makes IOS consider explicit-null as explicit-null, required for QoS
(config)# mpls traffic-eng signalling interpret explicit-null verbatim

; send implicit-null in Resv in lieu of explicit-null
(config)# mpls traffic-eng signalling advertise implicit-null [<ACL>]
(config-if)# ip unnumbered <intf>
(config-if)# tunnel mode mpls traffic-eng

; enables RRO in Resv
(config-if)# tunnel mpls traffic-eng record-route
# debug ip rsvp dump-messages

; setup and contents of tunnels
# debug mpls traffic-eng link-management ...

Fast reroute (FRR)

  • modes:
    • link protection
    • node protection
  • backup tunnel is created in advance, allowing not to lose packets that are already in-flight
  • backup tunnel is not built through links with same SRLG as protected link
  • backup tunnels preference:
    1. NNHOP better than NHOP
    2. tunnel with least available but sufficient BW (unlimited BW is always the worst)
    3. tunnel servicing specific pool (global pool has more proirity over sub-pool)
  • every protected LSP is assigned a single backup tunnel; backup tunnel can be used by multiple protected LSPs; assignment is periodically refreshed (5 mins)
  • RSVP Hello can be used to detect outage:
    • active – sends Hello
    • passive – sends Hello Ack
  • reroute trigger:
    • link down
    • RSVP Hello Ack loss

IOS CLI

; force = backup never uses links with the same SRLG
; preferred = backup does not use links with the same SRLG if possible
(config)# mpls traffic-eng auto-tunnel backup srlg exclude [force | preferred]

; LSP-to-backup reassignment period
(config)# mpls traffic-eng fast-reroute timers frequency <sec>

; enable Hellos globally
(config)# ip rsvp signalling hello
; assign link to SRLG, can be in several groups at once
(config-if)# mpls traffic-eng srlg <num>

; 200ms default
(config-if)# ip rsvp signalling hello refresh interval <sec>

; 4 default
(config-if)# ip rsvp signalling hello refresh misses <num>

; on TE tunnel head, disabled by default, request link protection
(config-if)# tunnel mpls traffic-eng fast-reroute [node-protect]

; on physical interface
(config-if)# ip rsvp signalling hello
# show mpls traffic-eng topology brief
# show mpls traffic-eng topology srlg
  • link protection: all TE tunnels through link are protected by one backup
  • NHOP – next-hop
  • PLR – point of local repair, start of backup
  • MP – merge point, end of backup and broken pert of LSP
  • backup always to NHOP, does not tranport regular traffic
  • backup is set up with RSVP (label exchange), NHOP labels are already known
  • if label space is platform-wide, routers do not check the interface that received labeled packet (no RPF-like function)
  • tunnel is temporary:
    • till LSP converges
    • allows not dropping packets in-flight
  • on link down:
    1. PLR sends PathErr with LocalRepair = 1 – head understands that it mustn’t tear down old tunnel before building a new one
    2. IGP signals the broken link
    3. backup is active till PathTear
    4. all Path messages routed via broken link are steered into backup
  • no BW protection: oversubscription on backup → drop extra
  • head sends Path with Session attributes::flags = 0x01 ≡ local protection required

IOS CLI

; backup config, must not have "autoroute announce" or "forwarding adjacency" ≡ does not forward regular traffic
(config-if)# tunnel mpls traffic-eng path-option ...

; on protected link
(config-if)# mpls traffic-eng backup-path <tunnel intf>
; need to remove NHOP from calculations for dynamic tunnel
(cfg-ip-expl-path)# exclude-address <IP>
# show mpls traffic-eng fast-reroute database detail
# show ip rsvp fast-reroute

FRR Node protection

  • NNHOP: next-next-hop
  • always backup towards NNHOP
  • head sends Path with Session Attributes::flags = 0x10 ≡ node protection required
  • NNHOP label is carried in RRO in Resv
  • steers all Path messages for the broken router into backup

IOS CLI

; remove R2 from LSP for backup, IP – link address or RID
(cfg-ip-expl-path)# exclude-address <IP>

; on link towards protected node
(config-if)# mpls traffic-eng backup-path <tunnel intf>

Forwarding onto MPLS TE

  • methods:
    • static routing
    • PBR
    • autoroute announce: prefixes downstream from tail are known via tunnel; tail – nearest
    • forwarding adjacency: IGP considers tunnel as a link
    • direct mapping of AToM on TE
    • CBTS: class-based tunnel selection, mapping EXP onto TE
  • forwarding adjacency has to be enabled on two tunnels: head → tail + tail → head ≡ bidirectional link
  • unidirectional links are not included into SPF (~ stub link)
  • LSDB and RSVP are used to detect bidirectionality
  • CBTS does not allow to load-balance within same EXP ≡ IP precedence

IOS CLI

; ~ SPF modification, tunnel as next-hop for downstream prefixes for which tail is the nearest
(config-if)# tunnel mpls traffic-eng autoroute announce

; holdtime – time before flood after tunnel is down, 0 by default
(config-if)# tunnel mpls traffic-eng forwarding-adjacency holdtime <ms>

; CBTS, default = all EXP that are not explicitly mapped, for IP → MPLS EXP ≡ IP precedence
(config-if)# tunnel mpls traffic-eng exp <default | <num> ...>
# show mpls traffic-eng forwarding-adjacency

P-router = tail/head

  • LDP on all links
    1. tLDP between head and tail – tail sends its label to head
    2. OR enable LDP on TE tunnel (= automatic tLDP session)
  • 3 labels: VPN, LDP, TE

Autobandwidth

  • tracks TE tunnel load and adapts reservation of the tunnel
  • adjust interval: time for track traffic spikes
  • adjust threshold: if exceeded, RSVP rebuilds LSP
  • if traffic exceeded threshold during interval – adjust BW request by TE tunnel
  • overflow/underflow: if peaks exceed threshold significantly, RSVP rebuilds LSP before adjust interval expires
  • vendor-specific
  • can be used to dynamically rebuild tunnel according to traffic demand and priority markings

IOS CLI

; 300s by default, enables auto BW globally
(config)# mpls traffic-eng auto-bw timers [frequency <sec>]
; sec = 24h, N = M = ∞ by default, BW is updated once per second, sec must be lower than global value 
(config-if)# tunnel mpls traffic-eng auto-bw max-bw <M> min-bw <N> frequency <sec>

Auto-tunnel

  • after receiving flag from ingress LSR, all LSR along the LSP calculate Bypass tunnels for FRR
  • autoconfig of tunnels betwen LSR of the same group, based on TE LSDB ≡ within area
  • tunnel numbers: first 100 ­primary, last 100 – backup for primary by default

IOS CLI

; all LSR, build NHOP and NNHOP tunnels
(config)# mpls traffic-eng auto-tunnel backup [nhop-only]

; builds tunnels via tLDP to all next-hops, FRR replacement
(config)# mpls traffic-eng auto-tunnel primary onehop

; enable LDP on autotunnels
(config)# mpls traffic-eng auto-tunnel primary config mpls ip

Mesh

(config)# interface auto-template 1
(config-if)# ip unnumbered <intf>
(config-if)# tunnel mode mpls

; ACL specifies LSP addresses from the same mesh group
(config-if)# tunnel destination access-list <ACL>

(config-if)# tunnel mpls traffic-eng autoroute announce
(config)# mpls traffic-eng auto-tunnel mesh

Inter-area TE

  • no auto-tunnel mesh, path-option dynamic, autoroute, affinity, IGP cost, reoptimization
  • interarea tunnel ≈ intra-area via ABR (hop in ERO)
  • cost TE = cost to ABR
  • hops in ERO from other areas – loose

Path protection

  • building backup LSPs that satisfy the required resources for primary LSP
  • order of usage:
    1. primary LSP
    2. secondary LSP
      • standby: calculated and signalled
      • non-standby: calculated, not signalled (no idle BW, longer switchover)
    3. best effort
  • if LSP goes down, only in-flight packets are lost

IOS CLI

; num – primary option number
(config)# tunnel mpls traffic-eng path-option protect <num> ...

AToM & TE

  • autoroute announce steers all traffic into TE tunnel
  • fallback: if TE tunnel is not available, AToM uses RIB/LIB by default

IOS CLI

(config-pw-class)# prefered-path interface <intf> [disable-fallback]
(config-pw-class)# prefered-path peer <IP> [disable-fallback]

P2MP TE

  • sub-LSPs towards different IPs within single LSP (RSVP session)
  • RFC 4875
  • SSM multicast only because TE tunnel is unidirectional ≡ no PIM adjacency
  • no support for PHP ⇒ explicit-null or label

IOS CLI

(config)# mpls traffic-eng destination list name <LST>
(cfg-te-dest-list)# ip <IP> path-option <n> dynamic | explicit

; permits mLDP to use P2P TE
(config)# mldp path traffic-eng

; on tail, permits receiving mcast from TE
(config)# ip multicast mpls traffic-end

; for RPF, automatic mapping to mLSP
(config)# ip mroute <src> <mask> <headend>
; for autoroute announce: RPF for unicast – TE, mcast – physical, for cases when mcast is transported natively
(config-router)# mpls traffic-eng multicast-intact
(config-if)# tunnel mode mpls traffic-eng point-to-multipoint
(config-if)# tunnel destination list mpls traffic-eng name <LST>

; steers mcast into TE tunnel
(config-if)# ip igmp static-group <MIP> source <IP>
(config-if)# ip pim passive