In real-world deployments, there will always be cases where the WAN edge routers cannot be directly connected to each available WAN transport as is shown in figure 1. In the example on the left, only one WAN edge device is connected to a single transport. This significantly reduces the overall availability and creates inefficiencies in the overlay fabric. TLOC-extension feature is designed to overcome this problem by extending the WAN transports to both SD-WAN routers without requiring direct attachment to both service provider clouds.

WAN Edges with a Single WAN Trasnport
Figure 1. WAN Edges with a Single WAN Trasnport

Of course, the SD-WAN routers can connect to each WAN through front-facing switches as shown on the right but this is not a recommended approach because it adds additional costs and results in having another device to operate.

What is TLOC Extension?

TLOC extension is a feature that allows a WAN Edge router to communicate over the WAN transport connected to the adjacent WAN Edge router through a TLOC-extension interface. In the example shown in figure 2, vEdge-1 is directly connected to the Internet and also uses the TLOC extension feature to connect to the MPLS transport via vEdge-2. In the end, vEdge-1 has an overlay tunnel built over both WAN clouds even though it is directly attached only to one of the WAN transports. On the opposite side, vEdge-2 is attached to the MPLS cloud and uses the TLOC extension feature to connect to the Internet via WAN Edge 1.

 

TLOC Extension with Directly Connected vEdges
Figure 2. TLOC Extension with Directly Connected vEdges

The feature is set up in a per-interface manner and provides transparent connectivity from one interface (called a TLOC extension interface) to a particular WAN transport. If we take vEdge-1 for example, it is unaware that the blue tunnel to the MPLS cloud passes through another WAN edge device because vEdge-2 extends their directly connected interface transparently to the MPLS transport.

TLOC Extension Types

Cisco SD-WAN allows for multiple ways of implementing TLOC extensions on WAN edge routers. The most typical and straightforward way of extending the transports is by using directly connected interfaces between the vEdges. However, depending on the local site design, the SD-WAN routers can be connected through Layer 2 switches as is illustrated in figure 3. L2 TLOC-extensions describe extensions between vEdges that are Layer2-adjacent are situated in the same broadcast domain/the same subnet.

TLOC Extension via L2
Figure 3. TLOC Extension via L2

The SD-WAN routers can also be connected at Layer 3 via any sort of IP routing. L3 TLOC-extensions describe extensions between two vEdges that are separated by an L3 device and are situated in different IP subnets. L3 TLOC extensions are implemented using GRE tunnels.

TLOC Extension via L3
Figure 4. TLOC Extension via L3

I'd also like to point out that TLOC-extensions can be configured using physical interfaces such as Ge0/0, Eth0, etc., and also using L3 subinterfaces.

Notable Limitations

At this point, it is a good time to point out some notable limitation of this feature:

  • As you might already know, TLOCs are only supported on routing interfaces. Well, the same applies to TLOC-extension interfaces as well. They are only supported on L3 routed interfaces. Switchports and SVIs cannot be configured as WAN/Overlay ports and can only be used within the service VPNs.
  • LTE can not be used as a TLOC extension interface between WAN Edge routers.
  • L3 TLOC-extension isn't supported on Viptela vEdge routers but only on SD-WAN devices running IOS-XE.
  • The feature would not work on transport ports that are bound to the loopback tunnels.

Configuring TLOC Extensions

Routing Considerations

A TLOC extension interface is always configured in VPN 0 and has an IP address assigned. Then the WAN interface to which it is bound is specified. For example, the vEdge-1's extension interface is ge0/4 and is bound to the Internet through ge0/0 and vEdge-2's extension interface is ge0/5 and is bound to the MPLS cloud through ge0/1. Then static default routes are configured in VPN0 on each WAN Edge router, pointing to the adjacent vEdge's IP as a next-hop.

However, some IP reachability considerations must be done in order for the overlay tunnels and BFD sessions to come up with remote peers over the TLOC extension interfaces. For example, to reach the MPLS cloud, vEdge-1 must be configured with a default route pointing to Ge0/5 of vEdge-2. Another very important part is the reverse IP reachability. To make sure that packets can be routed back to the TLOC extension interface, WAN Edge 2 must advertise subnet B to the MPLS provider. This can be done using any dynamic routing protocol that the service provider is willing to run. In a typical production deployment, some sort of inbound route policy is applied to deny all incoming dynamic routes from the provider since there is a static route pointing to the MPLS cloud. 

TLOC Extension
Figure 5.TLOC Extension via L3

In the case of WAN edge router 2, to reach the Internet cloud, it must be configured with a default route pointing to vEdge-1’s ge0/4 IP address. In a typical production deployment, subnet-A would be from the RFC1918 address space and NAT must be utilized on WAN edge-1 towards the INET cloud in order to ensure that traffic can be routed back from the Internet to vEdge-2 over the extension interface.

Let's go through a configuration example and see how this feature is implemented.

Extending vEdge-1 to the MPLS

Let's first extend router-1 to the MPLS cloud. For this example, we are going to use the private subnet 10.51.1.0/30 between Ge0/5 interfaces of both routers. This prefix 10.51.1.0/30 must also be advertised to the MPLS provider in order to have IP reachability in the other direction. For this purpose, we are going to utilize BGP.

Configuring vEdge-1
Configuring vEdge-1

It is important to understand that in order to extend vEdge-1 to the MPLS cloud, the actual TLOC extension happens on the adjacent router. From the perspective of router-1, Ge0/5 is connected to the MPLS provider in the same way as it would have been with a direct attachment link. Therefore, there is nothing special in the configuration on the vEdge-1 as you can see below. The interface is configured with IP address, color, and encapsulation type and a default route is specified. That's about it.

!
vpn 0
 !
 interface ge0/5
  ip address 10.51.1.1/30
  tunnel-interface
   encapsulation ipsec
   color mpls restrict
   allow-service all
  !
  no shutdown
 !
 ip route 0.0.0.0/0 10.51.1.2
 ip route 0.0.0.0/0 50.1.1.2
!

The "magic" happens on the neighboring router-2. In there, we should configure two things. The first one is to tell interface Ge0/5 that it extends the MPLS cloud attached to Ge0/1. The other important thing is to advertise the subnet 10.51.1.0/30 toward the service provider using BGP.

!
vpn 0
 !
 interface ge0/1
  ip address 10.50.2.1/30
  tunnel-interface
   encapsulation ipsec
   color mpls restrict
   allow-service all
  !
  no shutdown
 !
 interface ge0/5
  ip address 10.51.1.2/30
  tloc-extension ge0/1
  no shutdown
 !
 ip route 0.0.0.0/0 10.50.2.2
!
 router
  bgp 64551
   address-family ipv4-unicast
    network 10.51.1.0/30
   !
   neighbor 10.50.2.2
    no shutdown
    remote-as 65051
    address-family ipv4-unicast
    !
   !
  !

Once both steps are configured, we can see that vEdge-1 has a valid TLOC marked with the mpls color that has a BFD session in UP state even though the router is not directly connected to the MPLS cloud.

vEdge-1# show omp tlocs
---------------------------------------------------
tloc entries for 50.50.50.50
                 mpls
                 ipsec
---------------------------------------------------
            RECEIVED FROM:                   
peer            0.0.0.0
status          C,Red,R
loss-reason     not set
lost-to-peer    not set
lost-to-path-id not set
    Attributes:
     attribute-type    installed
     encap-key         not set
     encap-proto       0
     encap-spi         256
     encap-auth        sha1-hmac,ah-sha1-hmac
     encap-encrypt     aes256
     public-ip         10.51.1.1
     public-port       12406
     private-ip        10.51.1.1
     private-port      12406
     public-ip         ::
     public-port       0
     private-ip        ::
     private-port      0
     bfd-status        up
     domain-id         not set
     site-id           50
     overlay-id        not set
     preference        0
     tag               not set
     stale             not set
     weight            1
     version           2
    gen-id             0x80000019
     carrier           default
     restrict          1
     groups            [ 0 ]
     border             not set
     unknown-attr-len  not set

 

Extending vEdge-2 to the Internet

Let's now extend WAN edge 2 to the Internet provider. The configuration is simpler than the MPLS one because we do not need to advertise the interconnecting subnet 192.168.51.0/30 to the Internet provider, but only to NAT it when leaving the transport interface toward the provider.

Configuring vEdge-2
Configuring vEdge-2

All required configuration is shown in the diagram above. Once the config is applied, we can see that vEdge-2 has a valid TLOC marked with the public-internet color. Note that in the TLOC route, we can see the private-public IP address pair.

vEdge-2# show omp tlocs 
---------------------------------------------------
tloc entries for 50.50.50.51
                 public-internet
                 ipsec
---------------------------------------------------
            RECEIVED FROM:                   
peer            0.0.0.0
status          C,Red,R
loss-reason     not set
lost-to-peer    not set
lost-to-path-id not set
    Attributes:
     attribute-type    installed
     encap-key         not set
     encap-proto       0
     encap-spi         256
     encap-auth        sha1-hmac,ah-sha1-hmac
     encap-encrypt     aes256
     public-ip         50.1.1.1
     public-port       12366
     private-ip        192.168.51.2
     private-port      12366
     public-ip         ::
     public-port       0
     private-ip        ::
     private-port      0
     bfd-status        up
     domain-id         not set
     site-id           50
     overlay-id        not set
     preference        0
     tag               not set
     stale             not set
     weight            1
     version           2
    gen-id             0x80000014
     carrier           default
     restrict          0
     groups            [ 0 ]
     border             not set
     unknown-attr-len  not set

If we now check whether there is a BFD session over that color, you can see that there is one is UP state sourced from the TLOC extension interface 192.168.51.2 to a public IP address 80.1.1.1.

vEdge-2# show bfd sessions 
                            SOURCE TLOC      REMOTE TLOC                   DST PUBLIC
SYSTEM IP    SITE ID STATE  COLOR            COLOR           SOURCE IP     IP        
-------------------------------------------------------------------------------------
80.80.80.80  80      up     mpls             mpls            10.50.2.1     10.80.1.1 
80.80.80.80  80      up     public-internet  public-internet 192.168.51.2  80.1.1.1