One of the most commonly used redundancy options for the service side VPNs is to provide a resilient default gateway to hosts in a segment using the Virtual Router Redundancy Protocol(VRRP). VRRP is a first-hop redundancy protocol that works by electing a master router that handles a virtual IP address hosts use as an IP gateway. Network engineers are pretty familiar with FHRP protocols such as HSRP, VRRP, and GLBP, so we will not get into details about how the protocol works. However, we have to make one important design consideration when designing service-side redundancy using VRRP in Cisco SD-WAN. Because vEdges never exchange control-plane information directly among themselves, when the master router loses connection to the overlay fabric, the hosts in the segment lose connection to the network domain.

Figure 1 visualizes this problem. When vEdge-1 loses connection to the overlay, it blackholes the service-side traffic because there is no control-plane exchange between vEdge-1 and vEdge-2. Three general design options resolve this issue by initiating a switchover to the backup VRRP router.

Design considerations

TLOC Preference

As we have seen many times through the course, TLOCs connect OMP routes to vEdge routers (each route entry in the VPN routing tables points to a TLOC). The TLOC Preference is an optional parameter that makes a TLOC more preferred in both traffic directions.

Generally, when we have a layer two high-availability design with a first-hop redundancy protocol, we want to steer the returning traffic on the primary node, as shown in orange in figure 2 below. Cisco SD-WAN has introduced a capability that achieves that by increasing the TLOC preference of the primary node's TLOCs, which steers the returning traffic to the primary node.

When the tloc-change-pref command is configured under the VRRP group, the TLOC preference value of the primary increases by 1. In case of a failover event, one of the backup routers is elected as primary and the tloc pref of its TLOCs is increased by 1. At the same time the default TLOC preference of all backup nodes is set back to 0 (or another default value).

Configuring automatic traffic steering to the primary node

Enabling the tloc-change-preference capability is as straightforward as applying one command under the VRRP group, as shown in the output below. Notice that we should enable this feature for all VRRP nodes in the segment. 

vEdge-1(config)# vpn 6
vEdge-1(config- vpn-6)# interface ge0/2
vEdge-1(config-interface-ge0/2)# vrrp 2
vEdge-1(config-vrrp-2)# ipv4 10.1.6.1
vEdge-1(config-vrrp-2)# priority 110
vEdge-1(config-vrrp-2)# tloc-change-pref

Verifying the configuration

In our example, vEdge-1 is a master router on the ge0/2 interface. Both vEdge-1 and 2 have default TLOC preferences (zero) on all local TLOCs. You can see that the tloc preference on the master is increased by 1.

vEdge-1# show vrrp detail

OMP status: up

group-id: 6, track-omp: yes, initialized: yes
  address: 10.1.6.1
  track-prefix-list: -, resolved: -
  state: Primary, down-reason: none, cfg-priority: 110, priority: 110
  adv-timer: 1, primary-down-timer: 3, sock-fd: 17, addr-count: 1
  adv-timer: Enabled (e: 2 v: 10 c: 1)
  primary-down-timer: Disabled (e: -1 v: 30 c: 3)
  virtual-mac: 0x0 0x0 0x5e 0x0 0x1 0x6
  TLOC Change Preference: Configured
  TLOC Change Preference value: 1
  TLOC Real Preference value: 1
  Group current adaptive priority: 0
  Total Tracking object : 0 (head: (nil))
  Group Address: 0x7fc80d424200

If we check the routing on any of the remote vEdges, we will see that the OMP routes via vEdge-1 (1.1.1.1) are more preferred than the ones via vEdge-2 (1.1.1.2). Therefore, the returning traffic goes back to the primary node as shown in figure 3.

vEdge-5# show omp routes vpn 6 10.1.6.0/24 | t

                 PATH                      ATTRIBUTE                                                       
FROM PEER        ID     LABEL    STATUS    TYPE       TLOC IP          COLOR            ENCAP  PREFERENCE  
-----------------------------------------------------------------------------------------------------------
1.1.1.30         31     1015     R         installed  1.1.1.2          mpls             ipsec  -           
1.1.1.30         32     1015     R         installed  1.1.1.2          biz-internet     ipsec  -           
1.1.1.30         33     1011     C,I,R     installed  1.1.1.1          mpls             ipsec  -           
1.1.1.30         34     1011     C,I,R     installed  1.1.1.1          biz-internet     ipsec  -           

Let's now initiate a failover to vEdge-2 and see what will change. 

vEdge-2# conf t
Entering configuration mode terminal
vEdge-2(config)# vpn 6 interface ge0/2 vrrp 6
vEdge-2(config-vrrp-6)# priority 120
vEdge-2(config-vrrp-6)# commit 
Commit complete.

Now let's verify that vEdge-2 has become the primary router.

vEdge-2# show vrrp detail 

OMP status: up

group-id: 6, track-omp: no, initialized: yes
  address: 10.1.6.1
  track-prefix-list: -, resolved: -
  state: Primary, down-reason: none, cfg-priority: 120, priority: 120
  adv-timer: 1, primary-down-timer: 3, sock-fd: 17, addr-count: 1
  adv-timer: Enabled (e: 8 v: 10 c: 1)
  primary-down-timer: Disabled (e: -1 v: 30 c: 3)
  virtual-mac: 0x0 0x0 0x5e 0x0 0x1 0x6
  TLOC Change Preference: Configured
  TLOC Change Preference value: 1
  TLOC Real Preference value: 1
  Group current adaptive priority: 0
  Total Tracking object : 0 (head: (nil))
  Group Address: 0x7f031a424600

Okay, now, if we check the routing on any of the remote vEdges, we will see that now the routes via vEdge-2 (1.1.1.2) are more preferred for the VRRP subnet.

vEdge-5# show omp routes vpn 6 10.1.6.0/24 | t

                 PATH                      ATTRIBUTE                                                       
FROM PEER        ID     LABEL    STATUS    TYPE       TLOC IP          COLOR            ENCAP  PREFERENCE  
-----------------------------------------------------------------------------------------------------------
1.1.1.30         31     1015     C,I,R     installed  1.1.1.2          mpls             ipsec  -           
1.1.1.30         32     1015     C,I,R     installed  1.1.1.2          biz-internet     ipsec  -           
1.1.1.30         33     1011     R         installed  1.1.1.1          mpls             ipsec  -           
1.1.1.30         34     1011     R         installed  1.1.1.1          biz-internet     ipsec  -          

KEY TOPIC: One essential thing you need to understand is that - the TLOC preference does not affect vSmart's path selection process and that the TLOC preference value isn't bound to any specific VPN. The TLOC preference value on a TLOC affects the OMP best-path algorithm for all VPNs on all vEdges with routes pointing to that particular TLOC.

We can verify that when vEdge-2 becomes a primary router for the VPN6 10.1.6.0/24 segment, it becomes a more preferred router for all VPNs, as you can see in the output below. 

vEdge-5# show omp routes | t                

                                         ATTRIBUTE                                  
VPN  PREFIX         FROM PEER  STATUS    TYPE       TLOC IP     COLOR          ENCAP
------------------------------------------------------------------------------------
4    10.1.4.0/24    1.1.1.30   C,I,R     installed  1.1.1.2     mpls           ipsec
                    1.1.1.30   C,I,R     installed  1.1.1.2     biz-internet   ipsec
                    1.1.1.30   R         installed  1.1.1.1     mpls           ipsec
                    1.1.1.30   R         installed  1.1.1.1     biz-internet   ipsec
5    10.1.5.0/24    1.1.1.30   C,I,R     installed  1.1.1.2     mpls           ipsec
                    1.1.1.30   C,I,R     installed  1.1.1.2     biz-internet   ipsec
                    1.1.1.30   R         installed  1.1.1.1     mpls           ipsec
                    1.1.1.30   R         installed  1.1.1.1     biz-internet   ipsec
6    10.1.6.0/24    1.1.1.30   C,I,R     installed  1.1.1.2     mpls           ipsec
                    1.1.1.30   C,I,R     installed  1.1.1.2     biz-internet   ipsec
                    1.1.1.30   R         installed  1.1.1.1     mpls           ipsec
                    1.1.1.30   R         installed  1.1.1.1     biz-internet   ipsec

Always keep in mind that changing the TLOC preference value affects all OMP routes in all VPNs! Learn more about TLOC Preference in our lesson here. 

VRRP Tracking Options

We have learned how to steer the returning traffic to the primary node when a failover event occurs in the VRRP segment. Now we will explore the available Cisco SD-WAN tracking options that initiate a failover when the primary VRRP node loses connection to the overlay.

Option #1: Tracking OMP

The first option is to track the OMP sessions status on the master VRRP router. If all OMP sessions go down on the master router, VRRP elects a new master router from the pool of the available backup VRRP routers on that segment, even if they have lower priority. However, this option is the slowest because the OMP state changes to down only when the OMP hold-timer expires (60 seconds by default). This option has another significant drawback - from the moment the router loses connectivity to the overlay until the time when the hold timer expires, the router drops all site-local traffic destined for the overlay. When at least one OMP session recovers and becomes up, the router claims itself as master again even before receiving any OMP routes from vSmart. The traffic is also dropped until the OMP routes are learned and installed in the VPN routing tables.