Recap of the previous lab
In lab 2, we have created a Centralized Policy named CENTRALIZED-CONTROL-POLICY-V2 using the vManage GUI. In that policy, we have created a Topology definition that matches the Transport Locators of the hub site ( based on the site-id of the hub) and accepts them, all other TLOCs are rejected by the default action at the end which is Reject. In another sequence, we matched all omp routes and accepted them. After that, we applied this policy in an outbound direction to the spoke sites. Note that the out direction is from the perspective of the OMP updates of the vSmart controller to the WAN edge devices.
So what are the results of this Centralized policy:
- Due to sequence 1 of the topology definition - the vEdges at the spoke sites receive only the transport locators (TLOCs) of the hub site. Therefore, they can only make overlay tunnels to the hub's vEdges which basically creates a hub-and-spoke overlay fabric.
- Due to sequence 11 - the vEdges at the spoke sites receive all omp routes from all sites. This means that each WAN edge router at the spoke sites receives omp routes from all other spoke vEdges. These omp routes have next-hop TLOC attributes pointing to the originating spoke vEdges. However, spoke vEdges do not know the transport locators of other spokes because spokes' TLOCs have been filtered out in sequence 1. That's is why the omp routes from other spokes have a status Invalid and Unresolved and are not installed in the routing table.
To enable spoke-to-spoke reachability, we have injected a default route from the hub's vEdges as a workaround to the issue created by sequence 11 of the centralized policy.
In this lab lesson, we are going to solve the issue of the Invalid routes by manipulating the omp route advertisements towards the spoke routers. Each OMP route that is advertised from the vSmart controllers has the following attributes.
Attributes: originator 126.96.36.199 type installed tloc 188.8.131.52, lte, ipsec ultimate-tloc not set domain-id not set overlay-id 1 site-id 60 preference not set tag not set origin-proto connected origin-metric 0 as-path not set unknown-attr-len not set
Typically, each attribute could be used as a match criteria and each one could also be modified with a set action. In this lab, we are going to manipulate the TLOC attribute of the routes that are advertised to the spoke vEdges. Because the spokes only know about the TLOCs of the hub site, we are going to change the tloc attributes of the omp routes to point to the available hub's TLOCs.
Modifying OMP routes with TLOC lists
Before we begin with this lab, let's first have a look at our topology diagram that we use for this Cisco SD-WAN Deep Dive series.
If we log into vManage and enter the Policy Wizard in Configuration > Policies we can see the Centralized Policy we have created in the last lesson - "CENTRALIZED-CONTROL-POLICY-V2" is currently active. As we learned in the previous lab lesson, centralized policies could not be edited while active. Therefore, we are going to create create a new version of the policy reusing some parts of the current one and when we are done, we are going to push the new version to vSmart.
For this lab, we are going to need a new list that matches the transport locators of the hub's vEdge routers. Let's create a new list called HUB-TLOCS by going to Custom Options > Lists.
In there, we select TLOC and enter the name of the list. After that, we specify the four transport locators that we have in the hub site. Note that a TLOC consists of three tuples of information - (TLOC IP, Color, Encap). The TLOC IP is typically the system IP address of the vEdge router. Based on personal experience, I can say that it is a common mistake to use the IP address of the interface connected to the transport network. I had personally done this many times before. For example, if you look at figure 3, it is a common mistake to use the TLOC (10.50.1.1, mpls, ipsec) for the transport locator of vEdge-1 that identifies the connection to the MPLS cloud. However, the correct TLOC tuple is (184.108.40.206., mpls, ipsec).
Once the HUB-TLOCS list is created, note that the Reference Count is 0, because it is not used anywhere at the moment.
Now we need to create the Topology definition that will change the next-hop TLOC attributes of the OMP routes advertisements to the spokes. Let's reuse the topology policy that is currently in use. To do this, we copy the existing one using a different name and we modify it as per the requirements.
Go to Custom Options > Topology.
In there we must see that policy FILTER-TLOCS-AND-ADV-ROUTES that we have created in the previous lab lesson. Note that the Reference Count is 1 because it is currently in use. To reuse it, we make a copy of it with a different name.
For this lab lesson, we are going to use the name TOPOLOGY-V3.
Now you can see that the new topology policy has a Reference Count of 0 meaning that it is not used anywhere. Therefore, we can safely edit it without touching anything in production. From the additional options, we select Edit.
In there on the left side, we can see the two sequence definitions that we have created in the previous lab. If we select the Route sequence type, we can see that it matches the ALL-ROUTES prefix-list and Accepts it. However, as we said earlier, we'd like to not only accept the routes but also to modify the next-hop TLOC attributes. That is why we are going to edit this match-action sequence.
There we go to the Actions tab > TLOC and select the HUB-TLOCS list that we defined earlier. Then we save the policy.
In the end, we should have a match condition that matches the pre-defined prefix-list ALL-ROUTES and the applied action should be the TLOC list HUB-TLOCS.
At this point, the new Topology definition is done but obviously, it is still not used anywhere.
At this point, all required parts for our new Centralized Control Policy are made so we can go ahead and define the policy through the wizard. We go to Policies > Add Policy. In there, we skip the first page "Create Groups of Interest" because we have already defined all required lists. On the next page "Configure Topology and VPN Membership" we go to Add Topology > Import Existing Topology because we have already defined a Topology policy (named TOPOLOGY-V3).
In the pop-up window, we go to Custom Control (Route and TLOC) and select the policy have defined earlier - TOPOLOGY-V3 from the drop-down menu.
Then we skip the next page "Configure Traffic Rules" and end up on the "Apply Policies to Sites and VPNs" page. In there, we specify a name and description of the policy. As we said in the previous lessons, it is a general rule of thumb to use all capital letters for the name and always include some kind of a version number.
After that apply the policy in an outbound direction to the spokes by selecting the site-list SPOKES.
Now you can see that we have a new Centralized Control policy created that has a V3 at the end of the name. Note that it is not activated though. Still, the policy that we have created in the previous lesson is active. To activate the new one we go to the options and click Activate.
If the policy is pushed successfully to all vSmart controllers (in our case it is only one), we must see a status - Success.
Verifying the results of the policy
Before we edited and applied a new centralized control policy, the spoke vEdges had only had a default route pointing to the hub's tlocs.
vEdge-4# show ip routes vpn 1 VPN PREFIX PROTOCOL TLOC IP COLOR ENCAP STATUS ---------------------------------------------------------------------------------- 1 0.0.0.0/0 omp 220.127.116.11 mpls ipsec F,S 1 0.0.0.0/0 omp 18.104.22.168 public-internet ipsec F,S 1 0.0.0.0/0 omp 22.214.171.124 mpls ipsec F,S 1 0.0.0.0/0 omp 126.96.36.199 public-internet ipsec F,S
However, we explicitly changed the next-hop tlocs of all OMP routes that are advertised to the spokes with the set tloc-list [list-name] action highlighted below.
sequence 11 match route prefix-list ALL-ROUTES ! action accept set tloc-list HUB-TLOCS ! !
Now if we check the routing table of any of the spokes, we are going to see that the WAN edge router has successfully installed all OMP routes into its routing table.
vEdge-4# show ip routes vpn 1 VPN PREFIX PROTOCOL TLOC IP COLOR ENCAP STATUS ---------------------------------------------------------------------------------- 1 0.0.0.0/0 omp 188.8.131.52 mpls ipsec F,S 1 0.0.0.0/0 omp 184.108.40.206 public-internet ipsec F,S 1 0.0.0.0/0 omp 220.127.116.11 mpls ipsec F,S 1 0.0.0.0/0 omp 18.104.22.168 public-internet ipsec F,S 1 172.16.50.0/24 omp 22.214.171.124 mpls ipsec F,S 1 172.16.50.0/24 omp 126.96.36.199 public-internet ipsec F,S 1 172.16.50.0/24 omp 188.8.131.52 mpls ipsec F,S 1 172.16.50.0/24 omp 184.108.40.206 public-internet ipsec F,S 1 172.16.60.0/24 omp 220.127.116.11 mpls ipsec F,S 1 172.16.60.0/24 omp 18.104.22.168 public-internet ipsec F,S 1 172.16.60.0/24 omp 22.214.171.124 mpls ipsec F,S 1 172.16.60.0/24 omp 126.96.36.199 public-internet ipsec F,S 1 172.16.80.0/24 omp 188.8.131.52 mpls ipsec F,S 1 172.16.80.0/24 omp 184.108.40.206 public-internet ipsec F,S 1 172.16.80.0/24 omp 220.127.116.11 mpls ipsec F,S 1 172.16.80.0/24 omp 18.104.22.168 public-internet ipsec F,S 1 172.16.90.0/24 omp 22.214.171.124 mpls ipsec F,S 1 172.16.90.0/24 omp 126.96.36.199 public-internet ipsec F,S 1 172.16.90.0/24 omp 188.8.131.52 mpls ipsec F,S 1 172.16.90.0/24 omp 184.108.40.206 public-internet ipsec F,S
And the ultimate test would be to do a traceroute from one spoke site toward another spoke. As you can see, the traffic is now going through the hub.
vEdge-4# traceroute vpn 1 172.16.80.1 Traceroute 172.16.80.1 in VPN 1 traceroute to 172.16.80.1 (172.16.80.1), 30 hops max, 60 byte packets 1 172.16.50.3 (172.16.50.3) 19.753 ms 19.626 ms 19.561 ms 2 172.16.80.1 (172.16.80.1) 77.535 ms 77.527 ms 77.513 ms
The full CLI representation of the applied centralized control policy is shown bellow:
policy control-policy TOPOLOGY-V3 sequence 1 match tloc site-list Hub ! action accept ! ! sequence 11 match route prefix-list ALL-ROUTES ! action accept set tloc-list HUB-TLOCS ! ! ! default-action reject ! lists prefix-list ALL-ROUTES ip-prefix 0.0.0.0/0 le 32 ! site-list Hub site-id 50 ! site-list Spokes site-id 60 site-id 70 site-id 80 site-id 90 ! tloc-list HUB-TLOCS tloc 220.127.116.11 color mpls encap ipsec tloc 18.104.22.168 color public-internet encap ipsec tloc 22.214.171.124 color mpls encap ipsec tloc 126.96.36.199 color public-internet encap ipsec ! ! ! apply-policy site-list Spokes control-policy TOPOLOGY-V3 out ! !