What is OMP?

Cisco SD-WAN Overlay Management Protocol (OMP) is the control plane protocol that runs between the WAN Edge devices and the Cisco vSmart controllers and also as a full mesh peering between the controllers themselves. Cisco SD-WAN applies the principles of Software-Defined Networking, which separates the control and data plane of the network. That means that control plane information is never exchanged between WAN edge routers. vEdges only send and receive information through the vSmart controllers as shown in figure 1. 

Cisco SD-WAN OMP Operation
Figure 1. Cisco SD-WAN OMP Operation

The Cisco vSmart’s role in the network is very similar to that of a BGP route reflector. The controller takes all routing and topology information received from every WAN edge device, calculates the best paths based on the configured policies, and then re-advertises the results to all other WAN Edge routers.

OMP Peering

OMP is enabled by default on all Cisco SD-WAN edge devices. When vEdges go through the Zero-Touch Provisioning process, they learn about the addresses of all available vSmart controllers and automatically initiate secure connections to them. By default, these connections are authenticated and encrypted via the Datagram Transport Layer Security (DTLS) protocol. Depending on the number of available transports, each vEdge router will try to establish a secure control connection via every TLOC. However, as shown in figure 2, the OMP peering uses the System-IPs, and only one peering session is established between one WAN Edge device and one vSmart controller even if there are multiple DTLS connections to the same controller

Cisco SD-WAN OMP Peering
Figure 2. Cisco SD-WAN OMP Peering

You can see in the following output that the WAN edge device has two DTLS control connections initiated to the vSmart controller with IP address 1.1.0.3.  One connection through the MPLS transport and another one through the Internet.

vEdge-1# show control connections 
                                            PEER                       
PEER    PEER PEER        SITE   PEER        PUB                        
TYPE    PROT SYSTEM IP   ID     PUBLIC IP   PORT  LOCAL COLOR     STATE
-----------------------------------------------------------------------
vsmart  dtls 1.1.0.3     1      1.1.1.70    12346 mpls            up    
vsmart  dtls 1.1.0.3     1      1.1.1.70    12346 public-internet up    
vbond   dtls -           0      1.1.1.60    12346 mpls            up    
vbond   dtls -           0      1.1.1.60    12346 public-internet up    
vmanage dtls 1.1.0.1     1      1.1.1.50    12346 mpls            up    

However, if we check how many omp peering sessions to the controller there are, you can see that there is only one.

vEdge-1# show omp peers
R -> routes received
I -> routes installed
S -> routes sent

                DOMAIN OVERLAY SITE                     
PEER    TYPE    ID     ID      ID    STATE  UPTIME      R/I/S
-------------------------------------------------------------
1.1.0.3 vsmart  1      1       1     up     0:14:25:18  4/1/8

Another important thing to know is that these DTLS control plane tunnels are used by other protocols as well. For example, besides OMP, NETCONF and SNMP will also be transported via these secure connections. By utilizing these encrypted DTLS tunnels, we no longer need to be concerned about the native security of protocols like SNMP, NTP, etc.

Cisco SD-WAN DTLS Connection
Figure 3. Cisco SD-WAN DTLS Connection

In a typical production deployment, there are at least two or three controllers for redundancy purposes. When we have multiple vSmarts, they also establish OMP peering between them in a full-mesh manner as shown in figure 4.

OMP Peering with multiple controllers
Figure 4. OMP Peering with multiple controllers

OMP Routes

As it is visualized in figure 5, vEdge routers advertise three types of routes via the Overlay Management Protocol (OMP) to the vSmart controllers:

●      vRoutes (also called OMP routes or just routes) are prefixes learned from local networks of a WAN edge router. These can be locally connected prefixes or ones learned from a dynamic routing protocol such as OSPF and BGP. Once the vEdge device learns these prefixes, it redistributes them into OMP as vRoutes so they can be carried across the overlay fabric. 

●      TLOC routes advertise Transport Locators of the connected WAN transports, along with additional attributes such as public and private IP addresses, color, TLOC preference, site ID, weight, tags, and encryption keys.

●      Service routes advertise embedded network services such as firewalls and IPS that are connected to the vEdge local-site network.

Cisco SD-WAN Overlay Management Protocol
Figure 5. Cisco SD-WAN Overlay Management Protocol

vRoutes

The following output shows an example of a Cisco SD-WAN vRoute.

---------------------------------------------------
omp route entries for vpn 1 route 172.16.50.0/24
---------------------------------------------------
            RECEIVED FROM:                   
peer            1.1.0.3
path-id         2
label           1
status          C,I,R
loss-reason     not set
lost-to-peer    not set
lost-to-path-id not set
    Attributes:
     originator       50.50.50.50
     type             installed
     tloc             50.50.50.50, mpls, ipsec
     ultimate-tloc    not set
     domain-id        not set
     overlay-id        1
     site-id          50
     preference       not set
     tag              not set
     origin-proto     connected
     origin-metric    0
     as-path          not set
     unknown-attr-len not set

Let's look at every attribute in the OMP route in more detail. Each one can be used as a match criteria when creating policies and also can be modified in order to influence the routing decisions across the overlay fabric.

  • Originator: The Originator attribute is pretty much self-explanatory. It identifies where the route was originally learned from.
  • TLOC: The Transport Location (TLOC) is the next hop tunnel endpoint of the route. It is similar to the BGP_NEXT_HOP attribute. In order for a vRoute to be considered valid, it must have a next-hop TLOC that is known and reachable, where reachable means that there must be at least one IPsec tunnel to that TLOC with a BFD session in UP state.
  • Site ID: The Site ID value represents the site of origin of the vRoute and is primarily used for loop-prevention. It is very similar to the BGP autonomous system number (ASN). The value can also be used for influencing routing decisions and policy orchestration. All sites must have unique Site-IDs and in sites where there are multiple WAN edge devices, they must have the same site ID for loop prevention.
  • Preference: The Preference value is designed to be used for influencing the best-path selection process for a given prefix. It is pretty much the same as LOCAL_PREF in BGP - a higher preference is preferred over a lower one. 
  • Tag: This is an optional, transitive attribute that operates similarly to the route tags in traditional routing protocols. It can be matched and acted upon via policy.
  • Origin: This value represents the originating protocol of the prefix. It can be BGP, OSPF, EIGRP, Connected, or Static. Origin is used in the OMP best-path selection for vRoutes and also can be influenced by a policy.
  • Origin-Metric: This value represents the originating protocol-metric of the prefix. 
  • VPN: This value shows what VPN this route was advertised from. VPN tags allow for logical separation of networks and the use of overlapping subnets, assuming that they live in different VPNs.

TLOCs

Transport Locators (TLOCs) represent the attachment points where a vEdge router connects to the available WAN transports. A TLOC is uniquely identified by a tuple of three values - System-IP, color, and encapsulation type. However, TLOC routes have a number of other attributes that are advertised to the vSmart controllers, such as private and public IP addresses, port numbers, and encryption keys. Each WAN edge device advertises its Transport Locators via OMP TLOC routes to all Cisco vSmart controllers. The controllers then redistribute the TLOC routes to all other vEdge routers. When a vEdge receives a new TLOC route, it attempts to form an IPsec tunnel and establish a BFD session over each available WAN transport. However, WAN edge devices do not form overlay tunnels to devices with the same site-id.

Cisco SD-WAN Tranport Locators (TLOCs)
Figure 6. Cisco SD-WAN Tranport Locators (TLOCs)

A TLOC route contains the following pieces of information:

  • TLOC private address: This is the private IP address of the WAN edge's interface attached to the WAN transport.
  • TLOC public address: This value contains the publicly routable IP address of the WAN transport interface if it is connected behind NAT. WAN edge devices use STUN (RFC 5389) protocol that allows both ends to find out their public address, the type of NAT they are behind, and associated port numbers. This is a very important part of supporting data plane connectivity across a NAT boundary. If both the public and private addresses are equal, the vEdge is considered to not be behind a NAT.
  • Color: Colors are logical abstractions used to identify WAN transports that are attached to WAN Edge devices. They are statically defined keywords and are globally significant and identify an individual transport as either public or private. Possible options are 3g, biz-internet, blue, bronze, custom1, custom2, custom3, default, gold, green, lte, metro-ethernet, mpls, private1, private2, private3, private4, private5, private6, public-internet, red, and silver.
  • Encapsulation type: The encapsulation type could be IPsec or GRE. To successfully form a data plane tunnel to another TLOC, both sides must use the same encryption type.
  • Preference: The TLOC preference value makes one TLOC more preferred over another when comparing the same OMP route. A higher value is better.
  • Site ID: This value identifies the originator of the TLOC route. It is used for loop prevention and it controls how data plane tunnels are built. WAN edge devices do not establish tunnels to other WAN edge routers in the same site (having the same site-id)
  • Tag: TLOC tags are used in the same way as OMP and route tags.
  • Weight: This value is pretty much the same as Weight in BGP. It is locally significant and is used for path selection manipulation. Higher is better.
---------------------------------------------------
tloc entries for 50.50.50.50
                 mpls
                 ipsec
---------------------------------------------------
            RECEIVED FROM:                   
peer            1.1.0.3
status          C,I,R
loss-reason     not set
lost-to-peer    not set
lost-to-path-id not set
    Attributes:
     attribute-type    installed
     encap-key         not set
     encap-proto       0
     encap-spi         256
     encap-auth        sha1-hmac,ah-sha1-hmac
     encap-encrypt     aes256
     public-ip         135.13.56.182
     public-port       12346
     private-ip        10.50.1.1
     private-port      12346
     public-ip         ::
     public-port       0
     private-ip        ::
     private-port      0
     bfd-status        up
     domain-id         not set
     site-id           50
     overlay-id        not set
     preference        0
     tag               not set
     stale             not set
     weight            1
     version           2
    gen-id             0x80000011
     carrier           default
     restrict          0
     groups            [ 0 ]
     border             not set
     unknown-attr-len  not set

Service routes

As the name implies, Service routes represent services like firewalls and IPS that are connected to a WAN edge device or to the local-network attached to the WAN edge device. It is important to note that these devices that provide network services for the overlay fabric must be Layer 2 adjacent to the WAN edge device (meaning that there cannot be any intermediate hops between the WAN Edge router and the device performing the service). 

Comments

Rynqlo

Fri, 10/15/2021 - 21:24

Thanks for the tutorial. I have a question for your.
In the TLOC route output, there is a field called Public IP. But is a MPLS circuit required a public IP?

In reply to by Rynqlo

abhisinghvaz

Sun, 10/24/2021 - 10:57

I think, this can be incase of second vedge or Control devices sitting on internet and Wedge behind MPLS network