In this lesson, we will explore the process of onboarding cEdge routers (CSR1k) to the Cisco SD-WAN fabric using Enterprise CA and manual bootstrap configuration. This lesson complements our two previous articles on setting up a Cisco SD-WAN home lab for practicing.
Initial Topology
Figure 1 shows the physical topology that we will use in this example. All cEdge routers are CSR 1000v running Cisco IOS-XE version 16.12.04. We won't go into the process of bringing up the SD-WAN controllers in this lesson and will jump straight into the cEdge onboarding process. If you want to understand how to deploy the Cisco SD-WAN controllers using local Enterprise CA, check out this lesson.
Notice that the example will only show how to onboard cEdge-1. If you want to set up the entire topology, you will need to repeat this for each WAN edge device.
Onboarding cEdge routers
Prepare the software image
When the router boots up, we first stop the PnP service so that the SD-WAN software packages can install. We do this using the following command in exec mode.
cEdge# pnpa service discovery stop
Once the PnP service has been stopped, we tell the router to install all underlying SD-WAN packages if necessary. Depending on the CSR1k software image, this may not be necessary. However, in practice environments, it could only help.
cEdge# request platform software sdwan software reset
The last step is to verify the software image using the following command.
cEdge# request platform software sdwan software upgrade-confirm
You should see that the sdwan software is ACTIVE and CONFIRMED as highlighted below.
cEdge# show sdwan soft
VERSION ACTIVE DEFAULT PREVIOUS CONFIRMED TIMESTAMP
---------------------------------------------------------------------------------
16.12.4.0.4480 true true false user 2022-04-03T08:20:13-00:00
Total Space:388M Used Space:87M Available Space:297M
Bootstrap the cEdge router
Once the router loads up with the SD-WAN software, we can go ahead and configure the minimal configuration required to join the SD-WAN overlay fabric. Notice that when the cEdge router runs in Controller mode (basically SD-WAN mode), we enter the configuration mode using the "config-transaction" command instead of the well-known "configure terminal" or simply "conf t".
In the following example, we configure the basic IP addressing and default routing alongside the essential system parameters such as system-IP, site-id, and vBond address. In this example, we will configure a DNS name for vBond, as is recommended by Cisco.
cEdge# config-transaction
hostname cEdge
!
int GigabitEthernet1
ip address 39.3.1.1 255.255.255.0
no shut
!
int GigabitEthernet2
ip address 10.10.1.1 255.255.255.0
no shut
!
ip route 0.0.0.0 0.0.0.0 39.3.1.254
ip route 0.0.0.0 0.0.0.0 10.10.1.254
ip host vbond.networkacademy.io 10.1.1.10
!
system
system-ip 1.1.1.1
site-id 1
organization-name "networkacademy-io"
vbond vbond.networkacademy.io
commit
At this point, you should be able to ping all Cisco SD-WAN controllers from the cEdge router that is being onboarded. If there is no IP connectivity between the WAN edge router and the controllers, there is no point in continuing further. You should troubleshoot the problem first.
If there is IP reachability between the cEdge router and the controllers, we are ready to configure the SD-WAN overlay tunnels. Notice something very important - the Tunnel keyword in the "interface Tunnel" command should always be with a capital T. It is not like in a regular Cisco IOS where you can create a new tunnel using the "interface tunnel 1" command.
sdwan
int GigabitEthernet1
tunnel-interface
color biz-internet
encapsulation ipsec
!
int GigabitEthernet2
tunnel-interface
color mpls restrict
encapsulation ipsec
!
interface Tunnel 1
ip unnumbered GigabitEthernet1
tunnel source GigabitEthernet1
tunnel mode sdwan
!
interface Tunnel 2
ip unnumbered GigabitEthernet2
tunnel source GigabitEthernet2
tunnel mode sdwan
commit
At this point, the CSR1K router has the required configuration to join the SD-WAN fabric. The next step is to install the root certificate and activate the routers.
Installing the Root CA Certificate
Now it is time to install the root CA certificate to the cEdge router. In our previous lessons, we have deployed the Cisco SD-WAN controllers using the Enterprise CA method with the vBond controller acting as a Root CA. If you have deployed the controllers following one of our lessons, you should have the Root CA certificate on vBond named ROOTCA.pem. The easiest way to install the root certificate on a cEdge CSR1K router is by creating a local file directly on the router using TCLSH, as shown in the following example. In the highlighted section, you should paste the ROOTCA.pem certificate which you take from vBond using the "cat ROOTCA.pem" command in vshell mode.
cEdge# tclsh
cEdge(tcl)# puts [open "flash:ROOTCA.pem" w+] {
+> paste root-cert-here
+> }
cEdge-1(tcl)# exit
In the end, you should have the root certificate in the cEdge router's bootflash, as shown below.
cEdge# more bootflash:ROOTCA.pem
-----BEGIN CERTIFICATE-----
MIIDjzCCAnegAwIBAgITHFuqFiCbYL/pMA0GCSqGSIb3DQEBCwUAMF4xCzAJBgNV
BAYTAlVTMQswCQYDVQQIDAJOWTELMAkGA1UEBwwCTlkxGjAYBgNVBAoMEW5ldHdv
cmthY2FkZW15LWlvMRkwFwYDVQQDDBByb290LmNlcnRpZmljYXRlMB4XDTIyMDQw
MzA3NTIzN1oXDTI1MTFHMTA3NTIzN1owXjELMAkGA1UEBhMCVVMxCzAJBgNVBAgM
Ak5ZMQswCQYDVQQHDAJOWTEaMBgGA1UECgwRbmV0d29ya2FjYWRlbXktaW8xGTAX
BgNVBAMMEHJvb3QuY2VydGlmaWNhdGUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAw
ggEKAoIBAQDOw7rS5HFTyuXBrsPISA9i8f5yR+gFDFeBCWlnXy6q1ztKQT5T+KtK
WoSVmAJTU9kAug1ACq4n/KSRXBxZu/YXzKF6qH8E26UBt6xofSAJBf3pIg3eIxjk
KY1+sSTxjUIg601GFMaGO0fYBQyj9xBJmIuiHWJOd0GHf6P0M48HJt39YJtKiV48
J1K20Arke5qPGNw86coRhFs//v/wqQesHobEGHuObCSZU73qTIrj92CjYiJ2gXID
PO+nKHNGhO3osGe9GFBQe1KFRsVbp7w/Nt8nrQUT2eW4/uie2YCd6Aw6e5A9pAWY
ob6QOzstw7TZ3AlLJ6uAbWMmdIlWhFO3AgMBAAGjUDBOMB0GA1UdDgQWBBRdh01N
M09x6ZVahuBFw498caqeUTAfBgNVHSMEGDAWgBRdh01NM09x6ZVahuBFw498caqe
UTAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQB+/HruVYFQr9suYBX8
WtyS8/SWSkXrjz4jQSYPGLZSSuKFUhaVOkV/Vld3oewAzEvGXamXwWJKRNFiJrSm
qiL1t18DrYQZ3PkqMbsrmjSCoxmY8lTFWEnWUoETmevgDDGA1y68n91yPRyQBpIZ
h5o66e4ZH7LHRxgDWRWOWjNSIrvQzKPQrwYg9p9yfUKboo0su5tuJBnyYV5s6GTI
JxeMfP8GBtwB6JcecE7EdnGI7gZh4bWJwGCBO4lZgUK8CjxAnMfR1mNrDHZhOqES
pK9s66SjJM5PAD5tJ4Qz0NWPJu5F1IzHXa3skAK3vHkgTixDocaIERh+DFYeDwkm
/dfY
-----END CERTIFICATE-----
The last step is to install the root certificate using the following command.
cEdge# request platform software sdwan root-cert-chain install bootflash:ROOTCA.pem
Uploading root-ca-cert-chain via VPN 0
Copying ... /bootflash/ROOTCA.pem via VPN 0
Updating the root certificate chain..
Successfully installed the root certificate chain
If everything has gone smoothly, you should see our custom root certificate installed on the router.
cEdge#show sdwan certificate root-ca-cert | in network
Issuer: C=US, ST=NY, L=NY, O=networkacademy-io, CN=root.certificate
Subject: C=US, ST=NY, L=NY, O=networkacademy-io, CN=root.certificate
Activating the cEdge router
Before the cEdge router can be able to join the SD-WAN fabric, it must have a device certificate, signed and installed by vManage. To do this, we need to have an available unused chassis number and token in vManage under Configuration > Certificates. We use them to activate the cEdge router as shown in the example below.
cEdge# request platform software sdwan vedge_cloud activate chassis-number CSR-D4B9356B-XXXX-XXXX-XXXX-XXXXXXXXXXXX token af6bd685d8674996xxxxxxxxxxxxxxxx
Once you've done, you should see in the logs that vManage logs into the cEdge using NETCONF over SSH, generates a CSR, then signs it and install a device certificate. Then the cEdge router should establish an OMP peering with vSmart and start receiving TLOCs and OMP routes.
cEdge#
*Apr 3 09:38:11.896: %DMI-5-AUTH_PASSED: R0/0: dmiauthd: User 'vmanage-admin' authenticated successfully from 1.1.1.20:41146 and was authorized for netconf over ssh. External groups:
*Apr 3 09:38:19.308: %Cisco-SDWAN-cEdge-2-action_notifier-6-INFO-1400002: R0/0: VCONFD_NOTIFIER: Notification: 4/3/2022 9:38:19 security-install-csr severity-level:minor host-name:default system-ip:1.1.1.1
*Apr 3 09:38:20.525: %DMI-5-AUTH_PASSED: R0/0: dmiauthd: User 'vmanage-admin' authenticated successfully from 1.1.1.20:41158 and was authorized for netconf over ssh. External groups:
*Apr 3 09:38:26.902: %Cisco-SDWAN-cEdge-2-action_notifier-6-INFO-1400002: R0/0: VCONFD_NOTIFIER: Notification: 4/3/2022 9:38:26 security-install-rcc severity-level:minor host-name:default system-ip:1.1.1.1
*Apr 3 09:38:45.329: %Cisco-SDWAN-cEdge-2-action_notifier-6-INFO-1400002: R0/0: VCONFD_NOTIFIER: Notification: 4/3/2022 9:38:45 security-install-certificate severity-level:minor host-name:default system-ip:1.1.1.1
*Apr 3 09:39:22.177: %Cisco-SDWAN-RP_0-OMPD-6-INFO-400002: vSmart peer 1.1.1.30 state changed to Init
*Apr 3 09:39:24.257: %Cisco-SDWAN-RP_0-OMPD-6-INFO-400002: R0/0: OMPD: vSmart peer 1.1.1.30 state changed to Handshake
*Apr 3 09:39:24.261: %Cisco-SDWAN-RP_0-OMPD-5-NTCE-400002: R0/0: OMPD: vSmart peer 1.1.1.30 state changed to Up
*Apr 3 09:39:24.263: %Cisco-SDWAN-RP_0-OMPD-6-INFO-400005: R0/0: OMPD: Number of vSmarts connected : 1
Verifications
If you've done everything and the onboarding process has been successful, the router should have both device and root certificates installed and a valid serial number, as highlighted below.
cEdge#show sdwan control local-properties
personality vedge
sp-organization-name networkacademy-io
organization-name networkacademy-io
root-ca-chain-status Installed
certificate-status Installed
certificate-validity Valid
certificate-not-valid-before Apr 03 09:38:19 2022 GMT
certificate-not-valid-after Mar 31 09:38:19 2032 GMT
enterprise-cert-status Not-Applicable
enterprise-cert-validity Not Applicable
enterprise-cert-not-valid-before Not Applicable
enterprise-cert-not-valid-after Not Applicable
dns-name vbond.networkacademy.io
site-id 1
domain-id 1
protocol dtls
tls-port 0
system-ip 1.1.1.1
chassis-num/unique-id CSR-D4B9356B-B36E-5EEC-XXXX-XXXXXXXXXXXX
serial-num CB17XXXX
token Invalid
keygen-interval 1:00:00:00
retry-interval 0:00:00:15
no-activity-exp-interval 0:00:00:20
dns-cache-ttl 0:00:02:00
port-hopped TRUE
time-since-last-port-hop 0:00:12:01
embargo-check success
number-vbond-peers 0
number-active-wan-interfaces 1
If you wonder what is the role of the root certificate, or what is the process of building and verifying the identity of sdwan devices, you could check out our lesson for SD-WAN Certificates.
The ultimate verification will be to check the control connections to the sdwan controllers.
cEdge#show sdwan control connections
PEER PEER CONTROLLER
PEER PEER PEER SITE DOMAIN PEER PRIV PEER PUB GROUP
TYPE PROT SYSTEM IP ID ID PRIVATE IP PORT PUBLIC IP PORT LOCAL COLOR PROXY STATE UPTIME ID
-------------------------------------------------------------------------------------------------------------------------------------
vsmart dtls 1.1.1.30 1 1 10.1.1.30 12346 10.1.1.30 12346 biz-internet No up 0:00:05:24 0
vmanage dtls 1.1.1.20 1 0 10.1.1.20 12546 10.1.1.20 12546 biz-internet No up 0:00:05:27 0
Key Takeaways
The following table shows the difference between the onboarding configuration fo a vEdge router running Viptela OS and a cEdge router running Cisco IOS-XE.
| vEdge(Viptela OS) Minimal Bootstrap Config | cEdge(IOS-XE) Minimal Bootstrap Config |
|---|---|
| Router# conf t | Router# config-t |
| Router(config)# | Router(config)# |
|
system |
hostname cEdge |
| vEdge(config)# commit | cEdge(config)# commit |