Step 2 - Deploying Controllers

Now it is time to deploy controllers.

Get the Software Images

To deploy the Cisco Catalyst SD-WAN controllers, of course, you first need to download the software images. Here you typically have two options:

Option 1: Download the images from Cisco.com

The first option is to download the controller images directly from Cisco.com. In this case, you need an account with the proper download entitlement. Therefore, if you work for an organization that already uses Cisco equipment, the easiest approach is usually to ask the Smart Account administrators to download the images for you.

An important detail is to download the appropriate images for your specific deployment. If you are going to use EVE-NG, GNS3, PNETlabs, or other similar KVM-based platforms, you need to download the KVM controller images with the extension qcow2.

For example, in our example, we are going to use the following controller images:

Cisco Catalyst SD-WAN Manager: viptela-vmanage-26.1.1.1-genericx86-64.qcow2
Cisco Catalyst SD-WAN Controller: viptela-smart-26.1.1.1-genericx86-64.qcow2
Cisco Catalyst SD-WAN Validator: viptela-bond-26.1.1.1-genericx86-64.qcow2

Option 2: Use Cisco CML as a virtual environment

If you do not have access to a corporate Smart Account, the easiest option is to use Cisco Modeling Labs (CML). Cisco CML provides the Catalyst SD-WAN images built-in (through the supplemental ISO). This means you can build an SD-WAN lab without manually searching for each controller image separately. This makes CML especially useful for students and engineers, who don't have access to a corporate SA account.

The only real downside of Cisco CML is that it is a paid product. At the time of writing, a personal CML license costs around US$199 per year. However, Cisco regularly offers discounts, sometimes up to 50%, which means you may be able to purchase it for around US$100.

Set up the Virtual Environment

Once you have the controller images, you need to set up your virtual environment.

First, download the EVE-NG virtual machine from its official website and allocate it sufficient resources. Keep in mind that Cisco SD-WAN requires significant processing power. For a small practicing topology consisting of 1 controller of each type and 3-4 cEdges, you should give the EVE-NG VM at least 16 vCPUs and 64GB of RAM. However, if you want to build a large topology with redundant controllers and many cEdge devices, you will need substantial computing resources.

Additionally, do not forget to enable the Intel VT-x or AMD-V option within the virtual machine and add a secondary NIC card.
Just for reference, all lab examples in this book were done using an EVE-NG Community VM with 8vCPUs and 96GB of RAM, as shown in Figure 12.9.

Images

If you want to copy-paste everything from here onward, you need to have the following Cisco SD-WAN images to set up this practice lab environment:

EVE-NG Image    Filename    Version
vmanage-20.9.1    viptela-vmanage-20.9.1-genericx86-64.qcow2    20.9.1
vsmart-20.9.1    viptela-smart-20.9.1-genericx86-64.qcow2    20.9.1
vbond-20.9.1    viptela-edge-20.9.1-genericx86-64.qcow2    20.9.1
c8000v-17.09.01a    c8000v-universalk9_8G_serial.17.09.01a.qcow2    17.09.01a

However, if you deploy another firmware version, you must modify some commands related to the file versions.
Once you have the images, you must create a folder for each and transfer them to the EVE-NG VM using a Frezzila or WinSCP tool. After uploading the images to their respective folders in the virtual machine, you must rename the original files to "virtioa". Additionally, you need to create an additional virtual disk for the vManage controller. You can do this with the command highlighted in green:

#  Creating Folders 
mkdir /opt/unetlab/addons/qemu/vtbond-20.9.1
mkdir /opt/unetlab/addons/qemu/vtsmart-20.9.1
mkdir /opt/unetlab/addons/qemu/vtmgmt-20.9.1
mkdir /opt/unetlab/addons/qemu/c8000v-17.09.01a
## vManage 
cd /opt/unetlab/addons/qemu/vtmgmt-20.9.1
mv viptela-vmanage-20.9.1-genericx86-64.qcow2 virtioa.qcow2
/opt/qemu/bin/qemu-img create -f qcow2 virtiob.qcow2 150G
## vSmart
cd /opt/unetlab/addons/qemu/vtsmart-20.9.1
mv viptela-smart-20.9.1-genericx86-64.qcow2 virtioa.qcow2
## vBond
cd /opt/unetlab/addons/qemu/vtbond-20.9.1
mv viptela-edge-20.9.1-genericx86-64.qcow2 virtioa.qcow2
## cEdge
cd /opt/unetlab/addons/qemu/c8000v-17.07.01a
mv c8000v-universalk9_8G_serial.17.09.01a.qcow2 virtioa.qcow2
## Fix permissions
/opt/unetlab/wrappers/unl_wrapper -a fixpermissions

Once you log into the EVE-NG GUI, you must edit the cEdge nodes and specify 10 Ethernet interfaces on each.
Physical Topology
The physical topology that we are going to use throughout the entire book is as shown in figure 12.10. You must re-create it on EVE-NG. If you want to copy/paste some of the configs, make sure that you use the same interfaces when creating the topology on EVE-NG.

Figure 12.10. EVE-NG physical topology.

Once the physical topology is re-created within the EVE-NG GUI, you can power up each node and log in via console.
Bootstrap Configuration

Default credentials for all devices are admin/admin. When vManage boots for the first time, it will ask on which storage device to install the software. Please make sure to use the virtual disk you have created in the previous step.

viptela 20.9.1
vmanage login: admin
Password: admin
Welcome to Viptela CLI
admin connected from 127.0.0.1 using console on vmanage
You must set an initial admin password.
Password: ********
Re-enter password: ********
1) COMPUTE_AND_DATA
2) DATA
3) COMPUTE
Select persona for vManage [1,2 or 3]: 1
You chose persona COMPUTE_AND_DATA (1)
Are you sure? [y/n] y
Available storage devices:
vdb     150GB
sr0     0GB
1) vdb
2) sr0
Select storage device to use: 1
Would you like to format vdb? (y/n): y

Once all devices boot up, it is time to enable basic connectivity between the Cisco SD-WAN controllers. The following bootstrap configurations are the minimum required to achieve basic connectivity. Notice that the tunnel interfaces are disabled in the beginning. Once all devices join the control plane, we will enable the overlay fabric.

vBond
system
host-name vBond
system-ip 1.1.1.10
site-id 100
organization-name networkacademy-io
vbond 39.1.1.10 local
!
vpn 0
interface ge0/0
 ip address 39.1.1.10/24
 no tunnel-interface
 no shutdown
!
ip route 0.0.0.0/0 39.1.1.254
!

vManage
!
system
host-name vManage
system-ip 1.1.1.20
site-id 100
organization-name networkacademy-io
vbond 39.1.1.10
!
vpn 0
interface eth0
 ip address 39.1.1.20/24
 no shutdown
!
ip route 0.0.0.0/0 39.1.1.254
!
vpn 512
interface eth1
 ip address 192.168.115.20/24
 no shutdown
!
!

vSmart
system
host-name vSmart
system-ip 1.1.1.30
site-id 100
organization-name networkacademy-io
vbond 39.1.1.10
!
vpn 0
interface eth0
 ip address 39.1.1.30/24
 no tunnel-interface
 no shutdown
!
ip route 0.0.0.0/0 39.1.1.254
!

At this point, each controller must be able to ping any other in VPN0 successfully. If, for whatever reason, there is no reachability to one of the devices, you should not continue ahead but troubleshoot and resolve the issue.

Certificates

Cisco SD-WAN Controllers can not be brought into operation unless an established chain of trust validates their identity. This identity validation process is intended to ensure that only trusted devices can join the SD-WAN solution while still retaining flexibility. Each controller must have a root certificate installed and a controller certificate installed and signed by a trusted Certification Authority (CA).

We will use the vBond controller as an Enterprise Root CA to create this lab environment.

vBond as a Root CA

The first step is to generate an RSA private key.

#Use “vshell” / “viptela_cli” to switch between Shell and CLI modes
vBond# vshell 
vBond:~$ openssl genrsa -out ROOTCA.key 2048
Generating RSA private key, 2048 bit long modulus
........+++..............................................+++
e is 65537 (0x10001)

Then we generate a ROOTCA.pem certificate and sign it with the private key we created. The ROOTCA.pem file will be the root certificate we must install on all cEdge routers and SD-WAN controllers.

vBond:~$ openssl req -x509 -new -nodes -key ROOTCA.key -sha256 -days 1024 \
>     -subj "/C=US/ST=NY/L=NY/O=networkacademy-io/CN=networkacademy-io" \
>     -out ROOTCA.pem

vBond:~$ ls -l
total 12
-rw-r--r-- 1 admin admin 1679 Mar 23 09:41 ROOTCA.key
-rw-r--r-- 1 admin admin 1285 Mar 23 09:41 ROOTCA.pem

Now you need to log in to the vManage GUI interface. This is done using a web browser and entering the URL “https://[vManage-VPN512-IP-address]:8443”. Once logged in, you need to configure the following:

Go to Administration > Settings and set the Organization Name to be “networkacademy-io”.
Edit the vBond address, and put it to 39.1.1.10.
Go to Controller Certificate Authorization and set it to Enterprise Root Certificate. Then log in to the vBond shell, take the content of ROOTCA.pem and paste it into the Certificate field. Then you click Import & Save.
Ensure that the WAN Edge Cloud Certificate Authorization is set to Automated.

Figure 12.11 shows where to configure the parameters on the vManage GUI.

Figure 12.11. vManage General Settings.

Adding Controllers

Once we complete these steps, the vManage controller will have our root certificate. It is time to add the other SD-WAN controllers via the vManage GUI. We go to Configuration > Devices > Controllers > Add Controller and add the vBond and vSmart controllers. We insert the VPN0 IP address of each controller alongside the username and password.
At this point, the vManage controller installs the root certificate to each SD-WAN controller. Once completed, each controller must have our Enterprise root certificate. We should verify this using the “show certificate root-ca-cert” command, as shown in the output below.

vManage# show certificate root-ca-cert | in networkacademy
     Issuer: C=US, ST=NY, L=NY, O=networkacademy-io, CN= networkacademy-io
     Subject: C=US, ST=NY, L=NY, O=networkacademy-io, CN=networkacademy-io

vSmart# show certificate root-ca-cert | in networkacademy
     Issuer: C=US, ST=NY, L=NY, O=networkacademy-io, CN=networkacademy-io
     Subject: C=US, ST=NY, L=NY, O=networkacademy-io, CN=networkacademy-io

vBond# show certificate root-ca-cert | in networkacademy
     Issuer: C=US, ST=NY, L=NY, O=networkacademy-io, CN=networkacademy-io
     Subject: C=US, ST=NY, L=NY, O=networkacademy-io, CN=networkacademy-io

If a controller does not have our Enterprise root of trust, you should not continue ahead, but instead, troubleshoot the problem.
Controller Certificates

Now it is time to install a device certificate on each SD-WAN controller. We go to Configuration > Certificates> Controllers and generate a certificate signing request (CSR) for each one, as shown in Figure 12.12.

Figure 12.12 Generating CSRs

Ensure that each controller's operational status is “CSR Generated.” If that’s the case, we can go ahead and copy the CSR files to our Root CA (which is the vBond controller) and sign them.

The following output shows how to copy the CSR files to vBond.

vBond# vshell
vBond:~$ scp admin@39.1.1.30:/home/admin/vsmart_csr vSmart.csr
vBond:~$ scp admin@39.1.1.20:/home/admin/vmanage_csr vManage.csr
vBond:~$ scp admin@39.1.1.10:/home/admin/vbond_csr vBond.csr

vBond:~$ ls -alh | grep csr
-rw-------  1 admin admin 1.2K Dec 31 16:16 vBond.csr
-rw-------  1 admin admin 1.2K Dec 31 16:16 vManage.csr
-rw-------  1 admin admin 1.2K Dec 31 16:16 vSmart.csr
-rw-r--r--  1 root  root  1.2K Dec 31 15:27 vbond_csr

Now that the CSR files are present, we sign them with the RootCA private key and generate a certificate for each controller, as shown in the output below.

vBond:~$ openssl x509 -req -in vBond.csr \
>     -CA ROOTCA.pem -CAkey ROOTCA.key -CAcreateserial \
>     -out vBond.crt -days 500 -sha256
Signature ok

vBond:~$ openssl x509 -req -in vSmart.csr \
>     -CA ROOTCA.pem -CAkey ROOTCA.key -CAcreateserial \
>     -out vSmart.crt -days 500 -sha256
Signature ok

 vBond:~$ openssl x509 -req -in vManage.csr \
>     -CA ROOTCA.pem -CAkey ROOTCA.key -CAcreateserial \
>     -out vManage.crt -days 500 -sha256
Signature ok

In the end, we must have three certificates, as seen in the following output.

vBond:~$ ls -alh | grep crt
-rw-r--r--  1 admin admin  26K Dec 31 15:27 master_root.crt
-rw-------  1 admin admin 1.4K Dec 31 16:25 vBond.crt
-rw-------  1 admin admin 1.4K Dec 31 16:30 vManage.crt
-rw-------  1 admin admin 1.4K Dec 31 16:26 vSmart.crt
vBond:~$

It is time to install all controller certificates via the vManage GUI. Go to Configuration > Certificates > Controllers > Install Certificate.

Now that you have all certificates (.crt) in vBond's directory, you simply cat each of them and paste the output in the Install Certificate window as shown in figure 12.13.

For example, we take the value of vManage.crt as shown in the output below.

vBond:~$ cat vManage.crt 
-----BEGIN CERTIFICATE-----
MIIDtzCCAp8CFECnnxKSiDGvEN88bclv/fsc/ajKMA0GCSqGSIb3DQEBCwUAMF8x
CzAJBgNVBAYTAlVTMQswCQYDVQQIDAJOWTELMAkGA1UEBwwCTlkxGjAYBgNVBAoM
EW5ldHdvcmthY2FkZW15LWlvMRowGAYDVQQDDBFuZXR3b3JrYWNhZGVteS1pbzAe
Fw0yMjEyMzExNjMwMzdaFw0yNDA1MTQxNjMwMzdaMIHQMQswCQYDVQQGEwJVUzET
MBEGA1UECBMKQ2FsaWZvcm5pYTERMA8GA1UEBxMIU2FuIEpvc2UxGjAYBgNVBAsT
ZAQ4EDJsOILeeW3/hWyXUUUrf2oWwkUtMiRSqxCSuBQm0DsifCKtPNlKzQ==
-----END CERTIFICATE-----
vBond:~$

And paste the value in the “Install Certificate” window, as shown in the screenshot below.

Figure 12.13. Installing Controller Certificates.

If everything is good up to this point, the certificate should install successfully.

You repeat this for all controllers. In the end, when you go to Configuration > Certificates > Controllers, you should see that all controllers have Certificate Serial numbers. If that is the case, you click the Send-to-vBond function to propagate this information to vBond.

At this point, all controllers should be operational with valid certificates. Now it is time to bring up the tunnel interfaces using the configuration below:

# on vManage and vSmart
vpn 0
interface eth0
 tunnel-interface
!

# on vBond
vpn 0
interface ge0/0
 tunnel-interface
  encapsulation ipsec
  allow-service all
!

Verification

If everything has been successfully completed up to this point, each controller should have both root and device certificates installed and a serial number, as shown in the output below.

vBond# show control local-properties 
personality                       vedge
sp-organization-name              networkacademy-io
organization-name                 networkacademy-io
root-ca-chain-status              Installed
root-ca-crl-status                Not-Installed
certificate-status                Installed
certificate-validity              Valid
certificate-not-valid-before      Dec 31 16:25:01 2022 GMT
certificate-not-valid-after       May 14 16:25:01 2024 GMT
dns-name                          39.1.1.10
site-id                           100
domain-id                         1
protocol                          dtls
tls-port                          0
system-ip                         1.1.1.10
chassis-num/unique-id             cc3ac783-82d5-4f43-898f-4071a20f5exx
serial-num                        40A79F12928831AF10DF3C6DC96FFDFB1CFDA8XX
subject-serial-num                N/A
token                             Invalid
keygen-interval                   1:00:00:00
retry-interval                    0:00:00:15
no-activity-exp-interval          0:00:00:20
dns-cache-ttl                     0:00:00:00
port-hopped                       TRUE
time-since-last-port-hop          0:00:00:40
pairwise-keying                   Disabled
embargo-check                     success
cdb-locked                        false
device-role                       edge-router
region-id-set                     N/A
number-vbond-peers                0
number-active-wan-interfaces      1

Additionally, as shown in the output below, each one must have established control connections to all other controllers.
vBond# show orchestrator connections | t

PEER SITE PUBLIC SYSTEM LOCAL
TYPE ID PUBLIC IP PORT PROTOCOL IP COLOR STATE UPTIME
-------------------------------------------------------------------------------
vsmart 100 39.1.1.30 12346 dtls 1.1.1.30 default up 0:00:01:19
vsmart 100 39.1.1.30 12446 dtls 1.1.1.30 default up 0:00:01:18
vmanage 100 39.1.1.20 12346 dtls 1.1.1.20 default up 0:00:01:32
vmanage 100 39.1.1.20 12446 dtls 1.1.1.20 default up 0:00:01:31
vmanage 100 39.1.1.20 12546 dtls 1.1.1.20 default up 0:00:01:31
vmanage 100 39.1.1.20 12646 dtls 1.1.1.20 default up 0:00:01:31
If everything is working up to this point, we can now go ahead and provision the Edge routers.