Cisco SD-WAN is a major topic in the CCE Enterprise blueprint now. Network engineers that want to pass the lab exam should have extensive hands-on experience with the solutions. However, for people who do not have the chance to touch it at work, it is actually not easy to get access to a practice lab. In this lesson, I will show you one way to set up a fully functional Cisco SD-WAN home lab on EVE-NG that can be used to practice every topic in the exam's blueprint.
One thing I'd like to mention is that Cisco SD-WAN requires a lot of processing power. For a small practicing topology consisting of 1 controller of each type and 3-4 vEdges, you should give the EVE-NG VM at least 8 vCPUs and 16GB of RAM. However, if you want to make a large topology with redundant controllers and many vEdge devices, you must have a lot of computing resources at your disposal.
Cisco SD-WAN Images
You will need to have the following Cisco SD-WAN images to set up this practice lab environment on EVE-NG:
Once you have the images, you need to create a folder for each one and then transfer the image to EVE-NG using a FileZilla or WinSCP tool. After you upload the images to their respective folders in EVE-NG, you need to convert the ova files to qcow2. Note that we need to create an additional virtual disk for vManage. This is done with the command highlighted in green:
# Creating Folders mkdir /opt/unetlab/addons/qemu/vtbond-16.3.2 mkdir /opt/unetlab/addons/qemu/vtsmart-16.3.2 mkdir /opt/unetlab/addons/qemu/vtmgmt-16.3.2 mkdir /opt/unetlab/addons/qemu/vtedge-17.1 ##vManage cd /opt/unetlab/addons/qemu/vtmgmt-16.3.2 tar -xvf viptela-vmanage-genericx86-64.ova /opt/qemu/bin/qemu-img convert -f vmdk -O qcow2 viptela-vmanage-genericx86-64-disk1.vmdk virtioa.qcow2 /opt/qemu/bin/qemu-img create -f qcow2 virtiob.qcow2 150G ##vSmart cd /opt/unetlab/addons/qemu/vtsmart-16.3.2 tar -xvf viptela-smart-genericx86-64.ova /opt/qemu/bin/qemu-img convert -f vmdk -O qcow2 viptela-smart-genericx86-64-disk1.vmdk virtioa.qcow2 ##vBond cd /opt/unetlab/addons/qemu/vtbond-16.3.2 tar -xvf viptela-edge-genericx86-64.ova /opt/qemu/bin/qemu-img convert -f vmdk -O qcow2 viptela-edge-genericx86-64-disk1.vmdk virtioa.qcow2 ##vEdge cd /opt/unetlab/addons/qemu/vtedge-17.1 tar -xvf viptela-edge-genericx86-64.ova /opt/qemu/bin/qemu-img convert -f vmdk -O qcow2 viptela-edge-genericx86-64-disk1.vmdk virtioa.qcow2 ##Fix permissions /opt/unetlab/wrappers/unl_wrapper -a fixpermissions
The physical topology that we are going to use is as follows. You should re-create it on EVE-NG. If you want to copy/paste some of the configs, make sure that you use the same interfaces when creating the topology on EVE-NG.
You have to decide how many vEdge devices you are going to need and add them in the same manner as vEdge-1. The process of building the lab starts with Cisco SD-WAN version 16.3.2. Once you upgrade to the higher versions you won't be able to add more vEdges.
Default credentials for all devices are admin/admin. When vManage boots for the first time, it will ask on which storage device to install the software. Please make sure to use the virtual disk you have created in the previous step.
viptela 16.3.2 vmanage login: admin Password: Welcome to Viptela CLI admin connected from 127.0.0.1 using console on vmanage Available storage devices: vda 18GB vda1 2GB vda2 16GB vdb 150GB hdc 3GB 1) vda 2) vda1 3) vda2 4) vdb 5) hdc Select storage device to use: 4 Would you like to format vdb? (y/n): y
Once all devices boot up it is time to enable basic connectivity between the controllers and all WAN edge devices. The following bootstrap snippets are the minimum required configuration in order to achieve basic connectivity.
! system host-name vBond system-ip 220.127.116.11 site-id 1 organization-name networkacademy-io vbond 10.1.1.10 local vbond-only ! vpn 0 interface ge0/0 ip address 10.1.1.10/24 no tunnel-interface no shutdown ! !
! system host-name vManage system-ip 18.104.22.168 site-id 1 organization-name networkacademy-io vbond 10.1.1.10 ! vpn 0 interface eth0 ip address 10.1.1.20/24 no shutdown ! ! vpn 512 interface eth1 ip address 192.168.115.20/24 no shutdown ! !
system host-name vSmart system-ip 22.214.171.124 site-id 1 organization-name networkacademy-io vbond 10.1.1.10 ! vpn 0 interface eth0 ip address 10.1.1.30/24 no tunnel-interface no shutdown ! !
system host-name vEdge-1 system-ip 126.96.36.199 site-id 100 organization-name networkacademy-io vbond 10.1.1.10 ! vpn 0 interface ge0/0 ip address 10.1.1.40/24 no tunnel-interface no shutdown ! !
At this point, each device should successfully ping any other in VPN0. If for whatever reason there is no reachability to one of the devices, you should not continue ahead but troubleshoot and resolve the issue.
Cisco SD-WAN Controllers can not be brought into operation unless their identity is validated by an established chain of trust. This identity validation process is intended to ensure that only trusted devices can join the SD-WAN solution while still retaining flexibility. Each controller must have a root certificate installed and a controller certificate installed and signed by a trusted CA (Certification Authority).
For creating this lab environment we are going to use the vBond controller as a Root CA.
vBond as Root CA
Configure vBond to act as a root of trust. The first step is to generate an RSA private key. Then we generate a ROOTCA.pem certificate and sign it with the ROOTCA.key private key that we have just created.
##Use vshell / viptela_cli to switch between shell and cli modes vBond# vshell vBond:~$ openssl genrsa -out ROOTCA.key 2048 Generating RSA private key, 2048 bit long modulus ........+++..............................................+++ e is 65537 (0x10001) vBond:~$ openssl req -x509 -new -nodes -key ROOTCA.key -sha256 -days 1024 \ > -subj "/C=AU/ST=NSW/L=NSW/O=networkacademy-io/CN=sdwan.lab" \ > -out ROOTCA.pem vBond:~$ ls -l total 12 -rw-r--r-- 1 admin admin 1679 Mar 23 09:41 ROOTCA.key -rw-r--r-- 1 admin admin 1285 Mar 23 09:41 ROOTCA.pem -rw-r--r-- 1 admin admin 392 Mar 23 09:37 archive_id_rsa.pub
Once that is completed, the root certificate should be installed on all other devices. It will act as a root-of-trust for all controller certificates.
# Excecute the following command in CLI mode on all devices request root-cert-chain install scp://email@example.com:/home/admin/ROOTCA.pem vpn 0 Uploading root-ca-cert-chain via VPN 0 Copying ... firstname.lastname@example.org:/home/admin/ROOTCA.pem via VPN 0 Warning: Permanently added '10.1.1.10' (ECDSA) to the list of known hosts. viptela 16.3.2 email@example.com's password: ROOTCA.pem 100% 1285 1.1MB/s 00:00 Successfully installed the root certificate chain
If that step is successful, that means that all devices will have their chain-of-trust pointing to the vBond controllers ROOTCA.pem.
At this point, each controller should have a root certificate installed. This can be checked with the following command:
vSmart# show certificate root-ca-cert Certificate: Data: Version: 3 (0x2) Serial Number: a4:f7:2a:2f:d8:64:74:98 Signature Algorithm: sha256WithRSAEncryption Issuer: C=AU, ST=NSW, L=NSW, O=networkacademy-io, CN=sdwan.lab Validity Not Before: Apr 10 09:25:18 2021 GMT Not After : Jan 29 09:25:18 2024 GMT Subject: C=AU, ST=NSW, L=NSW, O=networkacademy-io, CN=sdwan.lab Subject Public Key Info: Public Key Algorithm: rsaEncryption Public-Key: (2048 bit) ## lines omitted
Now you need to log in to the vManage GUI interface. This is done using a web browser and entering the URL https://[vManage-VPN512-IP-address]:8443. Default credentials are admin/admin.
Once logged in, you need to go to Administration > Settings and set the Organization Name to be networkacademy-io, then edit the vBond address, and set it to 10.1.1.10. Make sure that Certificate Authorization is set to Manual as shown on the screenshot below.
Once that is done, go to Configuration > Devices > Controllers > Add Controller and add both vBond (10.1.1.10) and vSmart (10.1.1.30) using the GUI.
When both controllers are added you should see something similar to the screenshot below. Note that the Certificate Status of all controllers is Not-installed.
Now you need to go to Configuration > Certificates > Controllers and Generate CSR for all controllers. When you are done, all should be in Status "CSR Generated".
At this point, each controller must have a certificate signing request (CSR) generated. Now it is time to transfer all CSRs to vBond and sign them with the Root CA Private Key.
vBond# vshell vBond:~$ scp firstname.lastname@example.org:/home/admin/vsmart_csr vSmart.csr vBond:~$ scp email@example.com:/home/admin/vmanage_csr vManage.csr vBond:~$ scp firstname.lastname@example.org:/home/admin/vbond_csr vBond.csr vBond:~$ ls -l total 28 -rw-r--r-- 1 admin admin 1679 Mar 23 09:41 ROOTCA.key -rw-r--r-- 1 admin admin 1285 Mar 23 09:41 ROOTCA.pem -rw-r--r-- 1 admin admin 392 Mar 23 09:37 archive_id_rsa.pub -rw-r--r-- 1 admin admin 1224 Mar 23 12:12 vBond.csr -rw-r--r-- 1 admin admin 1228 Mar 23 12:12 vManage.csr -rw-r--r-- 1 admin admin 1224 Mar 23 12:12 vSmart.csr -rw-r--r-- 1 root root 1224 Mar 23 12:05 vbond_csr
When you list the directory and see that all CSRs are present, you can go ahead and sign them.
vBond:~$ openssl x509 -req -in vBond.csr \ > -CA ROOTCA.pem -CAkey ROOTCA.key -CAcreateserial \ > -out vBond.crt -days 500 -sha256 Signature ok vBond:~$ openssl x509 -req -in vSmart.csr \ > -CA ROOTCA.pem -CAkey ROOTCA.key -CAcreateserial \ > -out vSmart.crt -days 500 -sha256 Signature ok vBond:~$ openssl x509 -req -in vManage.csr \ > -CA ROOTCA.pem -CAkey ROOTCA.key -CAcreateserial \ > -out vManage.crt -days 500 -sha256 Signature ok
Now you should see all controller certificates.
vBond:~$ ls -alh | grep crt -rw-r--r-- 1 admin admin 1.3K Mar 23 12:16 vBond.crt -rw-r--r-- 1 admin admin 1.3K Mar 23 12:16 vManage.crt -rw-r--r-- 1 admin admin 1.3K Mar 23 12:18 vSmart.crt
It is time to install all controller certificates via the vManage GUI. Go to Configuration > Certificates > Controllers > Install Certificate.
Now that you have all certificates (.crt) in vBond's directory, you just cat each of them and paste the output in the Install Certificate window as shown below:
vBond:~$ cat vManage.crt -----BEGIN CERTIFICATE----- MIIDpjCCAo4CCQCpxofKmIPl+TANBgkqhkiG9w0BAQsFADBZMQswCQYDVQQGEwJB VTEMMAoGA1UECAwDTlNXMQwwCgYDVQQHDANOU1cxGjAYBgNVBAoMEW5ldHdvcmth Y2FkZW15LWlvMRIwEAYDVQQDDAlzZHdhbi5sYWIwHhcNMjEwNDEwMDk0MDQwWhcN MjIwODIzMDk0MDQwWjCB0DELMAkGA1UEBhMCVVMxEzARBgNVBAgTCkNhbGlmb3Ju aWExETAPBgNVBAcTCFNhbiBKb3NlMRowGAYDVQQLExFuZXR3b3JrYWNhZGVteS1p bzEUMBIGA1UEChMLdklQdGVsYSBJbmMxQzBBBgNVBAMUOnZtYW5hZ2VfOWVmYTcz Y2MtMmIxNi00NzNhLWFjMGUtNTQyZjhjNGRhMzg5XzAudmlwdGVsYS5jb20xIjAg BgkqhkiG9w0BCQEWE3N1cHBvcnRAdmlwdGVsYS5jb20wggEiMA0GCSqGSIb3DQEB AQUAA4IBDwAwggEKAoIBAQDO4mHeqgQ/i59TRuKjmB+i70wbzvTVf606nBzf44MD qgsFY3hwBgWcu5dcOEkkI8uCkTFoQqinQtgtw5Ucd3aE0e8skLwy6W1xzcl1YtvV 88C23BSVc66A7p/dT5g7Z4UWCm4O4BER46uNMalTZKYBGGRONMv0e1VfJgAUEN9/ bLgQMwyFdawhIYq/32rEUhms3mSunO56pAjmUBWcO19FVAFBJQXdUlpqFvniMfWW K6/YXFZQ5Y9kj7TH3tLLfkZQJ62/vohBVkdwLQvm9N2XJqL/ZEa8XPRSLs56Wy0W /WbqX2gs0emSXTQN99Npq7ILX8fF+w7W+Rqcb9Pqe3cRAgMBAAEwDQYJKoZIhvcN AQELBQADggEBABssqE1Ds8bEVp5+NXDzw1gNZmU1S+j/1Affk+FVHivIXXp8tJq5 iJU8yUYFQQeXSXQHtJWeDEIGKZ5q3Q7MB/NnxqQtyDzF3ejQ69DDWwoWI2YcCz0e g5H1Lk+2wKLRXYCvFkarMDIlbMNwoehUSiJDQUbDBJsNo5oqFa2bARB5NHoLYHrS O7yU24F0qSRBR4HmZuwAbvBgTrVMUyjj0NIgprb3IqpqdSpwIxtL1o8GS10Q82sR ICO1TBbYlqPvozd7HYRRgFGmG+9VIaORKLhPoJcM7tvIdQxmdrCItYATUy2djp6E GyOO7U0MLBDbTzCkfl4huCOcbu/dWNk6zJk= -----END CERTIFICATE-----
If everything is good up to this point, the certificate should install successfully.
You repeat this for all controllers. In the end, when you go to Configuration > Certificates > Controllers, you should see that all controllers have Certificate Serial numbers. If that is the case, you click the Send-to-vBond function to propagate this information to vBond.
At this point, all controllers should be operational with valid certificates. Now it is time to validate the vEdges.
We need to generate a certificate signing request (CSR) for every vEdge. This is done using the following command in CLI mode. Note that the name of the CSR changes for each vEdge.
vEdge-1# request csr upload scp://email@example.com:/home/admin/vedge-01.csr vpn 0 vEdge-2# request csr upload scp://firstname.lastname@example.org:/home/admin/vedge-02.csr vpn 0 ... vEdge-N# request csr upload scp://email@example.com:/home/admin/vedge-nn.csr vpn 0 Uploading CSR via VPN 0 Enter organization name : networkacademy-io Re-enter organization name : networkacademy-io Generating private/public pair and CSR for this vedge device Generating CSR for this vedge device ........(DONE) Copying ... firstname.lastname@example.org:/home/admin/vedge-01.csr via VPN 0 Warning: Permanently added '10.1.1.10' (ECDSA) to the list of known hosts. viptela 16.3.2 email@example.com's password: server.csr 100% 1224 56.4KB/s 00:00 CSR upload successful
When you finish with the above, on vBond you should see all WAN Edges' CSRs.
vBond:~$ ls -alh | grep vedge -rw-r--r-- 1 admin admin 1.2K Mar 23 13:34 vedge-01.csr -rw-r--r-- 1 admin admin 1.2K Mar 23 13:34 vedge-02.csr -rw-r--r-- 1 admin admin 1.2K Mar 23 13:34 vedge-nn.csr
Now it is time to sign each one of them. Note that the certification name changes for each vEdge - vEdge-01.crt, vEdge-02.crt and so on.
vBond:~$ openssl x509 -req -in vedge-01.csr \ -CA ROOTCA.pem -CAkey ROOTCA.key -CAcreateserial \ -out vEdge-01.crt -days 500 -sha256 vBond:~$ openssl x509 -req -in vedge-02.csr \ -CA ROOTCA.pem -CAkey ROOTCA.key -CAcreateserial \ -out vEdge-02.crt -days 500 -sha256 vBond:~$ openssl x509 -req -in vedge-nn.csr \ -CA ROOTCA.pem -CAkey ROOTCA.key -CAcreateserial \ -out vEdge-nn.crt -days 500 -sha256
Once all CSRs are signed and a valid certificate is generated for each WAN edge router, it is time to install the certificates. This is done by executing the following command on each vEdge in CLI mode:
vEdge-1# request certificate install scp://firstname.lastname@example.org:/home/admin/vEdge-01.crt vpn 0 vEdge-2# request certificate install scp://email@example.com:/home/admin/vEdge-02.crt vpn 0 ... vEdge-nn# request certificate install scp://firstname.lastname@example.org:/home/admin/vEdge-nn.crt vpn 0 Installing certificate via VPN 0 Copying ... email@example.com:/home/admin/vEdge-01.crt via VPN 0 Warning: Permanently added '10.1.1.10' (ECDSA) to the list of known hosts. viptela 16.3.2 firstname.lastname@example.org's password: vEdge-01.crt 100% 1326 31.3KB/s 00:00 Successfully installed the certificate
In the end, each vEdge must have a certificate serial number.
vEdge-1# show certificate serial Chassis number: 37a459dc-9447-4f44-b6cb-07e3c3190c8a serial number: E3C382FF265B63F1 ... vEdge-N# show certificate serial Chassis number: bfba45b7-038b-476c-83b7-a19742ff6091 serial number: E3C382FF265B63F2
Now you need to copy the output of the show certificate serial of all vEdges in a Notepad file in the format shown below, and then save the file as vedges.csv.
Then you go to Configuration > Devices > vEdge List > Upload vEdge List and select the vedges.csv file. Make sure to check the "Validate the uploaded vEdge list and send to controllers" option.
If the upload is successful you should see all vEdge devices having a Chassis number and Certificate Serial as shown below.
At this point, the validation of all devices is done. We need to bring up the control plane and upgrade to the CCIE Enterprise Infrastructure lab exam's version 18.4.4.
Tunnel-Interfaces bring up
To bring up the control plane, we must enable the overlay tunnels on all devices. This is done using the following configuration:
# on vManage and vSmart ! vpn 0 interface eth0 tunnel-interface ! # on vBond and vEdges ! vpn 0 interface ge0/0 tunnel-interface encapsulation ipsec allow-service all !
Once this is applied, you should be able to see that the control plane is up on all controllers. However, the vEdges would not be able to join the overlay before we upgrade to at least version17.2.8.
vBond# show orchestrator connections PEER PEER PEER PEER PEER SITE DOMAIN PEER PRIVATE PEER PUBLIC TYPE PROTOCOL SYSTEM IP ID ID PRIVATE IP PORT PUBLIC IP PORT REMOTE COLOR STATE UPTIME -------------------------------------------------------------------------------------------------------------------------------------------------------------- vsmart dtls 188.8.131.52 1 1 10.1.1.30 12346 10.1.1.30 12346 default up 0:00:00:38 vsmart dtls 184.108.40.206 1 1 10.1.1.30 12446 10.1.1.30 12446 default up 0:00:00:37 vmanage dtls 220.127.116.11 1 0 10.1.1.20 12346 10.1.1.20 12346 default up 0:00:00:49 vmanage dtls 18.104.22.168 1 0 10.1.1.20 12446 10.1.1.20 12446 default up 0:00:00:48 vmanage dtls 22.214.171.124 1 0 10.1.1.20 12546 10.1.1.20 12546 default up 0:00:00:48 vmanage dtls 126.96.36.199 1 0 10.1.1.20 12646 10.1.1.20 12646 default up 0:00:00:48 vManage# show control connections PEER PEER PEER PEER PEER SITE DOMAIN PEER PRIV PEER PUB INDEX TYPE PROT SYSTEM IP ID ID PRIVATE IP PORT PUBLIC IP PORT REMOTE COLOR STATE UPTIME -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0 vsmart dtls 188.8.131.52 1 1 10.1.1.30 12346 10.1.1.30 12346 default up 0:00:01:03 0 vbond dtls 184.108.40.206 0 0 10.1.1.10 12346 10.1.1.10 12346 default up 0:00:01:19 1 vbond dtls 220.127.116.11 0 0 10.1.1.10 12346 10.1.1.10 12346 default up 0:00:01:18 2 vbond dtls 18.104.22.168 0 0 10.1.1.10 12346 10.1.1.10 12346 default up 0:00:01:18 3 vbond dtls 22.214.171.124 0 0 10.1.1.10 12346 10.1.1.10 12346 default up 0:00:01:19
The images required for upgrading to 17.2.8 are as follow:
The software upgrade is pretty simple and straightforward. You upload the necessary files in the Software Repository by going to Maintenance > Software Upgrade > Controller > Repository.
In there, you select Add new software and upload the files for version 17.2.8 to vManage.
Then you go to Maintenance > Software Upgrade > vManage > Upgrade, select version 17.2.8 and select Upgrade. The upgrading process is quick, the new software will be installed on vManage, but will not be activated.
The last thing that you need to do is to Activate the new version software. This is done at Maintenance > Software Upgrade > vManage > Activate. At this point, the controller will reload. The process usually takes some time. In some cases, you may need to delete the browser's cookies to log in to the new version of the GUI once it boots with the new image.
Once you log in again, you will note that the GUI's layout is more modern. At this point, vManage is upgraded. However, the other controllers are not upgraded yet. You must go to Maintenance > Software Upgrade > Controller and select all controllers. Then select the new version 17.2.8 and check the Activate and Reboot option. Then click Upgrade. The controllers will reload.
Once all controllers are upgrade and fully loaded, you must reload all vEdges in order to make them join the control plane.
Full Content Access is for Registered Users Only (it's FREE)...
- Learn any CCNA, DevNet or Network Automation topic with animated explanation.
- We focus on simplicity. Networking tutorials and examples written in simple, understandable language for beginners.