Single DMVPN Cloud & Branch Configuration with Redundant Dual ISPs

You may run into an existing DMVPN implementation with one DMVPN cloud and Front-End VRFs on all routers. A new branch site (Spoke-2 in our case) is coming up, but because of its criticality, the branch office requires dual ISPs for redundancy.

So we have:

• One DMVPN cloud
• A hub site with one ISP and a front-end VRF
• Spoke-1 office with one ISP and one front-end VRF
• A new site, Spoke-2, with two ISPs and two front-end VRFs
  • ISP-1 is on VRF_1
  • ISP-2 is on VRF_2

Since a DMVPN tunnel is statically associated with a tunnel source interface, which in this case the tunnel0 interface is associated to G0/0/0 on VRF_1, the question is, how do you automatically switch your existing and only one DMVPN tunnel (tunnel0) from one VRF to another in the event of ISP_1 going down?

Let’s find out.

First, let’s take a quick look at the relevant DMVPN configuration of the hub and Spoke-1, both with a single ISP.

Hub Site DMVPN Configuration with One ISP & Front-End VRF

Here’s the relevant configuration for the hub DMVPN router.

crypto isakmp policy 10
 encr 256-aes
 authentication pre-share
 group 2
 exit

ip vrf ISP_A
 rd 1:1
 exit

crypto keyring DMVPN_KEY_1 vrf ISP_A
 pre-shared-key address 0.0.0.0 0.0.0.0 key 1$P1key123
 exit

crypto ipsec transform-set AES256_SHA512 esp-256-aes esp-sha512-hmac
 mode transport
 exit

crypto ipsec profile DMVPN_PROFILE
 set transform-set AES256_SHA512
 exit

interface GigabitEthernet0/0/0
 description ISP A
 ip vrf forwarding ISP_A ! Move this interface G0/0/0 from default VRF to ISP_A
 ip address 1.1.1.2 255.255.255.252
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 duplex full ! I recommend statically setting your duplex
 speed 1000 ! I recommend statically setting your speed
 no shutdown
 exit

ip route vrf ISP_A 0.0.0.0 0.0.0.0 1.1.1.1 name ISP_A_DEFAULT

interface Tunnel0
 description DMVPN HUB
 ip address 192.168.123.1 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip tcp adjust-mss 1360
 ip nhrp map multicast dynamic ! 
 ip nhrp network-id 1
 ip nhrp authentication s3cur3KEY
 ip nhrp holdtime 300
 ip nhrp redirect
 tunnel vrf ISP_A ! VRF where the source interface G0/0/0 is at
 tunnel source GigabitEthernet0/0/0
 tunnel mode gre multipoint
 tunnel key 1
 tunnel protection ipsec profile DMVPN_PROFILE
 exit

Again, notice that under tunnel0, the tunnel source G0/0/0 can be found under VRF ISP_A.

IMPORTANT: Although tunnel0’s source interface is found under a VRF, tunnel0 itself STILL belongs to the default VRF. VRF_A is only used as transport. This means that any routes received from a routing neighbor on tunnel0 will be placed on the default routing table and not on VRF_A’s routing table.

Spoke-1 DMVPN Configuration with One ISP & Front-End VRF

Now, let’s switch to the spoke with one ISP. Here’s the relevant DMVPN configuration.

crypto isakmp policy 10
 encr 256-aes
 authentication pre-share
 group 2
 exit

ip vrf ISP_B
 rd 1:1
 exit

crypto keyring DMVPN_KEY_1 vrf ISP_B
 pre-shared-key address 0.0.0.0 0.0.0.0 key 1$P1key123
 exit

crypto ipsec transform-set AES256_SHA512 esp-256-aes esp-sha512-hmac
 mode transport
 exit

crypto ipsec profile DMVPN_PROFILE
 set transform-set AES256_SHA512
 exit

interface GigabitEthernet0/0/0
 description ISP B
 ip vrf forwarding ISP_B
 ip address 2.1.1.2 255.255.255.252
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 duplex full
 speed 1000
 no shutdown
 exit

ip route vrf ISP_B 0.0.0.0 0.0.0.0 2.1.1.1 name ISP_A_DEFAULT

interface Tunnel0
 description DMVPN SPOKE
 ip address 192.168.123.2 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip tcp adjust-mss 1360
 ip nhrp map 192.168.123.1 1.1.1.2
 ip nhrp map multicast 1.1.1.2
 ip nhrp network-id 1
 ip nhrp holdtime 300
 ip nhrp nhs 192.168.123.1
 ip nhrp shortcut
 tunnel vrf ISP_B ! VRF where G0/0/0 is part of
 tunnel source GigabitEthernet0/0/0
 tunnel mode gre multipoint
 tunnel key 1
 tunnel protection ipsec profile DMVPN_PROFILE
 exit

With the ip nhrp map commands, this branch router, or spoke, creates a static tunnel to the hub where the NHRP server is. This tunnel, tunnel0, uses G0/0/0 as the source and is assigned to VRF ISP_B.

Spoke-2 DMVPN Configuration with Two ISPs & Front-End VRFs

Now, Spoke-2 presents a similar configuration.

crypto isakmp policy 10
 encr 256-aes
 authentication pre-share
 group 2
 exit

ip vrf ISP_1
 rd 1:1
 exit

ip vrf ISP_2
 rd 1:1
 exit

crypto keyring DMVPN_KEY_1 vrf ISP_1
 pre-shared-key address 0.0.0.0 0.0.0.0 key 1$P1key123
 exit

crypto keyring DMVPN_KEY_2 vrf ISP_2
 pre-shared-key address 0.0.0.0 0.0.0.0 key 1$P1key123
 exit

crypto ipsec transform-set AES256_SHA512 esp-256-aes esp-sha512-hmac
 mode transport
 exit

crypto ipsec profile DMVPN_PROFILE
 set transform-set AES256_SHA512
 exit

interface GigabitEthernet0/0/0
 description ISP 1
 ip vrf forwarding ISP_1
 ip address 3.1.1.2 255.255.255.252
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 duplex full
 speed 1000
 no shut
 exit

ip route vrf ISP_1 0.0.0.0 0.0.0.0 3.1.1.1 name ISP_1_DEFAULT

interface GigabitEthernet0/0/2
 description ISP 2
 ip vrf forwarding ISP_2
 ip address 3.2.2.2 255.255.255.252
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 duplex full
 speed 1000
 no shut
 exit

ip route vrf ISP_2 0.0.0.0 0.0.0.0 3.2.2.1 name ISP_2_DEFAULT

interface Tunnel0
 description DMVPN SPOKE
 ip address 192.168.123.3 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip tcp adjust-mss 1360
 ip nhrp map 192.168.123.1 1.1.1.2
 ip nhrp map multicast 1.1.1.2
 ip nhrp network-id 1
 ip nhrp holdtime 300
 ip nhrp nhs 192.168.123.1
 ip nhrp shortcut
 tunnel vrf ISP_1 ! VRF where G0/0/0 can be found
 tunnel source GigabitEthernet0/0/0
 tunnel mode gre multipoint
 tunnel key 1
 tunnel protection ipsec profile DMVPN_PROFILE
 exit

As you can see, tunnel0’s source interface is G0/0/0, which has been assigned to VRF ISP_1. In the event of ISP_1 going down, how can the router automatically reassign tunnel0 to ISP_2?

So we have two questions to answer:

• How can the router detect that Internet access over ISP_1 is down?; and
• How can the router automatically switch the tunnel source over to ISP_2?

Let’s take a look at how can we answer these two questions.

How can the DMVPN spoke router detect Internet service loss over the primary ISP?

To solve this problem, I suggest using VRF-aware IP SLA with Boolean OR Object Tracking. Here’s a sample configuration.

ip sla 1
 icmp-echo 8.8.8.8 source-interface GigabitEthernet0/0/0
  vrf ISP_1
  threshold 1999
  timeout 1999
  exit
ip sla schedule 1 start-time now life forever

ip sla 2
 icmp-echo 4.2.2.2 source-interface GigabitEthernet0/0/0
  vrf ISP_2
  threshold 1999
  timeout 1999
  exit
ip sla schedule 2 start-time now life forever

track 1 ip sla 1 reachability
 exit
track 2 ip sla 2 reachability
 exit

track 3 list boolean OR
 object 1
 object 2
 exit

When Spoke-3 loses IP reachability to Google DNS 8.8.8.8 “and” Level3 DNS 4.2.2.2, normally, that’s a good indication that Internet access has been lost over the primary ISP (ISP_1 in this case).

• If IP SLA 1 stops receiving ping replies, track 1 goes down.
• If IP SLA 2 stops receiving ping replies, track 2 goes down.
• If both track 1 and track 2 go down, track 3 goes down.

If track 1 goes down but track 2 stays up, track 3 stays up; likewise, if track 2 goes down but track 1 stays up, track 3 also stays up. Track 3 goes down when both, track 1 and track 2, go down. That’s how Boolean OR works.

So, if track 3 goes down, that’s a reliable sign that there’s no IP reachability to always-up hosts on the Internet.

How can the Cisco router change the DMVPN Tunnel0 from one VRF to another?

The Spoke-3 router needs to automatically change its tunnel source from VRF ISP_1 to ISP_2 and from GigabitEthernet0/0/0 to GigabitEthernet0/0/2. Remember, G0/0/2 was assigned to ISP_2.

To accomplish this reassignment, we’re going to make use of Embedded Event Manager or EEM. And this is how it goes.

When track 3 goes down, a syslog message is generated.

TRACK-6-STATE: 3 list boolean or Up -> Down

We’re going to create an EEM script that will monitor for this up-to-down syslog message and execute the commands necessary to reconfigure tunnel0 and reassign its source interface to ISP_2.

event manager applet FAILOVER
 event syslog pattern "TRACK-6-STATE: 3 list boolean or Up -> Down"
 action 01.0 cli command "enable"
 action 02.0 cli command "config term"
 action 03.0 cli command "interface tunnel0"
 action 04.0 cli command "shutdown"
 action 05.0 cli command "no tunnel vrf ISP_1"
 action 06.0 cli command "tunnel vrf ISP_2"
 action 07.0 cli command "no tunnel source g0/0/0"
 action 08.0 cli command "tunnel source g0/0/2"
 action 09.0 cli command "no shutdown"
 action 10.0 cli command "end"
 action 11.0 cli command "write mem"
 exit

And, when track 3 comes back up, a syslog message is also generated.

TRACK-6-STATE: 3 list boolean or Down -> Up

We also are going to create another EEM script to monitor for this down-to-up syslog message and execute the commands necessary to roll back the changes.

event manager applet FAILBACK
 event syslog pattern "TRACK-6-STATE: 3 list boolean or Down -> Up"
 action 01.0 cli command "enable"
 action 02.0 cli command "config term"
 action 03.0 cli command "interface tunnel0"
 action 04.0 cli command "shutdown"
 action 05.0 cli command "no tunnel vrf ISP_2"
 action 06.0 cli command "tunnel vrf ISP_1"
 action 07.0 cli command "no tunnel source g0/0/2"
 action 08.0 cli command "tunnel source g0/0/0"
 action 09.0 cli command "no shutdown"
 action 10.0 cli command "end"
 action 11.0 cli command "write mem"
 exit

I have deployed this configuration and similar configurations successfully at customer sites, and I hope they’re helpful to you as well.

I hope you enjoyed this post. If you have any questions, please leave a comment below and I’ll respond as soon as I can.

Thank you.