Single DMVPN Cloud & Branch Configuration with Redundant Dual ISPs

You may run into an existing DMVPN implementation with one DMVPN cloud and Front-End VRFs on all routers. A new branch site (Spoke-2 in our case) is coming up, but because of its criticality, the branch office requires dual ISPs for redundancy.

So we have:

  • One DMVPN cloud
  • A hub site with one ISP and a front-end VRF
  • Spoke-1 office with one ISP and one front-end VRF
  • A new site, Spoke-2, with two ISPs and two front-end VRFs
    • ISP-1 is on VRF_1
    • ISP-2 is on VRF_2

Since a DMVPN tunnel is statically associated with a tunnel source interface, which in this case the tunnel0 interface is associated to G0/0/0 on VRF_1, the question is, how do you automatically switch your existing and only one DMVPN tunnel (tunnel0) from one VRF to another in the event of ISP_1 going down?

Let’s find out.

Single DMVPN Branch Configuration with Redundant Dual ISPs

First, let’s take a quick look at the relevant DMVPN configuration of the hub and Spoke-1, both with a single ISP.

Advertisement

Hub Site DMVPN Configuration with One ISP & Front-End VRF

Here’s the relevant configuration for the hub DMVPN router.

crypto isakmp policy 10
 encr 256-aes
 authentication pre-share
 group 2
 exit

ip vrf ISP_A
 rd 1:1
 exit

crypto keyring DMVPN_KEY_1 vrf ISP_A
 pre-shared-key address 0.0.0.0 0.0.0.0 key 1$P1key123
 exit

crypto ipsec transform-set AES256_SHA512 esp-256-aes esp-sha512-hmac
 mode transport
 exit

crypto ipsec profile DMVPN_PROFILE
 set transform-set AES256_SHA512
 exit

interface GigabitEthernet0/0/0
 description ISP A
 ip vrf forwarding ISP_A ! Move this interface G0/0/0 from default VRF to ISP_A
 ip address 1.1.1.2 255.255.255.252
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 duplex full ! I recommend statically setting your duplex
 speed 1000 ! I recommend statically setting your speed
 no shutdown
 exit

ip route vrf ISP_A 0.0.0.0 0.0.0.0 1.1.1.1 name ISP_A_DEFAULT

interface Tunnel0
 description DMVPN HUB
 ip address 192.168.123.1 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip tcp adjust-mss 1360
 ip nhrp map multicast dynamic ! 
 ip nhrp network-id 1
 ip nhrp authentication s3cur3KEY
 ip nhrp holdtime 300
 ip nhrp redirect
 tunnel vrf ISP_A ! VRF where the source interface G0/0/0 is at
 tunnel source GigabitEthernet0/0/0
 tunnel mode gre multipoint
 tunnel key 1
 tunnel protection ipsec profile DMVPN_PROFILE
 exit

Again, notice that under tunnel0, the tunnel source G0/0/0 can be found under VRF ISP_A.

IMPORTANT: Although tunnel0’s source interface is found under a VRF, tunnel0 itself STILL belongs to the default VRF. VRF_A is only used as transport. This means that any routes received from a routing neighbor on tunnel0 will be placed on the default routing table and not on VRF_A’s routing table.

Advertisement

Spoke-1 DMVPN Configuration with One ISP & Front-End VRF

Now, let’s switch to the spoke with one ISP. Here’s the relevant DMVPN configuration.

crypto isakmp policy 10
 encr 256-aes
 authentication pre-share
 group 2
 exit

ip vrf ISP_B
 rd 1:1
 exit

crypto keyring DMVPN_KEY_1 vrf ISP_B
 pre-shared-key address 0.0.0.0 0.0.0.0 key 1$P1key123
 exit

crypto ipsec transform-set AES256_SHA512 esp-256-aes esp-sha512-hmac
 mode transport
 exit

crypto ipsec profile DMVPN_PROFILE
 set transform-set AES256_SHA512
 exit

interface GigabitEthernet0/0/0
 description ISP B
 ip vrf forwarding ISP_B
 ip address 2.1.1.2 255.255.255.252
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 duplex full
 speed 1000
 no shutdown
 exit

ip route vrf ISP_B 0.0.0.0 0.0.0.0 2.1.1.1 name ISP_A_DEFAULT

interface Tunnel0
 description DMVPN SPOKE
 ip address 192.168.123.2 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip tcp adjust-mss 1360
 ip nhrp map 192.168.123.1 1.1.1.2
 ip nhrp map multicast 1.1.1.2
 ip nhrp network-id 1
 ip nhrp holdtime 300
 ip nhrp nhs 192.168.123.1
 ip nhrp shortcut
 tunnel vrf ISP_B ! VRF where G0/0/0 is part of
 tunnel source GigabitEthernet0/0/0
 tunnel mode gre multipoint
 tunnel key 1
 tunnel protection ipsec profile DMVPN_PROFILE
 exit

With the ip nhrp map commands, this branch router, or spoke, creates a static tunnel to the hub where the NHRP server is. This tunnel, tunnel0, uses G0/0/0 as the source and is assigned to VRF ISP_B.

Advertisement

Spoke-2 DMVPN Configuration with Two ISPs & Front-End VRFs

Now, Spoke-2 presents a similar configuration.

crypto isakmp policy 10
 encr 256-aes
 authentication pre-share
 group 2
 exit

ip vrf ISP_1
 rd 1:1
 exit

ip vrf ISP_2
 rd 1:1
 exit

crypto keyring DMVPN_KEY_1 vrf ISP_1
 pre-shared-key address 0.0.0.0 0.0.0.0 key 1$P1key123
 exit

crypto keyring DMVPN_KEY_2 vrf ISP_2
 pre-shared-key address 0.0.0.0 0.0.0.0 key 1$P1key123
 exit

crypto ipsec transform-set AES256_SHA512 esp-256-aes esp-sha512-hmac
 mode transport
 exit

crypto ipsec profile DMVPN_PROFILE
 set transform-set AES256_SHA512
 exit

interface GigabitEthernet0/0/0
 description ISP 1
 ip vrf forwarding ISP_1
 ip address 3.1.1.2 255.255.255.252
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 duplex full
 speed 1000
 no shut
 exit

ip route vrf ISP_1 0.0.0.0 0.0.0.0 3.1.1.1 name ISP_1_DEFAULT

interface GigabitEthernet0/0/2
 description ISP 2
 ip vrf forwarding ISP_2
 ip address 3.2.2.2 255.255.255.252
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 duplex full
 speed 1000
 no shut
 exit

ip route vrf ISP_2 0.0.0.0 0.0.0.0 3.2.2.1 name ISP_2_DEFAULT

interface Tunnel0
 description DMVPN SPOKE
 ip address 192.168.123.3 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip tcp adjust-mss 1360
 ip nhrp map 192.168.123.1 1.1.1.2
 ip nhrp map multicast 1.1.1.2
 ip nhrp network-id 1
 ip nhrp holdtime 300
 ip nhrp nhs 192.168.123.1
 ip nhrp shortcut
 tunnel vrf ISP_1 ! VRF where G0/0/0 can be found
 tunnel source GigabitEthernet0/0/0
 tunnel mode gre multipoint
 tunnel key 1
 tunnel protection ipsec profile DMVPN_PROFILE
 exit

As you can see, tunnel0’s source interface is G0/0/0, which has been assigned to VRF ISP_1. In the event of ISP_1 going down, how can the router automatically reassign tunnel0 to ISP_2?

So we have two questions to answer:

  • How can the router detect that Internet access over ISP_1 is down?; and
  • How can the router automatically switch the tunnel source over to ISP_2?

Let’s take a look at how can we answer these two questions.

How can the DMVPN spoke router detect Internet service loss over the primary ISP?

To solve this problem, I suggest using VRF-aware IP SLA with Boolean OR Object Tracking. Here’s a sample configuration.

ip sla 1
 icmp-echo 8.8.8.8 source-interface GigabitEthernet0/0/0
  vrf ISP_1
  threshold 1999
  timeout 1999
  exit
ip sla schedule 1 start-time now life forever

ip sla 2
 icmp-echo 4.2.2.2 source-interface GigabitEthernet0/0/0
  vrf ISP_2
  threshold 1999
  timeout 1999
  exit
ip sla schedule 2 start-time now life forever

track 1 ip sla 1 reachability
 exit
track 2 ip sla 2 reachability
 exit

track 3 list boolean OR
 object 1
 object 2
 exit

When Spoke-3 loses IP reachability to Google DNS 8.8.8.8 “and” Level3 DNS 4.2.2.2, normally, that’s a good indication that Internet access has been lost over the primary ISP (ISP_1 in this case).

  • If IP SLA 1 stops receiving ping replies, track 1 goes down.
  • If IP SLA 2 stops receiving ping replies, track 2 goes down.
  • If both track 1 and track 2 go down, track 3 goes down.

If track 1 goes down but track 2 stays up, track 3 stays up; likewise, if track 2 goes down but track 1 stays up, track 3 also stays up. Track 3 goes down when both, track 1 and track 2, go down. That’s how Boolean OR works.

So, if track 3 goes down, that’s a reliable sign that there’s no IP reachability to always-up hosts on the Internet.

How can the Cisco router change the DMVPN Tunnel0 from one VRF to another?

The Spoke-3 router needs to automatically change its tunnel source from VRF ISP_1 to ISP_2 and from GigabitEthernet0/0/0 to GigabitEthernet0/0/2. Remember, G0/0/2 was assigned to ISP_2.

To accomplish this reassignment, we’re going to make use of Embedded Event Manager or EEM. And this is how it goes.

When track 3 goes down, a syslog message is generated.

TRACK-6-STATE: 3 list boolean or Up -> Down

We’re going to create an EEM script that will monitor for this up-to-down syslog message and execute the commands necessary to reconfigure tunnel0 and reassign its source interface to ISP_2.

event manager applet FAILOVER
 event syslog pattern "TRACK-6-STATE: 3 list boolean or Up -> Down"
 action 01.0 cli command "enable"
 action 02.0 cli command "config term"
 action 03.0 cli command "interface tunnel0"
 action 04.0 cli command "shutdown"
 action 05.0 cli command "no tunnel vrf ISP_1"
 action 06.0 cli command "tunnel vrf ISP_2"
 action 07.0 cli command "no tunnel source g0/0/0"
 action 08.0 cli command "tunnel source g0/0/2"
 action 09.0 cli command "no shutdown"
 action 10.0 cli command "end"
 action 11.0 cli command "write mem"
 exit

And, when track 3 comes back up, a syslog message is also generated.

TRACK-6-STATE: 3 list boolean or Down -> Up

We also are going to create another EEM script to monitor for this down-to-up syslog message and execute the commands necessary to roll back the changes.

event manager applet FAILBACK
 event syslog pattern "TRACK-6-STATE: 3 list boolean or Down -> Up"
 action 01.0 cli command "enable"
 action 02.0 cli command "config term"
 action 03.0 cli command "interface tunnel0"
 action 04.0 cli command "shutdown"
 action 05.0 cli command "no tunnel vrf ISP_2"
 action 06.0 cli command "tunnel vrf ISP_1"
 action 07.0 cli command "no tunnel source g0/0/2"
 action 08.0 cli command "tunnel source g0/0/0"
 action 09.0 cli command "no shutdown"
 action 10.0 cli command "end"
 action 11.0 cli command "write mem"
 exit

I have deployed this configuration and similar configurations successfully at customer sites, and I hope they’re helpful to you as well.

I hope you enjoyed this post. If you have any questions, please leave a comment below and I’ll respond as soon as I can.

Thank you.

Getting CCNA or CCNP Certified?

Self-paced Books. On-demand Courses. Practice Tests.

Sign up for a 10-day free trial with unlimited access!

Alirio Zavarce Faceshot

ABOUT THE AUTHOR

Alirio Zavarce, CCIE #28672, is a seasoned enterprise route-switch consultant with 30 years of experience with data networks. Alirio started this networking blog to help his peers become better network engineers and share all his everyday experiences and troubleshooting tips. More about me...

If Alirio had to prepare to take it again, here's what he would do to pass the CCIE lab.

Please Share

2 thoughts on “Single DMVPN Cloud & Branch Configuration with Redundant Dual ISPs”

    • Hi Sabir. Are you talking about when I said “I have deployed this configuration and similar configurations successfully at customer sites…” at the end of the article?

      Reply

Leave a Comment

Share to...