I began one in every of my VXLAN assessments with a easy setup – two switches connecting two hosts over a VXLAN-enabled (grey tunnel) pink VLAN. The switches are related with a single blue hyperlink.
Check lab
I configured VLANs and VXLANs, and began OSPF on S1 and S2 to get connectivity between their loopback interfaces. Right here’s the configuration of one of many Arista cEOS switches:
Related configuration on S1
vlan 1000
identify pink
!
interface Ethernet1
switchport entry vlan 1000
!
interface Ethernet2
description s1 -> s2
no switchport
ip tackle 10.1.0.1/30
ip ospf value 20
!
interface Loopback0
ip tackle 10.0.0.1/32
!
interface Vlan1000
description VLAN pink (1000) -> [h1,h2,s2]
ip tackle 172.16.0.1/24
!
interface Vxlan1
vxlan source-interface Loopback0
vxlan udp-port 4789
vxlan vlan 1000 vni 101000
vxlan vlan 1000 flood vtep 10.0.0.2
!
router ospf 1
community 0.0.0.0/0 space 0.0.0.0
max-lsa 12000
log-adjacency-changes element
Every part regarded superb, OSPF adjacencies the place there, however the hosts weren’t in a position to ping one another.
Attempt to determine what the issue is earlier than transferring on.
Gave up? Subsequent trace: the switches would periodically generate bizarre OSPF messages – it regarded like an adjacency would day out and be instantly reestablished. That bizarre occasion occurred nearly precisely each 40 seconds.
OSPF logging messages on S1
Oct 22 11:49:57 s1 Ospf: Occasion 1: %OSPF-4-OSPF_ADJACENCY_TEARDOWN: NGB 10.0.0.2, interface 172.16.0.1 adjacency dropped: inactivity timer expired, state was: FULL
Oct 22 11:49:57 s1 Ospf: Occasion 1: %OSPF-4-OSPF_STATE_CHANGE: NGB 10.0.0.2, intf 172.16.0.2 change from <FULL> to <DOWN>
Oct 22 11:50:07 s1 Ospf: Occasion 1: %OSPF-4-OSPF_STATE_CHANGE: NGB 10.0.0.2, intf 172.16.0.2 change from <DOWN> to <DOWN>
Oct 22 11:50:07 s1 Ospf: Occasion 1: %OSPF-4-OSPF_STATE_CHANGE: NGB 10.0.0.2, intf 172.16.0.2 change from <DOWN> to <INIT>
Oct 22 11:50:07 s1 Ospf: Occasion 1: %OSPF-4-OSPF_STATE_CHANGE: NGB 10.0.0.2, intf 172.16.0.2 change from <INIT> to <2 WAYS>
Oct 22 11:50:07 s1 Ospf: Occasion 1: %OSPF-4-OSPF_STATE_CHANGE: NGB 10.0.0.2, intf 172.16.0.2 change from <2 WAYS> to <EXCH START>
Oct 22 11:50:12 s1 Ospf: Occasion 1: %OSPF-4-OSPF_STATE_CHANGE: NGB 10.0.0.2, intf 172.16.0.2 change from <EXCH START> to <EXCHANGE>
Oct 22 11:50:12 s1 Ospf: Occasion 1: %OSPF-4-OSPF_STATE_CHANGE: NGB 10.0.0.2, intf 172.16.0.2 change from <EXCHANGE> to <FULL>
Oct 22 11:50:12 s1 Ospf: Occasion 1: %OSPF-4-OSPF_ADJACENCY_ESTABLISHED: NGB 10.0.0.2, interface 172.16.0.1 adjacency established
Figured it out? Not but? OK, right here we go…
Simply In Case You Gave Up 😉
A few of us realized the lesson the laborious manner when constructing networks transporting bizarre protocols over GRE tunnels: operating the identical routing protocol within the underlay and over the tunnel won’t be the most effective concept you ever had.
VXLAN is a tunnel like some other tunnel and if you run the identical routing protocol throughout the bodily community and on a VXLAN-enabled VLAN interface you generally get the outcomes you deserve. Simply since you’re working with information middle switches as a substitute of with conventional routers doesn’t imply you could ignore the realities of routing.
In my lab, S1:
- Established OSPF periods with S2 over bodily interfaces and VLAN interfaces
- Discovered that it’s greatest to make use of the VLAN interfaces to ship the visitors to S2 loopback IP tackle.
- Tried to encapsulate VLAN visitors into VXLAN and ship that VXLAN packet over VLAN.
Because of the delay between preliminary database change and the SPF run, the OSPF adjacency received utterly fashioned earlier than the routing tables had been tousled, and the routers solely gave up after the useless interval.
As soon as the OSPF adjacency was gone, VXLAN visitors began flowing over the bodily interface, VXLAN-encapsulated OSPF hellos (for Vlan1000 interface) had been efficiently exchanged between S1 and S2, and the entire community received bricked for an additional 40 seconds.
Options
Listed below are a couple of concepts:
- It may be a foul concept to make use of community 0.0.0.0/0 space 0 OSPF router configuration command if you wish to have any management over what’s occurring in your community. Enabling OSPF on particular person interfaces offers you higher management.
- You could possibly allow OSPF simply on the bodily interfaces and redistribute related routes into OSPF.
- You could possibly additionally allow OSPF in all places (to keep away from route redistribution) and make VLAN interfaces passive.
Nonetheless, by far the most effective answer (if it matches into your design) is to make use of VRFs for VLAN interfaces. That achieves clear separation between overlay (tenant) visitors and underlay (transport) visitors. It additionally makes it inconceivable to mess up the transport community with tenant routing data, which is successfully what we did.
Construct Your Personal
Need to repeat the experiment?
The netlab topology recordsdata are in my GitHub repository:
