Some of the widespread causes of Web routing leaks is an undereducated end-customer configuring EBGP periods with two (or extra) upstream ISPs.
With out basic-level BGP information or additional steerage from the service suppliers, the shopper community engineer may begin a BGP routing course of and configure two EBGP periods, much like the next industry-standard CLI configuration:
Typical end-customer BGP configuration copied from the primary hit on Google
router bgp 65100
neighbor 10.1.0.2 remote-as 65001
neighbor 10.1.0.6 remote-as 65002
The outcomes of such a simplistic configuration are too well-known: the shopper begins leaking routes between the 2 service suppliers, and if one in every of them occurs to be Verizon, now we have an Web-wide meltdown.
I replicated that setup in a netlab lab utilizing the next topology:
Lab topology
The netlab topology file is so simple as it will probably get:
- We’re utilizing containerlab supplier to scale back the lab startup occasions.
- All lab gadgets are operating BGP
- We’re setting specific BGP AS numbers on each machine
- There are two hyperlinks within the lab (from buyer to ISP-A and ISP-B)
Netlab topology file
supplier: clab
module: [ bgp ]
defaults.machine: eos
nodes:
buyer:
bgp.as: 65100
isp_a:
bgp.as: 65001
isp_b:
bgp.as: 65002
hyperlinks: [ customer-isp_a, customer-isp_b ]
After I began the lab with Arista EOS gadgets, the route leaks have been straightforward to look at:
Buyer is leaking routes between ISP-A and ISP-B
isp-a>sh ip bgp
BGP routing desk info for VRF default
Router identifier 10.0.0.2, native AS quantity 65001
Route standing codes: s - suppressed contributor, * - legitimate, > - lively, E - ECMP head, e - ECMP
S - Stale, c - Contributing to ECMP, b - backup, L - labeled-unicast
% - Pending BGP convergence
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI Origin Validation codes: V - legitimate, I - invalid, U - unknown
AS Path Attributes: Or-ID - Originator ID, C-LST - Cluster Record, LL Nexthop - Hyperlink Native Nexthop
Community Subsequent Hop Metric AIGP LocPref Weight Path
* > 10.0.0.1/32 10.1.0.1 0 - 100 0 65100 i
* > 10.0.0.2/32 - - - - 0 i
* > 10.0.0.3/32 10.1.0.1 0 - 100 0 65100 65002 i
Subsequent, I modified the shopper router to FRR utilizing conventional defaults. I had to make use of a customized configuration template as a result of the newer FRR variations ship with datacenter defaults.
The customized configuration template configured the standard defaults, eliminated the BGP routing course of (issues get bizarre when you change the defaults after you’ve already configured BGP), and recreated the BGP configuration:
Customized FRR BGP configuration template
frr defaults conventional
!
no router bgp {{ bgp.as }}
!
router bgp {{ bgp.as }}
{% for af in ['ipv4'] %}
{% for n in bgp.neighbors if n[af] is outlined %}
neighbor {{ n[af] }} remote-as {{ n.as }}
neighbor {{ n[af] }} description {{ n.title }}
{% endfor %}
{% if loopback[af] is outlined and bgp.advertise_loopback %}
community {ipaddr(0) }
{% endif %}
{% endfor %}
Excellent news: the route leaks disappeared.
Unhealthy information: the shopper router refused to just accept ISP prefixes and stopped asserting its routes to the 2 ISPs – it was time for one more troubleshooting session.
The BGP desk on the shopper router had no ISP routes. It was just like the router failed to ascertain the BGP periods.
BGP desk on the shopper router
buyer# sh ip bgp
BGP desk model is 1, native router ID is 192.168.121.101, vrf id 0
Default native pref 100, native AS 65100
Standing codes: s suppressed, d damped, h historical past, * legitimate, > greatest, = multipath,
i inside, r RIB-failure, S Stale, R Eliminated
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V legitimate, I invalid, N Not discovered
Community Subsequent Hop Metric LocPrf Weight Path
*> 10.0.0.1/32 0.0.0.0 0 32768 i
Displayed 1 routes and 1 whole paths
The BGP abstract printout was complicated. It contained (Coverage) within the column the place one would count on to see the variety of prefixes obtained from or despatched to a neighbor.
BGP abstract printout on the shopper router
buyer# present bgp abstract
IPv4 Unicast Abstract (VRF default):
BGP router identifier 192.168.121.101, native AS quantity 65100 vrf-id 0
BGP desk model 1
RIB entries 1, utilizing 192 bytes of reminiscence
Friends 2, utilizing 1434 KiB of reminiscence
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc
10.1.0.2 4 65001 8 5 0 0 0 00:02:44 (Coverage) (Coverage) isp_a
10.1.0.6 4 65002 8 5 0 0 0 00:02:44 (Coverage) (Coverage) isp_b
Whole variety of neighbors 2
FRR documentation isn’t as intensive as what I’m used to. For instance, the description of present bgp abstract printout accommodates no point out of the (Coverage) worth. I don’t know how somebody unfamiliar with FRR would determine what’s happening; I already knew that FRR implements RFC 8212, a beautiful RFC that might cease most fat-finger leaks as soon as carried out by main distributors.
RFC 8212 makes two small modifications to the BGP route choice course of:
- The BGP route choice course of shouldn’t think about routes obtained from an EBGP neighbor with out an specific import coverage.
- A router shouldn’t promote its greatest routes to an EBGP neighbor with out an specific export coverage.
FRR enforces RFC 8212 guidelines when utilizing the conventional profile: ignoring all routes obtained from its EBGP neighbors and never sending any routes to them except you will have configured specific neighbor insurance policies.
With a little bit of soul-searching, I used to be capable of finding the instructions to configure AS-path entry lists on FRR, ensuing within the following BGP configuration:
Last FRR BGP configuration
router bgp 65100
neighbor 10.1.0.2 remote-as 65001
neighbor 10.1.0.2 description isp_a
neighbor 10.1.0.6 remote-as 65002
neighbor 10.1.0.6 description isp_b
!
address-family ipv4 unicast
community 10.0.0.1/32
neighbor 10.1.0.2 filter-list the whole lot in
neighbor 10.1.0.2 filter-list own-as out
neighbor 10.1.0.6 filter-list the whole lot in
neighbor 10.1.0.6 filter-list own-as out
exit-address-family
exit
!
bgp as-path access-list the whole lot seq 5 allow .*
bgp as-path access-list own-as seq 5 allow ^$
Finish consequence: the shopper is asserting its BGP prefixes to the 2 ISPs, however not leaking routes between them:
Last BGP desk on ISP-A
isp-a>present ip bgp
BGP routing desk info for VRF default
Router identifier 10.0.0.2, native AS quantity 65001
Route standing codes: s - suppressed contributor, * - legitimate, > - lively, E - ECMP head, e - ECMP
S - Stale, c - Contributing to ECMP, b - backup, L - labeled-unicast
% - Pending BGP convergence
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI Origin Validation codes: V - legitimate, I - invalid, U - unknown
AS Path Attributes: Or-ID - Originator ID, C-LST - Cluster Record, LL Nexthop - Hyperlink Native Nexthop
Community Subsequent Hop Metric AIGP LocPref Weight Path
* > 10.0.0.1/32 10.1.0.1 0 - 100 0 65100 i
* > 10.0.0.2/32 - - - - 0 i
Clearly, you might use related configuration instructions on Arista EOS and get the identical outcomes. Nonetheless, you’re not pressured to take action (however then FRR modified its default conduct away from RFC8212 in latest launch as properly), which brings us to the apparent query: which distributors assist RFC8212 with the default settings?
The simplest strategy to discover the reply is to search for Job Snijders’ RFC 8212 compliance monitoring GitHub repository: the one business implementations compliant with RFC 8212 out-of-the-box are Cisco IOS XR and Nokia SR OS/SR Linux. You’ll be able to configure Arista EOS, Cisco IOS XE, and Junos to be RFC8212-compliant, however that’s not their default conduct, An end-customer deploying BGP on Cisco IOS XE or Junos router for the primary time would nonetheless be capable to unknowingly leak routes between upstream ISPs.
There are no less than three causes I can see for the seller reluctance to implement RFC 8212:
- RFC8212 compliance completely destroys the “utilizing EBGP as a greater IGP is easy” fable. That’s why FRR makes use of conventional and datacenter profiles with datacenter profile being the default.
- Help prices. Altering the default conduct ends in clients opening assist circumstances when the 20-year-old cheatsheet they acquired as the primary Google hit doesn’t work.
- Backward compatibility. Upgrading the software program would break some networks. Distributors have been coping with the identical downside up to now (for instance: migrating to BGP handle households on Cisco IOS), so it’s not like the answer is a superb unknown. In fact, it takes some effort to implement and doc it.
And that’s why we nonetheless see tons of buyer route leaks greater than half a decade after RFC 8212 was printed.
You’ll discover extra BGP route leak horror tales within the Web Routing Safety a part of the Community Safety Fallacies part of the How Networks Actually Work webinar. For a deep dive, watch the Web Routing Safety webinar.
