• networking
• DynamicRouting
• BGP

Border Gateway Protocol (BGP)

Subpages

About BGP

• Wiki EN - Border Gateway Protocol

• Wiki DE - Border Gateway Protocol

• Wiki EN - Path-Vector routing protocol

  • In general distance-path routing protocols (RPs) have higher convergency times in comparison to Link-State RPs (like OSPF).
• Two flavors

  • eBGP External BGP Wiki EN - Exterior Gateway Protocol (EGP)

    • There is another EGP called EGP that should not be confused

      Wiki EN - EGP

      • IETF RFC904 - Exterior Gateway Protocol Formal Specification

      • Obsolete

  • iBGP Interal BGP Wiki EN - Interior Gateway Protocol (IGP)

• Youtube - Kevin Wallace Training, LLC - BGP Deep Dive

  • Many thanks for the introduction!
• Web-Tools

  • https://www.cidr-report.org/as2.0/

  • https://www.cidr-report.org/v6/as2.0/

  • https://bgphelp.com

  • networkstraining.com Search for "BGP"

  • https://notes.networklessons.com/

  • Hurricane Electric BGP Toolkit

• Interesting reads

  • noction.com - BGP AS Migration Process

    • IETF RFC7705 - Autonomous System Migration Mechanisms and Their Effects on the BGP AS_PATH Attribute

    • Network Startup Resource Center (NSRC) - BGP for All

    • BGP Filter Guide

• Forms neighborships
  • Explicitly configured neighbors

  • Neighborships communicate over TCP-Session tcp/179

    • No dynamic neighbor discovery using multicast (like in e. g. OSPF)
• Advertises address prefix and prefix length in NRLIs packages
• Advertises a collection if path attributes used for path selection
• Does not consider bandwidth or cost for path selection
• Scales good
  • OSPF may not scale to the size required in large enterprises.

Standards

• IETF RFCs relevant for BGP

  • IETF RFC4271 - A Border Gateway Protocol 4 (BGP-4)

  • IETF RFC4724 - Graceful Restart Mechanism for BGP

  • IETF RFC6286 - Autonomous-System-Wide Unique BGP Identifier for BGP-4

  • IETF RFC6793 - BGP Support for Four-Octet Autonomous System (AS) Number Space

• IETF Workgroup BGP Enabled ServiceS (bess)

• Best Current Practices (BCP)
  • IETF

    • IETF BCP - BGP

      • Many BCPs

  • RIPE

    • ripe.net - RIPE-706 - Mutually Agreed Norms for Routing Security (MANRS) Implementation Guide

      • https://manrs.org/

    • ripe.net - RIPE-580 - RIPE Routing Working Group Recommendations on Route Flap Damping

    • ripe.net - RIPE-431 - RIPE Anti-Spoofing Task Force HOW-TO

    • ripe.net - RIPE-399 - RIPE Routing Working Group Recommendations on Route Aggregation

• IANA

  • Border Gateway Protocol (BGP) Parameters

  • Border Gateway Protocol (BGP) Well-known Communities

  • IANA - Border Gateway Protocol (BGP) Extended Communities

Implementations

• https://frrouting.org/

  • Licenses:
    • Source code: various Licenses
    • Composite binary: GPLv2 or later
  • Evolved from

    • Wiki EN - Quagga, GPLv2

    • Evolved from GNU Zebra, GPL

  • github.com - FRRouting/FRR

  • FRRouting - Docs

    • FRRouting Docs - User Guide

      • FRRouting Docs - BGP

Sniffing

In wireguard I find the follwing filter very handy, which filters out keep-alive messages (of type 4)

   1 bgp.type!=4

Abbreviations

Path attributes (PA)

PAs by ID

WIP

Best path selection

Cost or bandwidth are not considered for path selection.

During path selection PAs are compared. If they are not equal a routing decision can be done, but if they are tie the next attribute is compared.

Neighbors

• There is no dynamic BGP neighbor discovery
  • Neighbors must be configured explicitly
• eBGP neighbors have different AS numbers

• iBGP neighbors simply have the same remote-as AS-NUMBER during configuration

Show neighbors

   1 show ip bgp summary
   2 show ip bgp neighbor

Neighbor formation states

1. State Idle: Doing nothing

2. State Connect: establish TCP-session

3. State Active: established TCP-session

4. State Open Sent: initiate establish neighborship

5. State Open Confirm: confirm establish neighborship

6. State Established: established neighborship

7. State Shutdown: administratively down/not up

Peer group

• Simplify configuration by grouping of neighbors
• Various configuration (like policies, route-maps, filters, …) can be applied to the peer group

  • Example: apply #Prefix filtering to peer group

     1 conf term
     2   router bgp AS-NUMBER
     3   neighbor PG_NAME peer group
     4   !APPLY PREFIX LIST INBOUND
     5   neighbor PG_NAME prefix-list PREFIX_LIST_NAME in
     6   !ASSIGN NEIGHBOR TO PEER-GROUP 
     7   neighbor IP-ADDRESS1 peer-group PG_NAME
     8   neighbor IP-ADDRESS2 peer-group PG_NAME
  

• Removing a peer group

     1 conf term
     2   router bgp AS-NUMBER
     3   no neighbor PG_NAME peer group
  

Messages

• Open: BGP version number, Local AS Number, Hold Time, BGP router ID, optional parameters

• Keepalive: Keeps holdtime timer from expiring

  • Defaults:
    • Keepalive: 60s
    • Holdtime: 180s (3 * default Keepalive)

• Update: Can contain withdrawn routes, path attributes and NLRI

• Notification: Contains an error code, error sub-code, and information about the error

Timers

• IETF RFC4271 - A Border Gateway Protocol 4 (BGP-4) #10. BGP Timers

• networkstraining.com - The Most Important Border Gateway Protocol (BGP) Timers Explained

Defaults

• Keepalive: 60s
• Holdtime: 180s (3 * keepalive)

Other timers

• update-delay: 0s (disabled)
  • readonly BGP until update-delay expired

Network advertisement

• You should not advertise the networks of the peers of the other AS, unless you want to become a transit AS/area and you want to route the traffic through your devices
• Only advertise your AS-internal networks you want to propagate throughout the internet
• Your router will not advertise the network defined by the network statement unless there is a corresponding route in the routing table.
  • Add a null route

       1 ip route add A.B.C.D/SNM interface null 0
       2 ipv6 route add A::B/SNM interface null 0
    

    • A null route does not replace the directly connected route, even if the prefixes match perfectly including the prefix-length.
    • A connected route is also not removed if you remove the matching null route.
  • In cisco
  • Under some instances a directly connected interface may not appear in the local routing. This may also only be a glitch, as IMHO they should. Shutting down and and enabling the interface may help.

       1 configure terminal
       2 interface loopback 0
       3  shutdown
       4  no shutdown
    

   1 conf term
   2   router bgp AS-NUMBER
   3     network PREFIX mask SNM

Synchronization (with IGP)

• To avoid unintentional black holes
• Only advertises a route learned from an iBGP peer to an eBGP peer when there is an exact match of that route learned from an IGP in the routing table
• Disabled by default in Cisco IOS 12.2(8)T
• Not really necessary
  • There may be no IGP on the eBGP router

•    1 conf term
     2   router bgp AS-NUMBER
     3     synchronization
  

Hard restart

Cisco: In EXEC mode

   1 clear ip bgp *

Graceful restart (GR)

• IETF RFC4724 - Graceful Restart Mechanism for BGP

• FRR Docs - BGP #Send Hard Reset CEASE Notification for Administrative Reset

• FRR Docs - BGP #Graceful Restart Graceful restart is a mechanism for BGP that would help minimize the negative effects on routing caused by BGP restart. An End-of-RIB marker is specified and can be used to convey routing convergence information. A new BGP capability, termed "Graceful Restart Capability", is defined that would allow a BGP speaker to express its ability to preserve forwarding state during BGP restart.

• Also available for OSPF, EIGRP, IS-IS, …
• GR towards RR is especially useful
• GR is preferred to Non-Stop-Routing
• Timers
  • Restart timer:
    • Default: 120 s
    • Speaker must come back in this time or is considered dead.
  • Selection_Deferral_Timer:
    • Default: ? s
  • Stale-path timer:
    • Default: ? s
• Each routing protocol has to be checked, that the peer is capable and aware of GR
• Usage of EoR marker is beneficial for convergence time, since computation can start on reception

Process:

1. Capability to perform graceful restart (Capability: 64) is announced to neighbors in OPEN message.
   • May be enabled on a per neighbor basis
2. Peers are notified that a GR takes place using an OPEN message with restart bit set.
   • On a peer
     • the routes in the routing table are marked as stale
     • traffic is still forwarded to the restarting speaker
     • the restart timer is started
3. On the restarting speaker the Data-plane continues to forward traffic, while the control-plane restarts.
4. When a new BGP session is established
   • On the peer
     • the restart timer is stopped (heard back)
     • the stale-path timer is started
   • On the restarting router
     • initial updates are exchanged with neighbors
     • performs Path-Selection
       • on End-Of-RIB (EoR) by all peers with GR capability set or
       • Selection_deferal_timer has expired
     • send updates and EoR to neighbors
5. On the peer
   • the stale-path timer is stopped
   • the stale prefixes are flushed
   • new prefixes are injected
6. Convergence

   1 

Graceful shutdown procedure

• Youtube.com - NLNOG day 2017 - Job Snijders (NTT) - BGP Graceful Shutdown

• Simple procedure to reduce negative impact of shutting down BGP sessions (e.g. during maintenance)
• Can be combined with

  IETF RFC8203 - BGP Administrative Shutdown Communication

• Can be part of the operational procedure as outlined in

  IETF BCP214 - Mitigating the Negative Impact of Maintenance through BGP Session Culling

• Graceful shutdown is a "Make Before Break" mechanism
• Does not help against unplanned outages
• Not to be confused with "BGP Graceful Restart"
• Only useful for multi-homed AS

Scenario topology

   1       RR
   2      /  \
   3 ASBR1   ASBR2
   4      \  /
   5       RA

Normal shutdown

1. Router A (RA) is shutdown

2. BGP session to ASBR2 is torn down
3. ASBR2 withdraws routes sending a NLRI to the route reflector (RR)
   1. ASBR2 has no more routes to prefixes announced by RA and blackholes legitimate traffic
4. RR selects a new best path and updates ASBR2
   1. ASBR2 starts forwarding traffic using the new alternative path

Graceful shutdown

1. Before maintenance Router A (RA)

   1. Attaches the GRACEFUL_SHUTDOWN well-known community 65535:0 (as per IANA) to the prefixes

   2. Waits for a reasonable time (some minutes)
2. ASBR2 receives updates from RA

   1. Finds the community GRACEFUL_SHUTDOWN in the NLRI

   2. Lowers the LOCAL_PREFERENCE (AS-wide) to 0
   3. Updates RR
   4. RR p

   5. RR sends new best path to ASBR2 and other peers

      1. Community GRACEFUL_SHUTDOWN propagates through AS

         1. Other intermediate AS may select a new path prior to shutdown
   6. ASBR2 selects new best path received from RR
      • Alternative path are already in place
      • RA is drained
      • Minimal package loss
3. RA can now be shutdown without impact
   1. BGP session to ASBR2 is torn down without impact
4. …

Graceful shutdown configuration

Allow peers to graceful shutdown

• Configure this on each and every BGP session

• Example Cisco IOS XR

     1 conf term
     2   route-policy AS-ASN_NEIGH-ebgp-inbound
     3     if community matches-any (65535:0) then
     4       set local-preference 0
     5     endif
     6   end-policy
     7   !
     8   router bgp ASN_OWN
     9     neighbor IP_NEIGH remote-as ASN_NEIGH
    10     address-family ipv6 unicast
    11       send-community-ebgp
    12       route-policy AS-ASN_NEIGH-ebgp-inbound in
    13       end
  

• Example ARISTA/Brocade/IOS/Quagga/FRR

     1 conf term
     2   ip community-list standard gshut 65535:0
     3   !
     4   route-map ebgp-in permit 10
     5     match community gshut
     6     set local-preference 0
     7     continue
     8     end
  

Graceful shutdown your systems

Cisco IOS XR (automatically shuts down BGP session)

   1 conf term
   2   router bgp ASN_OWN
   3     neighbor IP_NEIGHBOR remote-as ASN_NEIGH graceful-maintenance active
   4     neighbor-group GROUP-NEIGH graceful-maintenance active

Route aggregation

• FRR Doc - BGP #Route Aggregation

• Reduces the size and therefor increases the efficiency of the routing table.
• Algorithm
  • Binary AND of network prefixes to determine common bits.
  • Appending bits with value 0 of the resulting network address are counted, subtracted from the maximum length of the SNM (32/128) and converted to decimal to calculate the subnet mask.
  • Summarized network address and SNM are concatenated to the summarized network prefix.
• Summarized address
  • Be careful, because summarization will be performed also with wholes, which may cause issues.
  • Should work because advertisements of network falling into that route are more specific.
  • Network with the least cost is picked as cost for the summarized route
• Blueprint

     1 conf term
     2   router bgp AS-NUMBER
     3     aggregate-address PREFIX SNM summary-only
  

eBGP multihop

• Form a neighborship with a not neighboring router

     1 conf term
     2   router bgp AS-NUMBER
     3     neighbor IP-ADDRESS remote-as REMOTE-AS-NUMBER
     4     neighbor IP-ADDRESS ebgp-multihop 2
  

iBGP

iBGP Assumptions

• Full Mesh - All routers running BGP are directly connected in the same AS
  • The number of neighborships grows quadratic

    with the number of routers participating in iBGP in the same AS.
    Number of adjacencies in a full mesh

    • Solutions
      • BGP Confederation using Sub Autonomous Systems
        • Sub-ASs from private AS address space
        • Sub-Autonomous Systems require a full mesh …
      • BGP Route reflectors
        • Mirrors received advertisements to other routers

• An iBGP neighbor has simply the same remote-as AS-NUMBER during configuration.

iBGP next hop self

• FRR Docs - BGP #next-hop-self

• iBGP router has learned a route from another eBGP peer and the route is shown in, but the next hop is set to an unreachable peer of the propagating eBGP router,

  where the iBGP router has no route to reach him
  show ip bgp

• The route is not in the routing table
  show ip route

• The eBGP router does not adjust the attribute "next hop"
• On the eBGP router set the attribute next hop for a specific neighbor

     1 conf term
     2   router bgp AS-NUMBER
     3     neighbor IP-ADDRESS next-hop-self
  

iBGP route reflector (RR)

• FRR Docs - BGP #Route Reflector

• To overcome the iBGP requirement of a full mesh an iBGP router can be configured as a route reflector.
• Therefor the iBGP neighbors of this [designated] router, that should receive mirrored route advertisements are configured to be a route-reflector-client

     1 conf term
     2   router bgp AS-NUMBER
     3     neighbor IP-ADDRESS1 route-reflector-client
     4     neighbor IP-ADDRESS2 route-reflector-client
  

• A RR breaks the split horizon rule.
  • A RR advertises routes learned from iBGP peers to other iBGP peers.
• Route reflectors do not modify any BGP path attributes, when they reflect a route, except the ORIGINATOR_ID and the CLUSTER_LIST PAs.

RR clusters

An RR and its clients form a cluster. The cluster ID is then attached to every route advertised by the RR to its client or nonclient peers. A cluster ID is a cumulative, non-transitive BGP attribute, and every RR must prepend the local cluster ID to the cluster list to avoid routing loops.

iBGP router types

• Route reflector server
  • send reflections
• Route reflector clients
  • receive reflections
• non-clients
  • Part of the full mesh

iBGP route reflection rules

RR servers propagate routes inside the AS based on the following rules:

• Routes are always reflected to eBGP peers.
• Routes are never reflected to the originator of the route.
• If a route is received from a non-client peer, reflect to client peers.
• If a route is received from a client peer, reflect to client and non-client peers.

iBGP loop prevention

• Split-horizon
  • A route learned from a iBGP peer is not advertised to other iBGP peers within the AS.
  • RR is disables this rule.
• Filtering on ORIGINATOR_ID (PA Originate)
  • type code: 9
  • Carries the router-id of the route originator
  • ORIGINATOR_ID is set by the first RR to the router-id of the router that originated the route into the AS. The ORIGINATOR_ID is not transitive and stays unmodified while passing other RRs.
  • If a router receives an update that contains its own router-id in the ORIGINATOR_ID field it discards the update. The route is ignored.
  • Only used by RRs
• Filtering on CLUSTER_LIST (PA CLUSTER_LIST)
  • type code: 10
  • Carries a sequence of cluster-ids of RRs that the route has passed through
  • When a RR reflects a route it prepends its local cluster-id to the CLUSTER_LIST PA
  • If an RR receives an route with a CLUSTER_LIST PA has includes its own cluster-id, it discards the update.
  • Only used by RRs

iBGP Confederation

• Makes sense in bigger topologies.
• to be evaluated

Route-map

Outbound path selection (LOCAL_PREF)

• Create a route-map that sets LOCAL_PREF

     1 conf term
     2   route-map LOCAL_PREF-ISP1
     3     ! 100 IS ALSO DEFAULT
     4     set local preference 100
     5   route-map LOCAL_PREF-ISP2
     6     ! 100 IS ALSO DEFAULT
     7     set local preference 200
  

• Apply route-map to ISP-routes to influence outbound path selection (AS-wide). (9

     1 conf term
     2   router bgp AS-NUMBER
     3     ! APPLY ROUTE_MAP TO INCOMING-ROUTES OF ISP1
     4     ! (ACTUALLY UNNECESSARY - DEFAULT IS 100)
     5     neighbor IP-ADDRESS_ISP1 route-map LOCAL_PREF-ISP1 in
     6     ! APPLY ROUTE_MAP TO INCOMING-ROUTES OF ISP2
     7     neighbor IP-ADDRESS_ISP2 route-map LOCAL_PREF-ISP2 in
  

Inbound path selection (AS-path prepending)

• Create a route-map that prepends additional instances of your own AS-number

     1 conf term
     2   route-map AS-PATH-ISP1
     3     ! 100 IS ALSO DEFAULT
     4     set as-path prepend AS-NUMBER AS-NUMBER
  

• Apply route-map to outbound routes to ISP1 to influence inbound path selection.

     1 conf term
     2   router bgp AS-NUMBER
     3     ! APPLY ROUTE_MAP TO INCOMING-ROUTES OF ISP1
     4     neighbor IP-ADDRESS_ISP1 route-map AS-PATH-ISP1 out 
  

Enable IPv6 unicast routing (Cisco)

   1 conf term
   2   ipv6 unicast-routing

Multiprotocol BGP (IPv4/IPv6)

• Using address-families for IPv4/IPv6
• Configure IPv6 addresses to the interfaces

     1 conf term
     2   int gig 0/3
     3     ipv6 address IPV6-ADDRESS
  

• #Enable IPv6 unicast routing (Cisco)

• Create route-map

     1 conf term
     2   route-map IPV6-NEXT-HOP
     3     ! SET OWN ADDRESS AS NEXT-HOP
     4     set ipv6 next-hop IPV6-ADDRESS
  

• Setup BGP router

     1 conf term
     2   router bgp AS-NUMBER
     3     neighbor IP-ADDRESS_ISP2 remote-as AS-ISP2
     4     !ENABLE BOTH ADDRESS FAMILIES
     5     address-family ipv4
     6       network IPV4-PREFIX mask SNM
     7     exit
     8     address-family ipv6
     9       network IPV6-PREFIX mask SNM
    10       ! ACTIVATE IPV6 PEER
    11       neighbor IP-ADDRESS_ISP2 activate
    12       neighbor IP-ADDRESS_ISP2 route-map IPV6-NEXT-HOP out
    13     exit
  

• Test with

     1 show ip route
     2 show ipv6 route
     3 show ip bgp
  

Equal cost multipath (ECMP)

• BGP and equal-cost multipath (ECMP)

• notes.networklessons.com - BGP Equal Cost Multipath AS_Path Attribute

• cisco.com - iBGP Multipath Load Sharing

• First make sure to have equal costs on the routes.

• There are multiple ways to achieve ECMP
  1. Inject multiple paths in routing table

        1 conf term
        2   router bgp AS-NUMBER
        3     neighbor IP-ADDRESS1 remote-as PEER-AS-NUMBER
        4     neighbor IP-ADDRESS2 remote-as PEER-AS-NUMBER
        5     maximum-paths N
        6     bgp bestpath as-path multipath-relax
     

  2. Configure loopback with address, set static host routes to neighbor loopbacks, enable ebgp-multihop and run BGP-session from loopback to loopback

        1 conf term
        2   ip route REMOTE-IP-ADDRESS1 255.255.255.255 REMOTE-IFACE-ADDRESS
        3   ip route REMOTE-IP-ADDRESS1 255.255.255.255 REMOTE-IFACE-ADDRESS
        4   router bgp AS-NUMBER
        5     neighbor REMOTE-IP-ADDRESS1 remote-as PEER-AS-NUMBER
        6     neighbor 172.16.31.2 update-source loopback0
        7     neighbor 172.16.31.2 ebgp-multihop 2
     

  3. Use LACP/802.3ad, … and configure a BGP session over this joint link.

Security

• Wiki EN - BGP hijacking

• Wiki EN - Bogon filtering

• Wiki EN - Resource Public Key Infrastructure

• Wiki EN - QoS Policy Propagation via BGP (QPPB)

• Wiki EN - Route filtering

Filter TCP/197

• ACL filters input on tcp/179 for devices not eligible to become neighbors.
  • Protection from DoS by resource exhaustion (bandwidth, CPU, …)
  • Spoofed TCP RST can bring down BGP sessions
  • On control-plane or interface level
  • Please also see:

    IETF RFC 6192 - Protecting the Router Control Plane

• Rate-limit tcp/179 to reserve platform resources

Filter inbound own source-addresses

• Block spoofed packets (packets with a source IP address belonging to their IP address space) at all edges of their network
  • protects the TCP session used by Internal BGP (IBGP) from attackers outside the Autonomous System

Prefix filtering

• in- or outbound
• match IPs, ASNs, or any other attribute of a prefix (e.g. communities)
• Filters
  • Drop IPv4/6 special purpose prefixes

    • IANA IPv4 Special-Purpose Address Registry

    • IANA IPv6 Special-Purpose Address Registry

    • prefixes of non-global scope
  • Drop IPv4/6 unallocated prefixes

    • IANA IPv4 Address Space Registry

    • Internet Protocol Version 6 Address Space

  • RIR-Allocated Prefix Filters

    • radb.net - Routing Assets Database

    • Internet Routing Registry ToolSet

      • github.com irrtoolset/irrtoolset

        • IRRToolSet is not actively maintained
        • Instead use BGPQ4

      • github.com bgp/bgpq4 - bgp filtering automation tool

• https://bgpfilterguide.nlnog.net/

Show prefix lists

   1 show ip prefix list

Create a prefix list

   1 conf term
   2   ip prefix-list BGP_RFC-1918
   3     seq 5 deny 10.0.0.0/8 le 32
   4     seq 10 deny 172.16.0.0/12 le 32
   5     seq 15 deny 192.168.0.0/16 le 32
   6     seq 20 permit 0.0.0.0/0
   7     seq 25 permit 0.0.0.0/8 ge 8

Session Protection

• Please also see:

  IETF RFC 6952 - Analysis of BGP, LDP, PCEP, and MSDP Issues According to the Keying and Authentication for Routing Protocols (KARP) Design Guide

Policies

• BGP is typically restricted by policies.
  • Common policies are
    • Bogon filtering
    • Limit the routes learned to a maximum AS-path length
    • Filter routes with origin "255" (devel)

Authentication

• Peer may be authenticated (using MD5)

  IETF RFC2385 - Protection of BGP Sessions via the TCP MD5 Signature Option

• IETF - The TCP Authentication Option (TCP-AO)

• Also possible using IPsec
• [[|IETF - ]]

Error tolerance

• Do not tear down BGP session, just withdraw the route with the potentially faulty attribute.

Some keywords

   1 ### JunOS
   2 bgp-error-tolerance;
   3 ### NOKIA SR-OS
   4 error-handling
   5   update-fault-tolerance
   6 exit