Finally I did IT – BGP Connection between Unifi UDM Pro & VMware VCF NSX

A lot has happened in my home lab over the last few days. I finally found the time to implement VCF 9.0.
This also brought the issue of connectivity/routing between my Unifi UDM-Pro and NSX back into focus.
Previously, I had the Tier0 router from NSX and the UDM-Pro communicate with each other using OSPF. Feel free to take a look at my previous blog article here. In any case, this only works with an MTU setting of 1500 on the UDM-Pro.
However, since the VCF NSX setup requires two VLANs for the uplink to the physical world, I had to make some changes to my UDM-Pro, as packets with a minimum size of 1700 must be transmitted between the VLANs – TEP network requirement.
Unfortunately, jumbo frames cannot be enabled or disabled for individual interfaces on the UDM-Pro, which means that if I enable jumbo frames globally on my UDM-Pro, my OSPF routing will no longer work.
Therefore, I finally decided to switch my routing to BGP.
This took me almost a day of trial and error, as the documentation from Unifi once again leaves a lot to be desired in this regard.
In addition, my colleague Daniel Krieger (SDN-Warrior.org) also provided me with some support so that I could finally get the scenario up and running.
There was still one major hurdle: although the BGP connection was there, no routes were being exchanged between the peers.

The following section describes how I carried out the setup and implementation in my lab. As always, this setup is sufficient for home lab use, but additional configuration steps may be necessary for production scenarios.

First of all, a short overview of my NSX Setup, which is connected to my physical Unifi World (UDM-Pro)

I initiated the workflow from VCF to deploy the edge nodes with static routing and subsequently configured the changes that were necessary on the NSX side for BGP.

As mentioned at the beginning, I use two uplink VLANs as specified by VCF, so I also used two BGP neighbours to establish the connection between the routers, each with corresponding interfaces in the respective subnets that I defined.

Then came the ‘fun part,’ or rather the ‘young scientists’ part, because unfortunately, the implementation of BGP on the Ubiquiti side is not very well documented.
I found out that you have to create a configuration file and upload it via the Unifi UI.
Long story short, it took 11 versions of this configuration file before I finally saw all the routes on both sides, or rather on both routers.
It almost drove me to despair that the BGP connection between the routers was there, but no routes were being exchanged.

!
router bgp 65000
bgp router-id 192.168.1.254
no bgp ebgp-requires-policy
!
neighbor 192.168.240.2 remote-as 65001
neighbor 192.168.240.3 remote-as 65001
neighbor 192.168.250.2 remote-as 65001
neighbor 192.168.250.3 remote-as 65001
!
address-family ipv4 unicast
  redistribute connected
  redistribute static
  redistribute kernel
!
  neighbor 192.168.240.2 activate
  neighbor 192.168.240.3 activate
  neighbor 192.168.250.2 activate
  neighbor 192.168.250.3 activate
!

  neighbor 192.168.240.2 soft-reconfiguration inbound
  neighbor 192.168.240.3 soft-reconfiguration inbound
  neighbor 192.168.250.2 soft-reconfiguration inbound
  neighbor 192.168.250.3 soft-reconfiguration inbound
!
exit-address-family
!
!

This is version 11 of my configuration file – the one that finally worked.
What ultimately cost me so much time and what I probably wouldn’t have been able to solve without the help of my colleague Daniel Krieger (SDN-Warrior.org) was the line ‘no bgp ebgp-requires-policy’.
Because if this line is not included in the configuration file, all routes are blocked unless you configure a specific policy to allow them. This is certainly important in production networks, but in my lab, I feel confident allowing everything.

Once the configuration was finally in place, the results could be assessed on both the NSX side and the Unifi side.

Except in the NSX GUI, you can also see everything directly on the NSX Edges.

And finally, the check directly on the UDM-Pro

That concludes the implementation of BGP routing in my lab. As mentioned earlier, this is my implementation in my home lab with Ubiquiti network components (you can think what you want about that). In a productive environment, further security aspects are certainly necessary, etc.
If you have any further questions or comments, please feel free to contact me.