In the introduction to VXLAN lesson, I explained what VXLAN is and how it works. In this lesson, I’ll show you how to configure VXLAN where we use the multicast “flood and learn” system to learn the mapping between a VTEP IP address and a MAC address.
Configuration
Here’s the topology we’ll use:
All devices are CSR1000V routers running Cisco IOS XE Software, version 16.06.01. I’m using CSR1000V routers since anyone can use these. I use custom MAC addresses because those are easy to recognize when we do a packet capture.
VTEP1 and VTEP2 are our VTEP devices. The core router is there to simulate our “IP network”. We are going to create a VXLAN tunnel with VNI 5012 so that H1 and H2 can communicate directly over layer 2.
I pre-configured OSPF so that we have connectivity between the VTEP devices and the core router.
hostname CORE
!
interface Loopback0
ip address 3.3.3.3 255.255.255.255
!
interface GigabitEthernet2
mac-address 0000.5e00.5303
ip address 192.168.13.3 255.255.255.0
!
interface GigabitEthernet3
mac-address 0000.5e00.5333
ip address 192.168.23.3 255.255.255.0
!
router ospf 1
network 3.3.3.3 0.0.0.0 area 0
network 192.168.13.0 0.0.0.255 area 0
network 192.168.23.0 0.0.0.255 area 0
!
end
hostname H1
!
interface GigabitEthernet2
mac-address 0000.5e00.5365
ip address 192.168.12.101 255.255.255.0
!
end
hostname H2
!
interface GigabitEthernet2
mac-address 0000.5e00.5366
ip address 192.168.12.102 255.255.255.0
!
end
hostname VTEP1
!
interface Loopback0
ip address 1.1.1.1 255.255.255.255
!
interface GigabitEthernet2
mac-address 0000.5e00.5301
!
interface GigabitEthernet3
mac-address 0000.5e00.5311
ip address 192.168.13.1 255.255.255.0
!
router ospf 1
network 1.1.1.1 0.0.0.0 area 0
network 192.168.13.0 0.0.0.255 area 0
!
end
hostname VTEP2
!
interface Loopback0
ip address 2.2.2.2 255.255.255.255
!
interface GigabitEthernet2
mac-address 0000.5e00.5302
!
interface GigabitEthernet3
mac-address 0000.5e00.5322
ip address 192.168.23.2 255.255.255.0
!
router ospf 1
network 2.2.2.2 0.0.0.0 area 0
network 192.168.23.0 0.0.0.255 area 0
!
end
Multicast
Let’s start with the configuration of multicast. With VXLAN, we don’t have a typical scenario where we have a few sources and many receivers. All VTEP devices communicate with each other so it makes sense to use bidirectional PIM. The core router will be the RP in this network.
Let’s enable multicast routing and bidirectional PIM on all VTEP devices and the core router:
VTEP1, VTEP2 & CORE
(config)#ip multicast-routing distributed
(config)#ip pim bidir-enable
We need to enable PIM sparse mode on all physical interfaces that connect to the IP network:
VTEP1 & VTEP2 & CORE
(config)#interface GigabitEthernet 3
(config-if)#ip pim sparse-mode
CORE(config)#interface GigabitEthernet 2
CORE(config-if)#ip pim sparse-mode
And don’t forget the loopback interfaces:
VTEP1, VTEP2 & CORE
(config)#interface Loopback 0
(config-if)#ip pim sparse-mode
Last but not least, configure the RP address:
VTEP1, VTEP2 & CORE
(config)#ip pim rp-address 3.3.3.3 bidir
This completes the multicast configuration.
VXLAN
We need to create a Network Virtualization Endpoint (NVE) interface. This is where we configure the VNI and multicast group that we will use. We source this interface from the loopback 0 interface, use VNI 5012, and use multicast group 239.1.1.1.
Here’s how to configure the NVE interface:
VTEP1 & VTEP2
(config)#interface NVE 1
(config-if)#no shutdown
(config-if)#source-interface Loopback 0
(config-if)#member vni 5012 mcast-group 239.1.1.1
Now we need to configure the Ethernet Flow Point (EFP) service instance. This is a logical interface that connects a bridge domain to a physical port (or EtherChannel). Under the service instance, we configure whether the incoming traffic is tagged or untagged. In our case, the hosts send untagged traffic. This is how to configure it:
VTEP1 & VTEP2
(config)#interface GigabitEthernet 2
(config-if)#service instance 1 ethernet
(config-if-srv)#encapsulation untagged
(config-if-srv)#exit
(config-if)#exit
Last but not least, we need to configure the Bridge Domain Interface (BDI):
The BDI is the IOS XE equivalent of the IOS Bridge-Group Virtual Interface (BVI).
This is where we combine the VNI, physical interface, and service-instance:
VTEP1 & VTEP2
(config)#bridge-domain 1
(config-bdomain)#member vni 5012
(config-bdomain)#member GigabitEthernet 2 service-instance 1
This completes our VXLAN configuration.
I’m showing the two
exit
commands on purpose because I configure the bridge-domain globally. You can also configure the bridge-domain under the service instance.Verification
Let’s verify our work.
Multicast
First, I’ll check if our multicast configuration is correct:
VTEP1#show ip mroute 239.1.1.1
IP Multicast Routing Table
(*, 239.1.1.1), 00:00:36/00:02:25, RP 3.3.3.3, flags: BCx
Bidir-Upstream: GigabitEthernet3, RPF nbr 192.168.13.3
Outgoing interface list:
Tunnel0, Forward/Sparse-Dense, 00:00:36/00:02:25
GigabitEthernet3, Bidir-Upstream/Sparse, 00:00:36/stopped
VTEP2#show ip mroute 239.1.1.1
IP Multicast Routing Table
(*, 239.1.1.1), 00:00:36/00:02:24, RP 3.3.3.3, flags: BCx
Bidir-Upstream: GigabitEthernet3, RPF nbr 192.168.23.3
Outgoing interface list:
Tunnel0, Forward/Sparse-Dense, 00:00:36/00:02:24
GigabitEthernet3, Bidir-Upstream/Sparse, 00:00:36/stopped
CORE#show ip mroute 239.1.1.1
IP Multicast Routing Table
(*, 239.1.1.1), 00:00:49/00:02:45, RP 3.3.3.3, flags: B
Bidir-Upstream: Null, RPF nbr 0.0.0.0
Outgoing interface list:
GigabitEthernet3, Forward/Sparse, 00:00:44/00:02:45
GigabitEthernet2, Forward/Sparse, 00:00:49/00:02:40
I’m seeing the (*,G) entry for the multicast group 239.1.1.1 and outgoing interfaces. This is looking good.
VXLAN
Let’s try some VXLAN specific commands. First, we’ll check if the NVE interface is up:
VTEP1#show nve interface nve1
Interface: nve1, State: Admin Up, Oper Up, Encapsulation: Vxlan,
BGP host reachability: Disable, VxLAN dport: 4789
VNI number: L3CP 0 L2DP 1
source-interface: Loopback0 (primary:1.1.1.1 vrf:0)
VTEP2#show nve interface nve1
Interface: nve1, State: Admin Up, Oper Up, Encapsulation: Vxlan,
BGP host reachability: Disable, VxLAN dport: 4789
VNI number: L3CP 0 L2DP 1
source-interface: Loopback0 (primary:2.2.2.2 vrf:0)
The command above tells us whether the NVE interface is up or not. We can add the
detail
parameter to also see the number of packets or bytes we transmitted or received on this interface:VTEP1#show nve interface nve1 detail
Interface: nve1, State: Admin Up, Oper Up, Encapsulation: Vxlan,
BGP host reachability: Disable, VxLAN dport: 4789
VNI number: L3CP 0 L2DP 1
source-interface: Loopback0 (primary:1.1.1.1 vrf:0)
Pkts In Bytes In Pkts Out Bytes Out
0 0 0 0
VTEP2#show nve interface nve1 detail
Interface: nve1, State: Admin Up, Oper Up, Encapsulation: Vxlan,
BGP host reachability: Disable, VxLAN dport: 4789
VNI number: L3CP 0 L2DP 1
source-interface: Loopback0 (primary:2.2.2.2 vrf:0)
Pkts In Bytes In Pkts Out Bytes Out
0 0 0 0
Things are quiet but that will change soon. Let’s check the VNIs and multicast group addresses we use with the NVE interface:
VTEP1#show nve vni
Interface VNI Multicast-group VNI state Mode BD cfg vrf
nve1 5012 239.1.1.1 Up L2DP 1 CLI N/A
VTEP2#show nve vni
Interface VNI Multicast-group VNI state Mode BD cfg vrf
nve1 5012 239.1.1.1 Up L2DP 1 CLI N/A
Here Take a look at the
show nve peers
command:VTEP1#show nve peers
Interface VNI Type Peer-IP RMAC/Num_RTs eVNI state flags UP time
VTEP1 doesn’t know about any other VTEPs right now. This will change when we generate some traffic. The last thing we need to check is the bridge domain:
VTEP1#show bridge-domain 1
Bridge-domain 1 (2 ports in all)
State: UP Mac learning: Enabled
Aging-Timer: 300 second(s)
GigabitEthernet2 service instance 1
vni 5012
AED MAC address Policy Tag Age Pseudoport
VTEP2#show bridge-domain 1
Bridge-domain 1 (2 ports in all)
State: UP Mac learning: Enabled
Aging-Timer: 300 second(s)
GigabitEthernet2 service instance 1
vni 5012
AED MAC address Policy Tag Age Pseudoport
The output above is empty because our hosts haven’t sent anything yet. Let’s change that by sending some ICMP packets between H1 and H2:
H1#ping 192.168.12.102
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.12.102, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 2/2/4 ms
Excellent, at least there is connectivity. Let’s see if our VTEP devices now know about each other:
VTEP1#show nve peers
Interface VNI Type Peer-IP RMAC/Num_RTs eVNI state flags UP time
nve1 5012 L2DP 2.2.2.2
VTEP2#show nve peers
Interface VNI Type Peer-IP RMAC/Num_RTs eVNI state flags UP time
nve1 5012 L2DP 1.1.1.1
VTEP1 and VTEP2 now know about each other. Let’s also look again at the NVE interface:
VTEP1#show nve interface nve1 detail
Interface: nve1, State: Admin Up, Oper Up, Encapsulation: Vxlan,
BGP host reachability: Disable, VxLAN dport: 4789
VNI number: L3CP 0 L2DP 1
source-interface: Loopback0 (primary:1.1.1.1 vrf:0)
Pkts In Bytes In Pkts Out Bytes Out
5 610 5 610
VTEP2#show nve interface nve1 detail
Interface: nve1, State: Admin Up, Oper Up, Encapsulation: Vxlan,
BGP host reachability: Disable, VxLAN dport: 4789
VNI number: L3CP 0 L2DP 1
source-interface: Loopback0 (primary:2.2.2.2 vrf:0)
Pkts In Bytes In Pkts Out Bytes Out
5 610 5 610
Above, we see our 5 packets. Here’s the bridge domain output again:
VTEP1#show bridge-domain 1
Bridge-domain 1 (2 ports in all)
State: UP Mac learning: Enabled
Aging-Timer: 300 second(s)
GigabitEthernet2 service instance 1
vni 5012
AED MAC address Policy Tag Age Pseudoport
0 0000.5E00.5365 forward dynamic 170 GigabitEthernet2.EFP1
0 0000.5E00.5366 forward dynamic 169 nve1.VNI5012, VxLAN
src: 1.1.1.1 dst: 2.2.2.2
VTEP2#show bridge-domain 1
Bridge-domain 1 (2 ports in all)
State: UP Mac learning: Enabled
Aging-Timer: 300 second(s)
GigabitEthernet2 service instance 1
vni 5012
AED MAC address Policy Tag Age Pseudoport
0 0000.5E00.5365 forward dynamic 149 nve1.VNI5012, VxLAN
src: 2.2.2.2 dst: 1.1.1.1
0 0000.5E00.5366 forward dynamic 149 GigabitEthernet2.EFP1
The output above is interesting. We can see that VTEP1 and VTEP2 learned about the MAC addresses of our hosts.
We verified that our configuration works but there are some interesting things we can try with this topology.
Unknown Unicast Traffic
How about we look at the “flood and learn” system in action? To demonstrate this, I’ll clear the bridge domain on our VTEP devices:
VTEP1 & VTEP2
#clear bridge-domain 1 mac table
Once again, the VTEP devices don’t know about any MAC addresses:
VTEP1#show bridge-domain 1
Bridge-domain 1 (2 ports in all)
State: UP Mac learning: Enabled
Aging-Timer: 300 second(s)
GigabitEthernet2 service instance 1
vni 5012
AED MAC address Policy Tag Age Pseudoport
VTEP2#show bridge-domain 1
Bridge-domain 1 (2 ports in all)
State: UP Mac learning: Enabled
Aging-Timer: 300 second(s)
GigabitEthernet2 service instance 1
vni 5012
AED MAC address Policy Tag Age Pseudoport
I’ll transmit two ICMP requests from H1. I do this on purpose, the first one will be flooded and the second one will be transmitted with unicast:
H1#ping 192.168.12.102 repeat 2
Type escape sequence to abort.
Sending 2, 100-byte ICMP Echos to 192.168.12.102, timeout is 2 seconds:
!!
Success rate is 100 percent (2/2), round-trip min/avg/max = 2/3/4 ms
Here’s the first ICMP request:
Above, we see that this ICMP request is flooded because the destination is multicast group address 239.1.1.1. The ICMP reply from H2 to H1 is transmitted with unicast:
The second ICMP request from H1 is now also transmitted with unicast:
If you want to take a look at this packet capture yourself, click on the button below:
Broadcast Traffic
What about broadcast traffic? This works the same as unknown unicast. Let me show you an example:
H1#ping 192.168.12.255 repeat 1
Type escape sequence to abort.
Sending 1, 100-byte ICMP Echos to 192.168.12.255, timeout is 2 seconds:
Reply to request 0 from 192.168.12.102, 4 ms
Above, you can see that this layer 2 broadcast traffic is also flooded to the multicast group.
This wraps up this lesson!
hostname CORE
!
interface Loopback0
ip address 3.3.3.3 255.255.255.255
!
interface GigabitEthernet2
mac-address 0000.5e00.5303
ip address 192.168.13.3 255.255.255.0
!
interface GigabitEthernet3
mac-address 0000.5e00.5333
ip address 192.168.23.3 255.255.255.0
!
router ospf 1
network 3.3.3.3 0.0.0.0 area 0
network 192.168.13.0 0.0.0.255 area 0
network 192.168.23.0 0.0.0.255 area 0
!
end
hostname H1
!
interface GigabitEthernet2
mac-address 0000.5e00.5365
ip address 192.168.12.101 255.255.255.0
!
end
hostname H2
!
interface GigabitEthernet2
mac-address 0000.5e00.5366
ip address 192.168.12.102 255.255.255.0
!
end
hostname VTEP1
!
ip multicast-routing distributed
!
redundancy
bridge-domain 1
member vni 5012
member GigabitEthernet2 service-instance 1
!
interface Loopback0
ip address 1.1.1.1 255.255.255.255
ip pim sparse-mode
!
interface GigabitEthernet2
mac-address 0000.5e00.5301
service instance 1 ethernet
encapsulation untagged
!
interface GigabitEthernet3
mac-address 0000.5e00.5311
ip address 192.168.13.1 255.255.255.0
ip pim sparse-mode
!
interface nve1
source-interface Loopback0
member vni 5012 mcast-group 239.1.1.1
!
router ospf 1
network 1.1.1.1 0.0.0.0 area 0
network 192.168.13.0 0.0.0.255 area 0
!
ip pim bidir-enable
ip pim rp-address 3.3.3.3 bidir
!
end
hostname VTEP2
!
ip multicast-routing distributed
!
redundancy
bridge-domain 1
member vni 5012
member GigabitEthernet2 service-instance 1
!
interface Loopback0
ip address 2.2.2.2 255.255.255.255
ip pim sparse-mode
!
interface GigabitEthernet2
mac-address 0000.5e00.5302
service instance 1 ethernet
encapsulation untagged
!
interface GigabitEthernet3
mac-address 0000.5e00.5322
ip address 192.168.23.2 255.255.255.0
ip pim sparse-mode
!
interface nve1
source-interface Loopback0
member vni 5012 mcast-group 239.1.1.1
!
router ospf 1
network 2.2.2.2 0.0.0.0 area 0
network 192.168.23.0 0.0.0.255 area 0
!
ip pim bidir-enable
ip pim rp-address 3.3.3.3 bidir
!
end
Conclusion
You have now learned how to configure VXLAN with Multicast bidirectional PIM, how to verify your configuration, and you have seen the flood and learn system in action. I hope you enjoyed this lesson. If you have any questions feel free to leave a comment!
No comments:
Post a Comment