[minipost] CEF FIB vs. Routing Table, or when a Routing Table lies
What if I were to tell you that there exist a situation where the router will NOT route according to the routing table … well, at least not for a while that is. Interesting, isn’t it ? Lets explore this small issue that was lately brought to my attention.
NOTE: This functionality can be observed in Cisco IOS versions 12.4 (19) which is tested here, the 15.x IOS versions do not keep the adjacency /32 prefix in CEF FIB and therefore this whole article is not applicable to the newer IOS versions.
The following topology is everything that is needed to provide this example of routing table having and argument with CEF FIB about there to route packets. In the shown topology, we have only a few basic routers with routing being pointed by default routes to the R2 router from all other topologies.
Step 1. Testing default connectivity
The connectivity is working just as expected, for our tests, we will always use pings from R1 to R4 and R3 as shown below.
R1(config)#do ping 5.100.23.3 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 5.100.23.3, timeout is 2 seconds: ..!!! Success rate is 60 percent (3/5), round-trip min/avg/max = 16/86/132 ms R1(config)#do ping 5.100.23.129 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 5.100.23.129, timeout is 2 seconds: .!!!! Success rate is 80 percent (4/5), round-trip min/avg/max = 20/55/104 ms
So the pings work (the few failed pings at the beginning of each test is because of ARP being resolved) and we can not try to create a conflict with routing table and CEF by introducing an interesting way to filter out traffic.
Step 2. Trying to to block part of 5.100.23.0/24 with more specific route to null
Ok, I know that access-list are more meant to be standard solution for filtering, but somehow, this solution seems to be completely equivalent and most people will expect this to work without a problem.
We want to block communication from R1 to R4 by sending all the 5.100.23.128/25 to interface “null 0”, where it will be dropped.
R2(config)#ip route 5.100.23.128 255.255.255.128 Null0
Now, the access from R1 to R4 should be blocked. Below is the R2 routing table and also a test from R1 is shown below (NOTE: We expected the ping test to fail):
R2(config)#do sh ip route -- OMITTED -- Gateway of last resort is not set 5.0.0.0/8 is variably subnetted, 3 subnets, 2 masks C 5.100.12.0/24 is directly connected, FastEthernet0/0 C 5.100.23.0/24 is directly connected, FastEthernet0/1 S 5.100.23.128/25 is directly connected, Null0 R2(config)#
R1(config)#do ping 5.100.23.129 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 5.100.23.129, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 20/44/68 ms
What the hell?! The routing table is telling us that all the traffic to 5.100.23.128/25 where the destination 5.100.23.129/32 belong to is rerouted to null 0 and should be dropped, so how come that the traffic is working ?!
Step 3. CEF overriding routing-table
This small problem encountered in a production environment recently has raised a few eyebrows in the office. In the process, we decided that CEF has an hardware issue on the router and we disabled CEF and the route to null has worked!
R2(config)#no ip cef
R1(config)#do ping 5.100.23.129 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 5.100.23.129, timeout is 2 seconds: U.U.U Success rate is 0 percent (0/5)
So it must be something with CEF…. but what?
Let’s reactivate CEF and look into the CEF Forwarding Information Base (FIB) table.
R2(config)#do sh ip cef Prefix Next Hop Interface 0.0.0.0/0 drop Null0 (default route handler entry) 0.0.0.0/32 receive 5.100.12.0/24 attached FastEthernet0/0 5.100.12.0/32 receive 5.100.12.1/32 5.100.12.1 FastEthernet0/0 5.100.12.2/32 receive 5.100.12.255/32 receive 5.100.23.0/24 attached FastEthernet0/1 5.100.23.0/32 receive 5.100.23.2/32 receive 5.100.23.3/32 5.100.23.3 FastEthernet0/1 5.100.23.128/25 attached Null0 5.100.23.129/32 5.100.23.129 FastEthernet0/1 5.100.23.255/32 receive 224.0.0.0/4 drop 224.0.0.0/24 receive 255.255.255.255/32 receive
The classic “longest-prefix-match” or routing to a most specific route is working the same in the CEF FIB, and CEF FIB takes precedence over routing-table!
Summary and Solution
When we done Step1, the CEF populated the CEF FIB with a /32 prefix route to the adjacent router for the 5.100.23.129/32 destination as the machine is in directly connected subnet.
To fix the connectivity issue is to clear the adjacency manually by either “clear adjacency” command (which didn’t worked for my IOS version) or “shutdown” and “no shutdown” the interface. When you to this in this order of operations, the arp will no longer resolve the 5.100.23.129 MAC address and the adjacency table will no longer create the /32 prefix destination. One might say that this whole problem is only caused by the order of operation because if the route to null 0 would be installed before arp resolved the destination, this problem would have been avioded.
However, this problem only occurred because we added the route to null interface AFTER the CEF adjacency /32 prefix was already present in the CEF FIB and we entered to the static route to the routing table afterwards. If we shutdown and un-shutdown the R2 interface to 5.100.23.0/24 network, the CEF FIB for this interface is cleared and WITH the route to null entered, the /32 will not be learned this time.
Then again, as mentioned at the beginning, this behavior “glitch” can be seen on IOS versions 12.4 (I used 12.4(19) in this topology) and is not applicable to version 15.x IOS. The 15.x IOS automatically cleared the adjacency table in EF FIB for the destination 5.100.23.129 automatically after I entered the static route command pointing to the null 0 interface.
I hope you found this small article interesting.
Peter