MFC changes relating to running multiple interfaces on different fibs but
with addresses on the same subnet.

MFC r266860

Fix unintended KBI change from r264905. Add _fib versions of
ifa_ifwithnet() and ifa_ifwithdstaddr() The legacy functions will call the
_fib() versions with RT_ALL_FIBS, preserving legacy behavior.

sys/net/if_var.h
sys/net/if.c
Add legacy-compatible functions as described above. Ensure legacy
behavior when RT_ALL_FIBS is passed as fibnum.

sys/netinet/in_pcb.c
sys/netinet/ip_output.c
sys/netinet/ip_options.c
sys/net/route.c
sys/net/rtsock.c
sys/netinet6/nd6.c
Call with _fib() functions if we must use a specific fib, or the
legacy functions otherwise.

tests/sys/netinet/fibs_test.sh
tests/sys/netinet/udp_dontroute.c
Improve the udp_dontroute test. The bug that this test exercises is
that ifa_ifwithnet() will return the wrong address, if multiple
interfaces have addresses on the same subnet but with different
fibs. The previous version of the test only considered one possible
failure mode: that ifa_ifwithnet_fib() might fail to find any
suitable address at all. The new version also checks whether
ifa_ifwithnet_fib() finds the correct address by checking where the
ARP request goes.

MFC r264917

Style fixes, mostly trailing whitespace elimination. No functional change.

MFC r264905

Fix subnet and default routes on different FIBs on the same subnet.

These two bugs are closely related. The root cause is that ifa_ifwithnet
does not consider FIBs when searching for an interface address.

sys/net/if_var.h
sys/net/if.c
Add a fib argument to ifa_ifwithnet and ifa_ifwithdstadddr. Those
functions will only return an address whose interface fib equals the
argument.

sys/net/route.c
Update calls to ifa_ifwithnet and ifa_ifwithdstaddr with fib
arguments.

sys/netinet/in.c
Update in_addprefix to consider the interface fib when adding
prefixes. This will prevent it from not adding a subnet route when
one already exists on a different fib.

sys/net/rtsock.c
sys/netinet/in_pcb.c
sys/netinet/ip_output.c
sys/netinet/ip_options.c
sys/netinet6/nd6.c
Add RT_DEFAULT_FIB arguments to ifa_ifwithdstaddr and ifa_ifwithnet.
In some cases it there wasn't a clear specific fib number to use.
In others, I was unable to test those functions so I chose
RT_DEFAULT_FIB to minimize divergence from current behavior. I will
fix some of the latter changes along with PR kern/187553.

tests/sys/netinet/fibs_test.sh
tests/sys/netinet/udp_dontroute.c
tests/sys/netinet/Makefile
Revert r263738. The udp_dontroute test was right all along.
However, bugs kern/187550 and kern/187553 cancelled each other out
when it came to this test. Because of kern/187553, ifa_ifwithnet
searched the default fib instead of the requested one, but because
of kern/187550, there was an applicable subnet route on the default
fib. The new test added in r263738 doesn't work right, however. I
can verify with dtrace that ifa_ifwithnet returned the wrong address
before I applied this commit, but route(8) miraculously found the
correct interface to use anyway. I don't know how.

Clear expected failure messages for kern/187550 and kern/187552.

MFC r263738

tests/sys/netinet/Makefile
tests/sys/netinet/fibs.sh
Replace fibs:udp_dontroute with fibs:src_addr_selection_by_subnet.
The original test was poorly written; it was actually testing
kern/167947 instead of the desired kern/187553. The root cause of the
bug is that ifa_ifwithnet did not have a fib argument. The new test
more directly targets that behavior.

tests/sys/netinet/udp_dontroute.c
Delete the auxilliary binary used by the old test

MFC r263445 and r265385. Note that the change to ObsoleteFiles.inc in
r265385 does not need to be MFCed, because the obsolete files in question
were never MFCed to any stable branch.

r265385
Remove the ifconfig test added in rev 263445. After discussion with
melifaro, we agreed that ifconfig's behavior was not a bug. The main
motivation for bin/187551 was to partially resolve kern/187549, but we
resolved kern/187549 in a different way instead.

ObsoleteFiles.inc
etc/mtree/BSD.tests.dist
sbin/ifconfig/tests/fibs_test.sh
sbin/ifconfig/tests/Makefile
sbin/ifconfig/Makefile
Remove /usr/tests/sbin/ifconfig

r263445
Add several ATF tests that deal with multiple fibs. They're described in
several different PRs, but the tests share some common code, so I'm
committing them together.

sbin/ifconfig/tests
sbin/ifconfig/tests/fibs_test.sh
sbin/ifconfig/tests/Makefile
sbin/ifconfig/Makefile
Add fibs_test.sh, which regresses bin/187551

tests/sys/netinet
tests/sys/netinet/fibs_test.sh
tests/sys/netinet/udp_dontroute.c
tests/sys/netinet/Makefile
tests/sys/Makefile
Add fibs_test.sh, which regresses kern/167947, kern/187552
kern/187549, kern/187550, and kern/187553

etc/mtree/BSD.tests.dist
Add newly created directories

MFC changes relating to running multiple interfaces on different fibs but
with addresses on the same subnet.

MFC r266860

Fix unintended KBI change from r264905. Add _fib versions of
ifa_ifwithnet() and ifa_ifwithdstaddr() The legacy functions will call the
_fib() versions with RT_ALL_FIBS, preserving legacy behavior.

sys/net/if_var.h
sys/net/if.c
Add legacy-compatible functions as described above. Ensure legacy
behavior when RT_ALL_FIBS is passed as fibnum.

sys/netinet/in_pcb.c
sys/netinet/ip_output.c
sys/netinet/ip_options.c
sys/net/route.c
sys/net/rtsock.c
sys/netinet6/nd6.c
Call with _fib() functions if we must use a specific fib, or the
legacy functions otherwise.

tests/sys/netinet/fibs_test.sh
tests/sys/netinet/udp_dontroute.c
Improve the udp_dontroute test. The bug that this test exercises is
that ifa_ifwithnet() will return the wrong address, if multiple
interfaces have addresses on the same subnet but with different
fibs. The previous version of the test only considered one possible
failure mode: that ifa_ifwithnet_fib() might fail to find any
suitable address at all. The new version also checks whether
ifa_ifwithnet_fib() finds the correct address by checking where the
ARP request goes.

MFC r264917

Style fixes, mostly trailing whitespace elimination. No functional change.

MFC r264905

Fix subnet and default routes on different FIBs on the same subnet.

These two bugs are closely related. The root cause is that ifa_ifwithnet
does not consider FIBs when searching for an interface address.

sys/net/if_var.h
sys/net/if.c
Add a fib argument to ifa_ifwithnet and ifa_ifwithdstadddr. Those
functions will only return an address whose interface fib equals the
argument.

sys/net/route.c
Update calls to ifa_ifwithnet and ifa_ifwithdstaddr with fib
arguments.

sys/netinet/in.c
Update in_addprefix to consider the interface fib when adding
prefixes. This will prevent it from not adding a subnet route when
one already exists on a different fib.

sys/net/rtsock.c
sys/netinet/in_pcb.c
sys/netinet/ip_output.c
sys/netinet/ip_options.c
sys/netinet6/nd6.c
Add RT_DEFAULT_FIB arguments to ifa_ifwithdstaddr and ifa_ifwithnet.
In some cases it there wasn't a clear specific fib number to use.
In others, I was unable to test those functions so I chose
RT_DEFAULT_FIB to minimize divergence from current behavior. I will
fix some of the latter changes along with PR kern/187553.

tests/sys/netinet/fibs_test.sh
tests/sys/netinet/udp_dontroute.c
tests/sys/netinet/Makefile
Revert r263738. The udp_dontroute test was right all along.
However, bugs kern/187550 and kern/187553 cancelled each other out
when it came to this test. Because of kern/187553, ifa_ifwithnet
searched the default fib instead of the requested one, but because
of kern/187550, there was an applicable subnet route on the default
fib. The new test added in r263738 doesn't work right, however. I
can verify with dtrace that ifa_ifwithnet returned the wrong address
before I applied this commit, but route(8) miraculously found the
correct interface to use anyway. I don't know how.

Clear expected failure messages for kern/187550 and kern/187552.

MFC r263738

tests/sys/netinet/Makefile
tests/sys/netinet/fibs.sh
Replace fibs:udp_dontroute with fibs:src_addr_selection_by_subnet.
The original test was poorly written; it was actually testing
kern/167947 instead of the desired kern/187553. The root cause of the
bug is that ifa_ifwithnet did not have a fib argument. The new test
more directly targets that behavior.

tests/sys/netinet/udp_dontroute.c
Delete the auxilliary binary used by the old test

MFC r263445 and r265385. Note that the change to ObsoleteFiles.inc in
r265385 does not need to be MFCed, because the obsolete files in question
were never MFCed to any stable branch.

r265385
Remove the ifconfig test added in rev 263445. After discussion with
melifaro, we agreed that ifconfig's behavior was not a bug. The main
motivation for bin/187551 was to partially resolve kern/187549, but we
resolved kern/187549 in a different way instead.

ObsoleteFiles.inc
etc/mtree/BSD.tests.dist
sbin/ifconfig/tests/fibs_test.sh
sbin/ifconfig/tests/Makefile
sbin/ifconfig/Makefile
Remove /usr/tests/sbin/ifconfig

r263445
Add several ATF tests that deal with multiple fibs. They're described in
several different PRs, but the tests share some common code, so I'm
committing them together.

sbin/ifconfig/tests
sbin/ifconfig/tests/fibs_test.sh
sbin/ifconfig/tests/Makefile
sbin/ifconfig/Makefile
Add fibs_test.sh, which regresses bin/187551

tests/sys/netinet
tests/sys/netinet/fibs_test.sh
tests/sys/netinet/udp_dontroute.c
tests/sys/netinet/Makefile
tests/sys/Makefile
Add fibs_test.sh, which regresses kern/167947, kern/187552
kern/187549, kern/187550, and kern/187553

etc/mtree/BSD.tests.dist
Add newly created directories