Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Remove zero-copy sockets code. It only worked for anonymous memory,
and the equivalent functionality is now provided by sendfile(2) over
posix shared memory filedescriptor.

Remove the cow member of struct vm_page, and rearrange the remaining
members. While there, make hold_count unsigned.

Requested and reviewed by: alc
Tested by: pho
Sponsored by: The FreeBSD Foundation
Approved by: re (delphij)

- Updated TOE support in the kernel.

- Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs.
These are available as t3_tom and t4_tom modules that augment cxgb(4)
and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as
usual with or without these extra features.

- iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the
works and will follow soon.

Build-tested with make universe.

30s overview
============
What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the
capabilities of an interface:
# ifconfig -m | grep TOE

Enable/disable TCP offload on an interface (just like any other ifnet
capability):
# ifconfig cxgbe0 toe
# ifconfig cxgbe0 -toe

Which connections are offloaded? Look for toe4 and/or toe6 in the
output of netstat and sockstat:
# netstat -np tcp | grep toe
# sockstat -46c | grep toe

Reviewed by: bz, gnn
Sponsored by: Chelsio communications.
MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)

Unbreak make depend.

MFp4 bz_ipv6_fast:

Allow LRO to work on IPv6 as well.
Fix the module Makefile to at least properly inlcude opt_inet6.h
and allow builds without INET or INET6.

Sponsored by: The FreeBSD Foundation
Sponsored by: iXsystems

Reviewed by: gnn (as part of the whole)
MFC After: 3 days

MFtbemd:

Use MACHINE_CPUARCH in preference to MACHINE_ARCH. The former is the
source code location of the machine, the latter the binary output. In
general, we want to use MACHINE_CPUARCH instead of MACHINE_ARCH unless
we're tesitng for a specific target. The isn't even moot for
i386/amd64 where there's momemntum towards a MACHINE_CPUARCH == x86,
although a specific cleanup for that likely would be needed...

There is no need to test __FreeBSD_version for features that have
been around for a long time now (7.1-ish or even earlier); assume
they are present. These includes MSI, TSO, LRO, VLAN, INTR_FILTERS,
FIRMWARE, etc.

Also, eliminate some dead code and clean up in other places as part
of this quick once-over.

MFC after: 1 week

cxgb(4) updates, including:
- support for the new Gen-2, BT, and LP-CR cards.
- T3 firmware 7.7.0
- shared "common code" updates.

Approved by: gnn (mentor)
Obtained from: Chelsio
MFC after: 1 month

Greatly simplify cxgb by removing almost all of the custom mbuf management logic

- remove mbuf iovec - useful, but adds too much complexity when isolated to
the driver

- remove driver private caching - insufficient benefit over UMA to justify
the added complexity and maintenance overhead

- remove separate logic for managing multiple transmit queues, with the
new drbr routines the control flow can be made to much more closely resemble
legacy drivers

- remove dedicated service threads, with per-cpu callouts one can get the same
benefit much more simply by registering a callout 1 tick in the future if there
are still buffered packets

- remove embedded mbuf usage - Jeffr's changes will (I hope) soon be integrated
greatly reducing the overhead of using kernel APIs for reference counting
clusters

- add hysteresis to descriptor coalescing logic

- add coalesce threshold sysctls to allow users to decide at run-time
between optimizing for forwarding / UDP or optimizing for TCP

- add once per second watchdog to effectively close the very rare races
occurring from coalescing

- incorporate Navdeep's changes to the initialization path required to
convert port and adapter locks back to ordinary mutexes (silencing BPF
LOR complaints)

- enable prefetches in get_packet and tx cleaning

Reviewed by: navdeep@
MFC after: 2 weeks

After r193232 rt_tables in vnet.h are no longer indirectly dependent on
the ROUTETABLES kernel option thus there is no need to include opt_route.h
anymore in all consumers of vnet.h and no longer depend on it for module
builds.

Remove the hidden include in flowtable.h as well and leave the two
explicit #includes in ip_input.c and ip_output.c.

Remove opt_mac.h generation for various kernel modules that no longer
require it.

Submitted by: pjd

This main goals of this project are:
1. separating L2 tables (ARP, NDP) from the L3 routing tables
2. removing as much locking dependencies among these layers as
possible to allow for some parallelism in the search operations
3. simplify the logic in the routing code,

The most notable end result is the obsolescent of the route
cloning (RTF_CLONING) concept, which translated into code reduction
in both IPv4 ARP and IPv6 NDP related modules, and size reduction in
struct rtentry{}. The change in design obsoletes the semantics of
RTF_CLONING, RTF_WASCLONE and RTF_LLINFO routing flags. The userland
applications such as "arp" and "ndp" have been modified to reflect
those changes. The output from "netstat -r" shows only the routing
entries.

Quite a few developers have contributed to this project in the
past: Glebius Smirnoff, Luigi Rizzo, Alessandro Cerri, and
Andre Oppermann. And most recently:

- Kip Macy revised the locking code completely, thus completing
the last piece of the puzzle, Kip has also been conducting
active functional testing
- Sam Leffler has helped me improving/refactoring the code, and
provided valuable reviews
- Julian Elischer setup the perforce tree for me and has helped
me maintaining that branch before the svn conversion

Rather than using hidden includes (with cicular dependencies),
directly include only the header files needed. This reduces the
unneeded spamming of various headers into lots of files.

For now, this leaves us with very few modules including vnet.h
and thus needing to depend on opt_route.h.

Reviewed by: brooks, gnn, des, zec, imp
Sponsored by: The FreeBSD Foundation

Fix world

Approved by: kmacy

Add opt_inet.h which has been needed since r184714, r184715 introducing
checks for #ifdef INET.

Submitted by: kmacy (r184876, I splitted lines)
MFC after: 54 days

- Remove default NIC dependency on ulp headers
- make toe module build dependent on kernel support

Submitted by: Chelsio Inc.
MFC after: 1 week

Update cxgb include paths to not require prefixing with dev/cxgb

Submitted by: Chelsio Inc.

Remove cxgb private lro implementation and switch to using system implementation.

Obtained from: Chelsio Inc.
MFC after: 1 week

import vendor fixes to cxgb

only build iw_cxgb on i386 and amd64

add iw_cxgb to the build

move cxgb_lt2.[ch] from NIC to TOE
move most offload functionality from NIC to TOE
factor out all socket and inpcb direct access
factor out access to locking in incpb, pcbinfo, and sockbuf

Add KMOD target.

Discovered by: Takahashi Yoshihiro

move remaining binaries in to blob headers

Move firmware in to separate module that can be compiled statically in to the kernel
Add utility for converting future firmware revs to a C header file

- update firmware to 5.0
- add support for T3C
- add DDP support (zero-copy receive)
- fix TOE transmit of large requests
- fix shutdown so that sockets don't remain in CLOSING state indefinitely
- register listeners when an interface is brought up after tom is loaded
- fix setting of multicast filter
- enable link at device attach
- exit tick handler if shutdown is in progress
- add helper for logging TCB
- add sysctls for dumping transmit queues

- note that TOE wxill not be MFC'd until after 7.0 has been finalized

MFC after: 3 days

add opt_global.h dependency

add cxgb_multiq.c to the build

Don't overload tcp_usrreqs unless the kernel doesn't provide offload support.

Make TCP offload work on HEAD (modulo negative interaction between sbcompress
and t3_push_frames).
- Import latest changes to cxgb_main.c and cxgb_sge.c from toestack p4 branch
- make driver local copy of tcp_subr.c and tcp_usrreq.c and override tcp_usrreqs so
TOE can also functions on versions with unmodified TCP

- add cxgb back to the build

Makefile for the TCP offload module

Don't use old style mbuf iovec interface

Move NIC driver (cxgb) into its own directory. Add toecore to support
registering different TOMs.

- fix qset to port binding as a proper fix for the problems encountered on the 4-port
- fix the use after free seen when sending packets small enough to fit as an immediate
and bpf peers are present
- update to firmware rev 4.7 along with various small vendor fixes

Supported by: Chelsio
Approved by: re (blanket)
MFC after: 3 days

- integrate most recent changes from vendor branch and upgrade to firmware revision 4.5.5
- add filter support
- further improvements for T304
- recover gracefully from spurious immediate packets

Approved by: re(blanket)
Supported by: Chelsio
MFC after: 3 days

- import new common code for the T304
- update to firmware version 4.1.0

- switch over to standard method for initializing cdevs (contributed by scottl@)
- break out timer_reclaim_task to be per-port
- move msix teardown into separate function
- fix bus_setup_intr for msi-x for the multi-port case so that msi-x resources
are not corrupted on unload
- handle 10/100/1000 base-T media and auto negotiation
- bind qset to cpu even for singleq case
- white space cleanups
- remove recursive PORT_LOCK
- move mtu setting to separate function
- stop and re-init port when changing mtu
- replace all direct references to m_data with calls to mtod
- handle attach failure better by not trying to de-initialize
taskqueues when they have not been allocated
- no longer default to jumbo frames

Sponsored by: Chelsio
MFC after: 3 days

When building cxgb as a module make include paths relative to the driver's root.
This will make it possible to build the module out of tree against an older src tree.

MFC after: 3 days

(MFp4)
- upgrade to reflect state of 1.0.0.86
- move from firmware rev 3.2 to 4.0.0
- import driver bits for offload functionality
- remove binary distribution clause from top level files as it
runs counter to the intent of purely supporting the hardware

MFC after: 3 days

hook uipc_mvec.c into build

Remove firmware version flags

MFC after: 3 days

Disable linking in of firmware on ia64 to avoid build failures from a
broken ld.

no-op cxgb_config.h for the module compilation case

Add modules Makefile for cxgb