ix(4): Move {mod,msf,mbx,fdir,phy,link}_task to lock protected handler

This patch introduces adapter->task_requests register responsible for recording
requests for mod_task, msf_task, mbx_task, fdir_task, phy_task and link_task
calls. Instead of enqueueing each of these tasks with GROUPTASK_ENQUEUE, new
task is created and all handlers are called from one task while holding
adapter->core_mtx lock.

SIOCGIFXMEDIA ioctl() call reads adapter->media list. The list is deleted and
rewritten in ixgbe_handle_msf() task without holding adapter->core_mtx lock.
This change is needed to maintain data coherency when sharing adapter info via
ioctl() calls.

Since handlers for abovementioned tasks will no longer act as task handlers,
but as regular functions, 'pending' parameter is removed from them.

This patch also removes ixgbe_update_link_status() call from
ixgbe_handle_link() handler. From now on, link status will be updated by
calling ixgbe_update_link_status() periodically from ixgbe_local_timer(). This
fixes problem with link flapping during changing interface state to UP.

Parameter keep_traffic is added to ixgbe_disable_intr(). This enables
ixgbe_handle_admin_task() to not disable and queue interrupts. Accordingly,
skip_traffic parameter is added to ixgbe_enable_intr() to let
ixgbe_handle_admin_task() skip enabling queues while enabling interrupts.

This patch is a port of r343621. r343621 can't be merged from current since
stable/11 contains ixgbe driver without iflib support.

Patch co-authored by Krzysztof Galazka <krzysztof.galazka@intel.com>.

Submitted by: Piotr Pietruszewski <piotr.pietruszewski@intel.com>
Reviewed by: #IntelNetworking
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D19711

Revert r330897:

This was intended to be a non-functional change. It wasn't. The commit
message was thus wrong. In addition it broke arm, and merged crypto
related code.

Revert with prejudice.

This revert skips files touched in r316370 since that commit was since
MFCed. This revert also skips files that require $FreeBSD$ property
changes.

Thank you to those who helped me get out of this mess including but not
limited to gonzo, kevans, rgrimes.

Requested by: gjb (re)

Partial merge of the SPDX changes

These changes are incomplete but are making it difficult
to determine what other changes can/should be merged.

No objections from: pfg

MFC r316541, r316544, and r320688 from HEAD to stable/11.

r316541:
Fix a double free in ixgbe_rxeof()

r316544:
Don't overwrite vf->flags variable at the end of ixgbe(4) ixgbe_add_vf()

r320688:
ixgbe(4): Update HEAD (p3) to 3.2.12-k

Includes:

- Support for X550EM devices.
- Support for Bypass adapters.
- Flow Director code moved to separate files
- SR-IOV code moved to separate files
- Netmap code moved to separate files

Sponsored by: Intel Corporation

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

ixgbe sysctl hardware defaults

Added hw.ix.flow_control which enables the default flow_control of all ix
interfaces to be set in loader.conf.

Added hw.ix.advertise_speed which enables the default advertised_speed of
all ix interfaces to be set in loader.conf.

Made enable_aim device independent based on hw.ix.enable_aim default.

Reviewed by: erj
MFC after: 1 week
Sponsored by: Multiplay
Differential Revision: https://reviews.freebsd.org/D5060

ixgbe(4): Update to version 3.1.13-k

Add support for two new devices: X552 SFP+ 10 GbE, and the single port
version of X550T.

Submitted by: erj
Reviewed by: gnn
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D4186

Add support for sysctl knobs to live tune the per interrupt rx/tx packet
processing limits in ixgbe(4)

Differential Revision: https://reviews.freebsd.org/D3719
Submitted by: jason wolfe (j-nitrology.com)
MFC after: 2 weeks

Add netmap support for ixgbe SRIOV VFs (that is, to if_ixv).

Differential Revision: https://reviews.freebsd.org/D2923
Reviewed by: erj, gnn
Approved by: jmallett (mentor)
Sponsored by: Norse Corp, Inc.

Catch up to the SRIOV API changes in r283670.

Delta D2489 - Add SRIOV support to the Intel 10G driver.

NOTE: This is a technology preview, while it has undergone
development testing, Intel has not yet completed full
validation of the feature. It is being integrated for
early access and customer testing.

Revert last commit, to remove added skeleton tree.

Delta D2489 - Add SRIOV support to the Intel 10G driver.

NOTE: This is a technology preview, while it has undergone development
tests, Intel has not yet completed full validation of the feature.
It is being integrated for early access and customer testing.

Add support for certain Intel X550 devices.

These include standalone X550 adapters, X552 10GbE backplane, and
X552/X557-AT 10GBASE-T; with the latter two being integrated into Xeon D SoCs.

As well, this bumps the ixgbe version number to 2.8.3, and includes updates
to shared code for support for the new devices.

Differential Revision: D2414
Reviewed by: gnn, adrian
Approved by: jfv (mentor), gnn (mentor)

Fix i386 LINT build issues, and remove unused variable.

Update to the Intel ixgbe driver:
- Split the driver into independent pf and vf loadables. This is
in preparation for SRIOV support which will be following shortly.
This also allows us to keep a seperate revision control over the
two parts, making for easier sustaining.
- Make the TX/RX code a shared/seperated file, in the old code base
the ixv code would miss fixes that went into ixgbe, this model
will eliminate that problem.
- The driver loadables will now match the device names, something that
has been requested for some time.
- Rather than a modules/ixgbe there is now modules/ix and modules/ixv
- It will also be possible to make your static kernel with only one
or the other for streamlined installs, or both.

Enjoy!

Submitted by: jfv and erj

Pad RX copy alignment calculation to avoid illegal memory accesses

The optimization made in r239940 is valid for struct mbuf's current structure
and size in FreeBSD, but hardcodes assumptions about sizes of struct mbuf,
which are unfortunately broken if additional data is added to the beginning of
struct mbuf

X-MFC note (discussed with rwatson):

This change requires the MPKTHSIZE definition, which is only available after
head@r277203 and will not be MFCed as it breaks mbuf(9) KPI.

A direct commit to stable/10 and merges to other branches to add the necessary
definitions to work with the code as-is will be done to facilitate this MFC

PR: 194314
MFC after: 2 weeks
Approved/Reviewed by: erj, jfv
Sponsored by: EMC / Isilon Storage Division

The error bits are not valid with EOP=0; so intermediary fragments should
not be discarded.

Submitted by: Marc De La Gueronniere <mdelagueronniere@verisign.com>
MFC after: 1 week
Sponsored by: Verisign, Inc.

* Add SIOCGI2C driver ioctl used to retrieve i2c info.
* Convert ixgbe to use this ioctl
* Convert ifconfig to use generic i2c handler for "ix" interfaces.

Approved by: Eric Joyner (ixgbe part)
MFC after: 2 weeks
Sponsored by: Yandex LLC

The r48589 promised to remove implicit inclusion of if_var.h soon. Prepare
to this event, adding if_var.h to files that do need it. Also, include
all includes that now are included due to implicit pollution via if_var.h

Sponsored by: Netflix
Sponsored by: Nginx, Inc.

Add quad port probe support, this gives the admin proper information about the slot
(which should be a PCIE Gen 3 slot for this adapter) by looking back thru the PCI
parent devices to the slot device.

The fix above also corrects the bandwidth display to GT/s rather than the
incorrect Gb/s

Next, allow the use of ALTQ if you select the compile option IXGBE_LEGACY_TX.

Allow the use of 'unsupported' optic modules by a compile option as well.

Add a phy reset capability into the stop code, this is so a static configured
driver will still behave properly when taken down (not being able to unload it).

This revision synchronizes the shared code with Intel internal current code,
and note that it now includes DCB supporting code, this was necessitated by
some internal changes with the code, but it also will provide the opportunity
to develop this feature in the core driver down the road.

I have edited the README to get rid of some of the worse anachronisms in it
as well, its by no means as robust as I might wish at this point however.

Oh, I also have included some conditional stuff in the code so it will be
compatible in both the 9.X and 10 environments.

Performance has been a focus in recent changes and I believe this revision
driver will perform very well in most workloads.

MFC after: 2 weeks

rename the 'tag' and 'map' fields used the rx ring to their
previous names, 'ptag' and 'pmap' -- p stands for packet.

This change reduces the difference between the code in stable/9
and head, and also helps using the same ixgbe_netmap.h on both branches.

Approved by: Jack Vogel

Patch #11 - The final patch: this one greatly improves the
TX hot path by getting rid of index calculations and simply
managing pointers. Much of the creative code is due to my
coworker here at Intel, Alex Duyck, thanks Alex!

Also, this whole series of patches was given the critical
eye of Gleb Smirnoff and is all the better for it, thanks
Gleb!

Patch #8 Performance changes - this one improves locality,
moving some counters and data to the ring struct from
the adapter struct, also compressing some data in the
move.

Patch #7 This is primarily about processing limit control.
- add a limit for both RX and TX, change the default to 256
- change the sysctl usage to be common, and now to be called
during init for each ring.
- the TX limit is not yet used, but the changes in the last
patch in this series uses the value.
- the motivation behind these changes is to improve data
locality in the final code.
- rxeof interface changes since it now gets limit from the
ring struct

Patch #6 Whitespace cleanup, and removal of some very old
defines (at Gleb's request). Also, change the defines around
the old transmit code to IXGBE_LEGACY_TX, I do this to make
it possible to define this regardless of the OS level (it is
not defined by default). There are also a couple changed
comments for clarity.

Patch #5 Cleanup unused IEEE1588 code fragments, the day may
come when this feature gets implemented, but its not here yet
and I see no reason to leave this laying around.

Patch #3 - Add a new ioctl to access SFP+ module diagnostic
data via the I2C routines in shared code.

Patch #2 - remove OACTIVE and DEPLETED notions from the
multiqueue code, this functionality has proven to be more
trouble than it was worth. Thanks to Gleb for a second
critical look over my code and help in the patches!

First of a series of 11 patches leading to new ixgbe version 2.5.0
This removes the header split and supporting code from the driver.

Merge similar fixes from 223198 from igb to ixgbe:
- Use a dedicated task to handle deferred transmits from the if_transmit
method instead of reusing the existing per-queue interrupt task.
Reusing the per-queue interrupt task could result in both an interrupt
thread and the taskqueue thread trying to handle received packets on a
single queue resulting in out-of-order packet processing and lock
contention.
- Don't define ixgbe_start() at all where if_transmit is used.

Tested by: Vijay Singh
Reviewed by: jfv
MFC after: 2 weeks

Heavily optimize the case of small RX packets of 160 bytes or less. For
this case, allocate a plain mbuf and copy the frame into it, then send the
copy up the stack, leaving the original mbuf+cluster in place in the
receive ring for immediate re-use. This saves a trip through 2 of the
3 zones of the compound mbuf allocator, a trip through busdma, and a trip
through the 1 of the 3 mbuf destructors. For our load at Netflix, this can
lower CPU consumption by as much as 20%. The copy algorithm is based on
investigative work from Luigi Rizzo earlier in the year.

Reviewed by: jfv
Obtained from: Netflix

If we pass down 64k - L2 hdr size + 1 to 64K L3+ data adding an ether
header will make the data go over the 64k limits announced to busdma as
maxsize and the transaction will fail.

With TSO this can result in a TCP regression due to the lost packet.

According to the data sheets ixgbe(4) 82598 and 82599 can handle up to
256k so increase the maximum.

Reported by: Jon Kåre Hellan, UNINETT (jon.kare.hellan uninett.no)
Tested by: Jon Kåre Hellan, UNINETT (jon.kare.hellan uninett.no)
MFC after: 1 week

New hardware support: Intel X540 adapter support added.
Some shared code reorganization along with the new adapter.
Sync changes to OACTIVE in igb into this driver.
Misc small fixes.

Add a #define for driver portability to older OS

- Add the RX refresh changes from igb to ixgbe
- Also a couple minor tweaks to the TX code from the same source.
- Add the INET ioctl code which has been missing from this driver,
and which caused IP aliases to reset the interface.
- Last, some minor logic changes that just reflect upcoming
hardware support, but have no other functional effect now.

MFC after a week

Update driver to version 2.3.8:
CRITICAL FIX - with stats changes the older 82598 will panic
and trash the stack on driver load, FCOE registers ONLY exist
in 82599 and must not be read otherwise.

kern/153951 - to correct incorrect media type on adapters
with pluggable modules I have eliminated the old static
table in favor of a new dynamic shared code routine. This
also has the benefit of detecting changes when a different
module is inserted.

Performance/enhancement to the Flow Director code from my
linux coworker (the developer of the code).

Fixes from Michael Tuexen - a data corruption problem on the
82599 (CRITICAL), fix so the buf size correctly adjusts as
the cluster changes, and max descriptors are set properly.
Also added 16K clusters for those REALLY big jumbos :)

In the RX path, the RX LOCK was not being released, and this
causes LOR problems. Add the code that igb already has.

Sync with in house shared code, this was necessary for the
Flow Director fix.

MFC in 2 days

Fix to kern/150247 - make ixgbe buildable for 7.x

and the header...

Update ixgbe driver to verion 2.3.6
- This adds a VM SRIOV interface, ixv, it is however
transparent to the user, it links with the ixgbe.ko,
but when ixgbe is loaded in a virtualized guest with
SRIOV configured this will be detected.
- Sync shared code to latest
- Many bug fixes and improvements, thanks to everyone
who has been using the driver and reporting issues.

Do not allocate multicast array memory in multicast filter
configuration function. For failed memory allocations, em(4)/lem(4)
called panic(9) which is not acceptable on production box.
igb(4)/ixgb(4)/ix(4) allocated the required memory in stack which
consumed 768 bytes of stack memory which looks too big.

To address these issues, allocate multicast array memory in device
attach time and make multicast configuration success under any
conditions. This change also removes the excessive use of memory in
stack.

Reviewed by: jfv

Fix build

Left out header change in last delta - new member
in adapter so that advertise changes can be done
to one port without the other changing.

Fix build.

Fixes for panic experienced in test at Intel, when
doing bidirectional stress traffic on 82598.

Also a couple bug fixes from Michael Tuexen, thank you!!

Add a workaround into the header so that 8 REL can use
the driver (adds local copy of ALTQ fix).

MFC: in a few days

A few changes:
When not defining header split do not allocate mbufs,
this can be a BIG savings in the mbuf memory pool.

Also keep seperate dma maps for the header and
payload pieces when doing header split. The basis
of this code was a patch done a while ago by
yongari, thank you :)

A number of white space changes.

MFC: in a few days

Thanks to Michael Tuexen for adding SCTP support for 82599,
also for finding a one character bug that kept TSO from working.

Sometimes with direct attach cables a failure can occur in init,
the old method of calling detach was broken, there is no way to
return an error to the system from init, so I have changed it to
return failure thru the ioctl.

And, have fixed the ALTQ code changes of Max Laier, sorry Max :)

Update the driver to Intel version 2.1.6
- add some new hardware support for 82599
- Big change to interrupt architecture, it now
uses a queue which contains an RX/TX pair as
the recipient of the interrupt. This will reduce
overall system interrupts/msix usage.
- Improved RX mbuf handling: the old get_buf routine
is no longer synchronized with rxeof, this allows
the elimination of packet discards due to mbuf
allocation failure.
- Much simplified and improved AIM code, it now
happens in the queue interrupt context and takes
into account both the traffic on the RX AND TX
side.
- variety of small tweaks, like ring size, that have
been seen as performance improvements.
- Thanks to those that provided feedback or suggested
changes, I hope I've caught all of them.

Update driver to Intel version 2.0.7:

This adds new feature support for the 82599, a hardware
assist to LRO, doing this required a large revamp to the
RX cleanup code because the descriptor ring may not be
processed out of order, this necessitated the elimination
of global pointers.

Additionally, the RX routine now does not refresh mbufs
on every descriptor, rather it will do a range, and then
update the hardware pointer at that time. These are
performance oriented changes.

The TX side now has a cleaner simpler watchdog algorithm
as well, in TX cleanup a read of ticks is stored, that
can then be compared in local_timer to determine if
there is a hang.

Various other cleanups along the way, thanks to all who
have provided input and testing.

Update for the Intel 10G driver, this adds support for
newest hardware, adds multiqueue tx interface, infrastructure
cleanup to allow up to 32 MSIX vectors on newer Nehalem systems.
Bug fixes, etc.

Update the ixgbe driver to 1.7.4, this includes support
for the new 82599 adapter family, adds header split, and
many small fixes. The driver should now be added to the
GENERIC kernel.

MFC after: 2 weeks

Updated ixgbe driver - version 1.6.2

-This version has header split, and as a result a number of
aspects of the code have been improved/simplified.
- Interrupt handling refined for performance
- Many small bugs fixed along the way

MFC after: ASAP - in time for 7.1

This updates the ixgbe driver to Intel internal version 1.4.7
Shared code changes, core driver fixes, vlan event/filter support

Also Kip Macy's fix to allow any number of queues, thanks Kip!

Change include to reflect tcp_lro.h move.

This is driver version 1.4.4 of the Intel ixgbe driver.
-It has new hardware support
-It uses a new method of TX cleanup called Head Write Back
-It includes the provisional generic TCP LRO feature contributed
by Myricom and made general purpose by me. This should move into
the stack upon approval but for this driver drop its in here.
-Also bug fixes and etc...

MFC in a week if no serious issues arise.

This is an update to the new Intel 10G 82598 driver.
The first drop was Beta, this code is expected to be the release version.
Note that this driver code will build in either 6.2 or 7. If you
use the code in 6.2 you will not get TSO or MSI/X support but it will
function in a legacy mode.

Approved by: re

New Driver for Intel 10G PCI-Express Dual Port Network adapter (82598)
- ixgbe.c, ixgbe.h, and ixgbe_osdep.h make the core OS driver.
- Others are OS independent shared code.
- This is still a Beta driver, the Intel ship is a month or so off,
so I welcome any feedback from early users.
Approved by: re

Add the actual source too :)

Approved by: re