MFC: r320916

Reset unsupported SFP tuneable back to original entry name.

Unbreak netmap(4) support in ixgbe(4) after r315333:
- Both ixgbe_netmap.c and ixv_netmap.c assumed a netmap(4) driver
newer than what's actually in stable/10.
- Additionally, at the bottom line ixv_netmap.c did exactly the same
as ixgbe_netmap.c, i. e. used IXGBE_TDH() as appropriate for PFs
only instead of IXGBE_VFTDH() and tried to configure CRC stripping
although the corresponding registers aren't available to VFs in the
first place.

With these changes, the netmap(4) support in ixgbe(4) is in line
again with the code in sys/dev/netmap/ixgbe_netmap.h as of r295008.
Breakage reported by: Slawa Olhovchenkov

Just like r315333 that never existed in head, this is a direct commit
to stable/10. However, ixgbe(4) in head has a related bug in that it
assumes a netmap(4) driver API older than what's in head and also
does the wrong things for VFs as it uses the PF-only ixgbe_netmap.c
for both PFs and VFs in the first place.

ixv(4): Fix more tinderbox builds by adding missing declarations.

Sponsored by: Intel Corporation

ixgbe(4): Fix build issue when compiling with netmap enabled

Sponsored by: Intel Corporation

ixgbe(4): Re-add mutex lock call that was dropped in a previous commit.

PR: 218062
Reported by: Terry Kennedy <terry-freebsd@glaver.org>
Sponsored by: Intel Corporation

ixgbe(4): Fix VF build failure on i386 caused by r315333

SYSCTL_ADD_UQUAD on i386 expects a uint64_t pointer, and not an
unsigned long pointer. So change the data type of the original variable to
u64 (uint64_t) from unsigned long, since the statistics are intended to be
64 bits, anyway.

Reported by: Peter Jeremy (peter@rulingia.com)

ixgbe(4): Update to 3.2.11-k

Includes:

1. Support for X553 (Denverton) Ethernet devices.
2. Initialization code refactoring.
3. ixgbe-specific netmap code moved to ixgbe folder
(but not removed from dev/netmap/).
4. VF driver full split. Some cleanup can be had due to redundant code,
but this split goes inline with the refactoring.

NOTE: This is a direct commit to stable/10. Newer branches will receive an
iflib-converted version of ixgbe, with these updates included, instead.

Differential Revision: https://reviews.freebsd.org/D9310
Submitted by: Jeb Cramer (jeb.j.cramer@intel.com)
Tested by: Jeffrey Pieper (jeffrey.e.pieper@intel.com)
Sponsored by: Intel Corporation

MFC r312544

Fix reference to free memory in ixgbe/if_media.c

When ixgbe receives an interrupt indicating that a new optical module
may have been inserted, it discards all of its current media types
by calling ifmedia_removeall() and then creates a new set of media
types for the supported media on the new module. However,
ifmedia_removeall() was maintaining a pointer to whatever the
current media type was before the call to ifmedia_removealL().
The result of this was that any attempt to read the current media
type of the interface (e.g. via ifconfig) would return potentially
garbage data from free memory (or if one were particularly unlucky
on an architecture that does not malloc() from a direct map, page
fault the kernel).

Fix this by NULL'ing out the current media field in if_media.c,
and have ixgbe update the current media type after recreating
them.

Submitted by: Matt Joras <matt.joras AT gmail DOT com>
Reviewed by: sbruno, erj
MFC after: 1 week
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D9164

MFC r303032
Fixup DA cable detection routines to not set the cable type to
unknown if they do not match one of two cable types.

PR: 150249

MFC - r302384 to 10-STABLE
Do not initialize the adapter on MTU change when adapter status is down.
This fixes long-standing problems when changing settings of the adapter.

Discussed in:
https://lists.freebsd.org/pipermail/freebsd-net/2016-June/045509.html

Differential Revision: https://reviews.freebsd.org/D7254

MFC r298224

Correct possible underflow conditions when checking for available space
in the tx h/w ring buffer.

Sponsored by: Limelight Networks

MFC r296922:

Prevent invalid ixgbe advertise setting warning

Sponsored by: Multiplay

Spell "maximum" correctly.

noted by mav@

MFC r295273:
In FreeBSD 10 and higher the driver announces SCTP checksum offloading support
also for 82598, which doesn't support it.
The legacy code has a check for it, which was missed when the code for dealing with
CSUM_IP6_* was added. Add the same check for FreeBSD 10 and higher.

Approved by: re (marius)
Differential Revision: D5192

MFC r294795

ixgbe sysctl hardware defaults

Approved by: re (marius)
Sponsored by: Multiplay

MFC r294734, r295093 & r295094 ixgbe fixes

MFC r294734
Fix ixgbe compliation with DBG 1

MFC r295093
Configure ixgbe phy & gbic power

MFC r295094
Fix ixgbe flow control autoneg reporting

Approved by: re (marius)
Sponsored by: Multiplay

Revert sysctl description change introduced in r295008 to correctly
reflect the maximum number of queues supported (8) by this driver.

Submitted by: jwd
Reviewed by: erj
Approved by: re (gjb)
Sponsored by: Intel Corporation and Limelight Networks

MFC r292674
Update ixgbe(4) to Intel FreeBSD Networking Group version 3.1.13-k.

MFC r292571 and r292697
Add support for X552 and X550T.

MFC r293334
Fix SFP module insertion post boot.

MFC r293338
Fix VF handling of VLANs for Amazon Cloud.

Reviewed by: erj smh ngie jeffrey.e.pieper@intel.com
Approved by: re (marius)
Relnotes: Yes
Sponsored by: Intel Corporation and Limelight Networks
Differential Revision: https://reviews.freebsd.org/D5117

MFC r294578:

Fix ix advertise value after media change

Approved by: re (gjb)
Sponsored by: Multiplay

Fixed up version of r294061 that was reverted due to breakage of features
(netmap) and architectures(i386). <I'll take the pointyhat on that one>

r283883
-- update to 3.1.0

r283893
-- update SRIOV API changes related to future possible MFC of SRIOV work

r285590
-- Fix ixgbe(4) SRIOV VF initialization bugs

r285591
-- Remove version check for FLOWID

r285592
-- Update netmap support for ixgbe SRIOV VFs.

r286238
-- Fixup MTU zeroing if INET/INET6 are undefined.

Submitted by: kevin bowling <kevin.bowling@kev009.com>
Differential Revision: https://reviews.freebsd.org/D4273

Revert r294061

A number of pieces needed (netmap(4)) etc are missing, i.e. the MFCs
were incomplete.

Revert the changes so they can be redone cleanly again and to unbreak GENERIC,
LINT, etc

Sponsored by: EMC / Isilon Storage

Multiple MFC for ixgbe -- v 3.1.0

r283883
-- update to 3.1.0

r283893
-- update SRIOV API changes related to future possible MFC of SRIOV work

r285590
-- Fix ixgbe(4) SRIOV VF initialization bugs

r285591
-- Remove version check for FLOWID

r285592
-- Update netmap support for ixgbe SRIOV VFs, needs ixgbe_netmap.h merge

r286238
-- Fixup MTU zeroing if INET/INET6 are undefined.

Submitted by: kevin bowling (kevin.bowling@kev009.com)
Reviewed by: smh
Relnotes: Yes
Sponsored by: Limelight Networks
Differential Revision: https://reviews.freebsd.org/D4273

MFC r289238

Add support for sysctl knobs to live tune the per interrupt rx/tx packet
processing limits in ixgbe(4)

Submitted by: jason wolfe (j-nitrology.com)
Sponsored by: Limelight Networks

MFC r286238

A misplaced #endif in ixgbe_ioctl() causes interface MTU to become
zero when INET and INET6 are undefined.

PR: 162028
Submitted by: hoomanfazaeli@gmail.com pluknet

MFC r270064,r270068,r270069,r270115,r270129,r270287,r270822,r271014,
r271524,r273541,r282967,r283009,r283364.

Add support for reading i2c SFP/SFP+ data from NIC driver and
presenting most interesting fields via ifconfig -v.
This version supports Intel ixgbe driver only.

Tested on: Cisco,Intel,Mellanox,ModuleTech,Molex transceivers

* Add new net/sff8436.h containing constants used to access
QSFP+ data via i2c inteface. These constants has been taken
from SFF-8436 "QSFP+ 10 Gbs 4X PLUGGABLE TRANSCEIVER" standard
rev 4.8.
* Add support for printing QSFP+ information from 40G NICs
such as Chelsio T5.

Example:
cxl1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,.....>
ether 00:07:43:28:ad:08
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet 40Gbase-LR4 <full-duplex>
status: active
plugged: QSFP+ 40GBASE-LR4 (MPO Parallel Optic)
vendor: OEM PN: OP-QSFP-40G-LR4 SN: 20140318001 DATE: 2014-03-18
module temperature: 64.06 C voltage: 3.26 Volts
lane 1: RX: 0.47 mW (-3.21 dBm) TX: 2.78 mW (4.46 dBm)
lane 2: RX: 0.20 mW (-6.94 dBm) TX: 2.80 mW (4.47 dBm)
lane 3: RX: 0.18 mW (-7.38 dBm) TX: 2.79 mW (4.47 dBm)
lane 4: RX: 0.90 mW (-0.45 dBm) TX: 2.80 mW (4.48 dBm)

Tested on: Chelsio T5
Tested on: Mellanox/Huawei passive/active cables/transceivers.

Sponsored by: Yandex LLC

Partial MFC of r285528 as full RSS support is not available in FreeBSD 10.
Expose full 32bit RSS hash from card regardless of whether RSS is defined or
not. When doing multiqueue, we are all setup to have full 32bit RSS hash from
the card. We do not need to hide that under "ifdef RSS" and should expose that
by default so others like lagg(4) can use that and avoid hashing the traffic by
themselves.

Approved by: re (gjb)
Sponsored by: Limelight Networks

Fix a typo in r285668. Replace hw.ixgbe.* with correct hw.ix.* for a couple of
sysctls.

Approved by: re (gjb)
Sponsored by: Limelight Networks

Loader tunable fetching has been broken on stable/10 since ix(4) rewrite
introduced by r280182. FreeBSD-head doesn't need TUNABLE_INT() now with
SYSCTL_INT() but stable/10 still does.
Note: This is a direct commit to stable/10.

PR: 201644
Reviewed by: erj
Approved by: re (gjb)
Sponsored by: Limelight Networks

Re-add if_hw_tso* assignments to if_ix.c that were removed in r283668.

These are used in stable/10 as well as 11.

Requested by: rmacklem@

Limit the number of autoconfigured queues to 8.

This limit was in a previous version of the driver, but it is being re-
added to match the behavior of previous versions of 10. It prevents the
driver from using too many MSI-X vectors on systems with a large number of
logical CPU cores.

Thanks to <jwd@slowblink.com> for bringing up this issue.

Approved by: jfv (mentor)

Fix build issue and incompatibilites introduced by r283620.

- Re-introduce "device ixgbe" option for kernel configs. This allows
users to continue using existing kernel configurations, while still
letting if_ix and if_ixv exist to allow ifconfig magic to work.
Though the amd64 GENERIC config has "device ix" and "device ixv" instead of
"device ixgbe".

This is achieved by making the if_ixgbe module a stub that depends
on if_ix and if_ixv.

- Replace if_baudrate assignment with if_initbaudrate() call.

Thanks to jhb@ for the stub module suggestion and pointing out the if_baudrate
error.

Approved by: jfv (mentor)

MFC ixgbe commits for 10.2:

- r280182 - Split the driver into independent pf/vf loadables
- r280197 - Resolve build issues
- r280204 - Fix multiple same-name devclasses
- r280228 - Fix i386 LINT build issues / remove unused variable
- r280252 - Fix building ixgbe with gcc
- r280962 - Make changes to busdma code similar to r257541
- r281772 & r281773 - Remove unused variable
- partial r282280 - stats counter update (ix-only)
- r282289 - Add X550 support
- r282290 - Add X550 makefile updates
- r282293 - Add ixgbe_x550.c to conf/files
- r282299 - Fix gcc compile (extraneous extern declaration)

Finally, add ix_txrx.c to conf/files because it's required for compile in stable/10.

Approved by: jfv (mentor)

Convert space to tabs in the bits that I changed in my last commit.
No functional change.

Reported by: jmallet@

Fix the media detected for copper cables NIC based on chipsest X540T.

Requested by: hiren@
MFH: 279803

MFC r275358 r275483 r276982 - Removing M_FLOWID by hps@

r275358:
Start process of removing the use of the deprecated "M_FLOWID" flag
from the FreeBSD network code. The flag is still kept around in the
"sys/mbuf.h" header file, but does no longer have any users. Instead
the "m_pkthdr.rsstype" field in the mbuf structure is now used to
decide the meaning of the "m_pkthdr.flowid" field. To modify the
"m_pkthdr.rsstype" field please use the existing "M_HASHTYPE_XXX"
macros as defined in the "sys/mbuf.h" header file.

This patch introduces new behaviour in the transmit direction.
Previously network drivers checked if "M_FLOWID" was set in "m_flags"
before using the "m_pkthdr.flowid" field. This check has now now been
replaced by checking if "M_HASHTYPE_GET(m)" is different from
"M_HASHTYPE_NONE". In the future more hashtypes will be added, for
example hashtypes for hardware dedicated flows.

"M_HASHTYPE_OPAQUE" indicates that the "m_pkthdr.flowid" value is
valid and has no particular type. This change removes the need for an
"if" statement in TCP transmit code checking for the presence of a
valid flowid value. The "if" statement mentioned above is now a direct
variable assignment which is then later checked by the respective
network drivers like before.

r275483:
Remove M_FLOWID from SCTP code.

r276982:
Remove no longer used "M_FLOWID" flag from mbuf.h and update the netisr
manpage.

Note: The FreeBSD version has been bumped.

Reviewed by: hps, tuexen
Sponsored by: Limelight Networks

MFC r279393:

Pad RX copy alignment calculation to avoid illegal memory accesses

The optimization made in r239940 is valid for struct mbuf's current structure
and size in FreeBSD, but hardcodes assumptions about sizes of struct mbuf,
which are unfortunately broken if additional data is added to the beginning of
struct mbuf

X-MFC note (discussed with rwatson):

This change requires the MPKTHSIZE definition, which is only available after
head@r277203 and will not be MFCed as it breaks mbuf(9) KPI.

A direct commit to stable/10 and merges to other branches to add the necessary
definitions to work with the code as-is will be done to facilitate this MFC

PR: 194314
Approved/Reviewed by: erj, jfv
Sponsored by: EMC / Isilon Storage Division

MFC r263710, r273377, r273378, r273423 and r273455:

- De-vnet hash sizes and hash masks.
- Fix multiple issues related to arguments passed to SYSCTL macros.

Sponsored by: Mellanox Technologies

MFC r273112: Set the DROP_EN bit before the RX queue is brought up and active.

He noticed issues setting this bit in SRRCTL after the queue was up,
so doing it from the sysctl handler isn't enough and may not actually
work correctly.

This commit doesn't remove the sysctl path or try to change its
behaviour. I'll talk with others about how to finish fixing that
before I tackle that.

PR: kern/194311
Submitted by: luigi
MFC after: 3 days
Sponsored by: Norse Corp, Inc

MFC r271648 - The error bits are not valid with EOP=0; so intermediary
fragments should not be discarded.

Submitted by: Marc De La Gueronniere <mdelagueronniere@verisign.com>
Sponsored by: Verisign, Inc.

Merge r271647 - Fix a double-free of mbufs in rx_ixgbe_discard().

fmp->buf at the free point is already part of the chain being freed,
so double-freeing is counter-productive.

Submitted by: Marc De La Gueronniere <mdelagueronniere@verisign.com>
Sponsored by: Verisign, Inc.

MFC: r271286

Use the right constants in comparisons. This is currently a nop, as
MIN_RXD == MIN_TXD and MAX_RXD == MAX_TXD.

Reviewed by: Eric Joyner @ Intel
Approved by: re (kib)

MFC 270063: update of netmap code
(vtnet and cxgbe not merged yet because we need some other mfc first)

Make the ixgbe tunables now match their sysctl counterparts.

Previously the tunables and sysctls had different names for example:
hw.ixgbe.enable_aim => hw.ix.enable_aim

Anyone using ixgbe tunables should ensure they update /boot/loader.conf.

This is a direct commit to stable as the changes to sysctls in head
already fix this issue in a different way.

Sponsored by: Multiplay

MFC r268790

Fix a typo.

MFC svn 267065 and 267187

make sure ifp->if_transmit returns 0 if a buffer is enqueued.
This should also be merged to stable/9.

After this fix, drivers still known to have this bug are igxbe/ixv
and i40e.

Drivers using if_transmit are correct, and so are most of the
other drivers that reassing if_transmit.

Among other things, this bug causes panics when using netmap emulation
on top of generic drivers.

MFH: sync the netmap code with the one in HEAD
(enhanced VALE switch, netmap pipes, emulated netmap mode).
See details in the log for svn 261909.

MFC: 257695

- Fix link loss on vlan reconfiguration.
- Fix issues with 'vlanhwfilter'.

Approved by: re (glebius)

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

There are some high performance NICs that count statistics in hardware,
and there are ifnets, that do that via counter(9). Provide a flag that
would skip cache line trashing '+=' operation in ether_input().

Sponsored by: Netflix
Sponsored by: Nginx, Inc.
Reviewed by: melifaro, adrian
Approved by: re (marius)

Expose system level ixgbe sysctls.
Device level sysctls are already exposed as dev.ix.<device>

Fixing the case where number of queues for igb is auto-tuned and
hw.igb.num_queues does not return current/updated value.

Reviewed by: jfv
Approved by: re (delphij)
MFC after: 2 weeks

Restructure the mbuf pkthdr to make it fit for upcoming capabilities and
features. The changes in particular are:

o Remove rarely used "header" pointer and replace it with a 64bit protocol/
layer specific union PH_loc for local use. Protocols can flexibly overlay
their own 8 to 64 bit fields to store information while the packet is
worked on.

o Mechanically convert IP reassembly, IGMP/MLD and ATM to use pkthdr.PH_loc
instead of pkthdr.header.

o Extend csum_flags to 64bits to allow for additional future offload
information to be carried (e.g. iSCSI, IPsec offload, and others).

o Move the RSS hash type enumerator from abusing m_flags to its own 8bit
rsstype field. Adjust accessor macros.

o Add cosqos field to store Class of Service / Quality of Service information
with the packet. It is not yet supported in any drivers but allows us to
get on par with Cisco/Juniper in routing applications (plus MPLS QoS) with
a modernized ALTQ.

o Add four 8 bit fields l[2-5]hlen to store the relative header offsets
from the start of the packet. This is important for various offload
capabilities and to relieve the drivers from having to parse the packet
and protocol headers to find out location of checksums and other
information. Header parsing in drivers is a lot of copy-paste and
unhandled corner cases which we want to avoid.

o Add another flexible 64bit union to map various additional persistent
packet information, like ether_vtag, tso_segsz and csum fields.
Depending on the csum_flags settings some fields may have different usage
making it very flexible and adaptable to future capabilities.

o Restructure the CSUM flags to better signify their outbound (down the
stack) and inbound (up the stack) use. The CSUM flags used to be a bit
chaotic and rather poorly documented leading to incorrect use in many
places. Bring clarity into their use through better naming.
Compatibility mappings are provided to preserve the API. The drivers
can be corrected one by one and MFC'd without issue.

o The size of pkthdr stays the same at 48/56bytes (32/64bit architectures).

Sponsored by: The FreeBSD Foundation

Update PCI drivers to no longer look at the MEMIO-enabled bit in the PCI
command register. The lazy BAR allocation code in FreeBSD sometimes
disables this bit when it detects a range conflict, and will re-enable
it on demand when a driver allocates the BAR. Thus, the bit is no longer
a reliable indication of capability, and should not be checked. This
results in the elimination of a lot of code from drivers, and also gives
the opportunity to simplify a lot of drivers to use a helper API to set
the busmaster enable bit.

This changes fixes some recent reports of disk controllers and their
associated drives/enclosures disappearing during boot.

Submitted by: jhb
Reviewed by: jfv, marius, achadd, achim
MFC after: 1 day

Improve the MSIX setup code in the drivers, thanks to Marius for
the changes. Make sure that pci_alloc_msix() does give us the vectors
we need and fall back to MSI when it doesn't, also release any that
were allocated when insufficient.

MFC after: 3 days

Make the various driver MSIX setup routines fallback to MSI more
gracefully. This change was suggested by Marius Strobl, thank you.

PR: kern/181016
MFC after: ASAP

Correct a fat-finger in the last delta.

MFC after: ASAP

A number of important fixes:
- mbuf reused after an RX_COPY optimized operation can sometimes have
a bogus cached address, resulting in TCP hangs. Add critical save points
to the cached address. Thanks to Michael and the team at Verisign for
finding this problem.
- A couple more spots where the rxbuf->flags member should be cleared just
to be sure no incorrect RX_COPY state is left around. Thanks to Adrian
for tracking these down.
- Remove the rearm_queues function from the driver, this was found to be
responsible for some out-of-order packets by Verisign, and was always a
bandaid, with the other fixes in this delta the bandaid can finally be
removed.
- In the other/link interrupt handler the entire state of the EICS register
was being writen back into EICR (which clears causes and thus re-enables
those interrupts), this was wrong, so now mask off the queue portion of
the register value, so we only clear the other/link interrupt we intend.
Marc from Verisign found this.
- Make the SFP+ unsupported option tuneable now, by customer request.
- Finally, just a couple of minor DEBUG string fixes.

I want to call out and thank all the participants in the 10G community/Intel
calls for helping track down these problems and make the driver better for
everyone!

MFC after: 3 days, these are critical fixes for 9.2!

Opps, need to change the VF code as well.

MFC after: ASAP

Remove the conditional define around the option headers,
when building the driver as a module the result of the present
system results in INET and INET6 being undefined, and will cause
the panic in ixgbe_tso_setup(). The Makefile in the module directory
now renders the conditional in the source unnecessary and wrong.

MFC after: ASAP - the panic as a module must not get into 9.2

Fix my last commit, flags rather than flag... duh.

MFC after: 2 days

Fix to a panic found internally, bad pointer during rxeof
processing. Thanks for John Baldwin for catching this. Not
clearing the flag member of the rxbuf could result in a NULL
mbuf pointer being used.

MFC after: 2 days (this needs to get into 9.2!)

Add quad port probe support, this gives the admin proper information about the slot
(which should be a PCIE Gen 3 slot for this adapter) by looking back thru the PCI
parent devices to the slot device.

The fix above also corrects the bandwidth display to GT/s rather than the
incorrect Gb/s

Next, allow the use of ALTQ if you select the compile option IXGBE_LEGACY_TX.

Allow the use of 'unsupported' optic modules by a compile option as well.

Add a phy reset capability into the stop code, this is so a static configured
driver will still behave properly when taken down (not being able to unload it).

This revision synchronizes the shared code with Intel internal current code,
and note that it now includes DCB supporting code, this was necessitated by
some internal changes with the code, but it also will provide the opportunity
to develop this feature in the core driver down the road.

I have edited the README to get rid of some of the worse anachronisms in it
as well, its by no means as robust as I might wish at this point however.

Oh, I also have included some conditional stuff in the code so it will be
compatible in both the 9.X and 10 environments.

Performance has been a focus in recent changes and I believe this revision
driver will perform very well in most workloads.

MFC after: 2 weeks

use netmap_rx_irq() / netmap_tx_irq() to handle interrupts in
netmap mode, removing the logic from individual drivers.

(note: if_lem.c not updated yet due to some other pending modifications)

Two small fixes:
Set promiscuous code was unconditionally turning off multicast when
turning off promiscuous mode, this should only be done when there are
less than MAX groups. Thanks to Mike Karels for this correction.

Second, the overtmp interrupt setup/detection was wrong, correcting it.

MFC after: one week

Fix a small, but important bug, a task drain was mistakenly
being compiled only when setting LEGACY_TX, this means you would
not get the drain when needed on detach!!

Thanks to Bryan Venteicher (bryanv@freebsd.org) for catching this
little gremlin!! :)

First, sync to internal shared code, and then

Fixes:
- flow control - don't override user value on re-init
- fix to make 1G optics work correctly
- change to interrupt enabling - some bits were incorrect
for certain hardware.
- certain stats fixes, remove a duplicate increment of
ierror, thanks to Scott Long for pointing these out.
- shared code link interface changed, requiring some
core code changes to accomodate this.
- add an m_adj() to ETHER_ALIGN on the recieve side, this
was requested by Mike Karels, thanks Mike.
- Multicast code corrections also thanks to Mike Karels.

revert 247035

Reduce excessive nesting.

This fixes a out-of-order problem with several
of the newer drivers. The basic problem was
that the driver was pulling the mbuf off the
drbr ring and then when sending with xmit(), encounting
a full transmit ring. Thus the lower layer
xmit() function would return an error, and the
drivers would then append the data back on to the ring.
For TCP this is a horrible scenario sure to bring
on a fast-retransmit.

The fix is to use drbr_peek() to pull the data pointer
but not remove it from the ring. If it fails then
we either call the new drbr_putback or drbr_advance
method. Advance moves it forward (we do this sometimes
when the xmit() function frees the mbuf). When
we succeed we always call advance. The
putback will always copy the mbuf back to the top
of the ring. Note that the putback *cannot* be used
with a drbr_dequeue() only with drbr_peek(). We most
of the time, in putback, would not need to copy it
back since most likey the mbuf is still the same, but
sometimes xmit() functions will change the mbuf via
a pullup or other call. So the optimial case for
the single consumer is to always copy it back. If
we ever do a multiple_consumer (for lagg?) we
will need a test and atomic in the put back possibly
a seperate putback_mc() in the ring buf.

Reviewed by: jhb@freebsd.org, jlv@freebsd.org

Use DEVMETHOD_END macro defined in sys/bus.h instead of {0, 0} sentinel on device_method_t arrays

Reviewed by: cognet
Approved by: cognet

Clean some 'svn:executable' properties in the tree.

Submitted by: Christoph Mallon
MFC after: 3 days

rename the 'tag' and 'map' fields used the rx ring to their
previous names, 'ptag' and 'pmap' -- p stands for packet.

This change reduces the difference between the code in stable/9
and head, and also helps using the same ixgbe_netmap.h on both branches.

Approved by: Jack Vogel

Mechanically substitute flags from historic mbuf allocator with
malloc(9) flags in sys/dev.

Remove the sysctl process_limit interface, after some
thought I've decided its overkill,a simple tuneable for
each RX and TX limit, and then init sets the ring values
based on that, should be sufficient.

More importantly, fix a bug causing a panic, when changing
the define style to IXGBE_LEGACY_TX a taskqueue init was
inadvertently set #ifdef when it should be #ifndef.

Patch #12 OK, I said there was only 11 patches, but unfortunately
the revamped sysctl code did not work, and needed a change. This
makes the limit get set at the time that all sysctl stats are
created and is actually more elegant imho anyway.

Patch #11 - The final patch: this one greatly improves the
TX hot path by getting rid of index calculations and simply
managing pointers. Much of the creative code is due to my
coworker here at Intel, Alex Duyck, thanks Alex!

Also, this whole series of patches was given the critical
eye of Gleb Smirnoff and is all the better for it, thanks
Gleb!

Patch #10 Performance - this changes the protocol offload
interface and code in the TX path,making it tighter and
hopefully more efficient.

Patch #9 Performance - improve the tx dma failure
path, similar to a change done in igb long ago.

Patch #8 Performance changes - this one improves locality,
moving some counters and data to the ring struct from
the adapter struct, also compressing some data in the
move.

Patch #7 This is primarily about processing limit control.
- add a limit for both RX and TX, change the default to 256
- change the sysctl usage to be common, and now to be called
during init for each ring.
- the TX limit is not yet used, but the changes in the last
patch in this series uses the value.
- the motivation behind these changes is to improve data
locality in the final code.
- rxeof interface changes since it now gets limit from the
ring struct

Patch #6 Whitespace cleanup, and removal of some very old
defines (at Gleb's request). Also, change the defines around
the old transmit code to IXGBE_LEGACY_TX, I do this to make
it possible to define this regardless of the OS level (it is
not defined by default). There are also a couple changed
comments for clarity.

Patch #5 Cleanup unused IEEE1588 code fragments, the day may
come when this feature gets implemented, but its not here yet
and I see no reason to leave this laying around.

Patch #4 - this does two things, it removes a number of statistics,
these are FCOE stats (fiber channel over ethernet), something that
FreeBSD does not yet have, they were mistaken for flow control by
the implementor I believe. Secondly, the real flow control stats
are oddly named with a 'link' tag on the front, it was requested
by my validation engineer to make these stats have the same name as
the igb driver for clarity and that seemed reasonable to me.

Patch #3 - Add a new ioctl to access SFP+ module diagnostic
data via the I2C routines in shared code.

Patch #2 - remove OACTIVE and DEPLETED notions from the
multiqueue code, this functionality has proven to be more
trouble than it was worth. Thanks to Gleb for a second
critical look over my code and help in the patches!

First of a series of 11 patches leading to new ixgbe version 2.5.0
This removes the header split and supporting code from the driver.

A few important fixes:
- Testing TSO6 has led me to discover that HW RSC is
a problematic feature, it is ONLY designed to work
with IPv4 in the first place, and if IP forwarding
is done it can't be disabled as LRO in the stack,
also initial testing we've done at Intel shows an
equal performance using TSO[46] on the TX and LRO
on RX, if you ran older code on 82599 or later hardware
you actually could have detrimental performance for
this reason. So I am disabling the feature by default
and all our adapters will now use LRO instead.

- If you have flow control off and multiple queues it
was possible when the buffer of one queue becomes
full that all RX movement is stalled, to eliminate
this problem a feature bit is now set that will allow
packets to be dropped when full rather than stall.
Note, the default is to have flow control on, and this
keeps this from happening.

- Because of the recent fixes in the stack, LRO is now
auto-disabled when problematic, so I have decided to
enable it by default in the capabilities in the driver.

- There are some 1G modules used by some customers, a couple
small tweaks to properly support those in the media code.

- A note: we have now done some testing of TSO6 and using
LRO with IPv6 and it all works great!! Seeing line rate
in both directions in best cases. Thanks bz for your
excellent work!!

Correct code that was lost somewhere in the past,
this was designed to keep duplicate null vlan tags from
being added. When doing vlans purely via the switch
this problem will occur. Reported by external customer.

Now that device disabling is generic, remove extraneous code from the
device drivers that used to provide this feature.

This is a subset of 241856 (which was reverted)

Reviewed by: des
Approved by: cperciva (implicit)
MFC after: 1 week

This isn't functionally identical. In some cases a hint to disable
unit 0 would in fact disable all units.

This reverts r241856

Approved by: cperciva (implicit)

Now that device disabling is generic, remove extraneous code from the
device drivers that used to provide this feature.

Reviewed by: des
Approved by: cperciva
MFC after: 1 week

provide helper if_initbaudrate() to set if_baudrate_pf and if_baudrate_pf.
again, use ixgbe(4) as an example of how to use new helper function.

Reviewed by: jhb
MFC after: 1 week

introduce concept of ifi_baudrate power factor. the idea is to work
around the problem where high speed interfaces (such as ixgbe(4))
are not able to report real ifi_baudrate. bascially, take a spare
byte from struct if_data and use it to store ifi_baudrate power
factor. in other words,

real ifi_baudrate = ifi_baudrate * 10 ^ ifi_baudrate power factor

this should be backwards compatible with old binaries. use ixgbe(4)
as an example on how drivers would set ifi_baudrate power factor

Discussed with: kib, scottl, glebius
MFC after: 1 week

The drbr(9) API appeared to be so unclear, that most drivers in
tree used it incorrectly, which lead to inaccurate overrated
if_obytes accounting. The drbr(9) used to update ifnet stats on
drbr_enqueue(), which is not accurate since enqueuing doesn't
imply successful processing by driver. Dequeuing neither mean
that. Most drivers also called drbr_stats_update() which did
accounting again, leading to doubled if_obytes statistics. And
in case of severe transmitting, when a packet could be several
times enqueued and dequeued it could have been accounted several
times.

o Thus, make drbr(9) API thinner. Now drbr(9) merely chooses between
ALTQ queueing or buf_ring(9) queueing.
- It doesn't touch the buf_ring stats any more.
- It doesn't touch ifnet stats anymore.
- drbr_stats_update() no longer exists.

o buf_ring(9) handles its stats itself:
- It handles br_drops itself.
- br_prod_bytes stats are dropped. Rationale: no one ever
reads them but update of a common counter on every packet
negatively affects performance due to excessive cache
invalidation.
- buf_ring_enqueue_bytes() reduced to buf_ring_enqueue(), since
we no longer account bytes.

o Drivers handle their stats theirselves: if_obytes, if_omcasts.

o mlx4(4), igb(4), em(4), vxge(4), oce(4) and ixv(4) no longer
use drbr_stats_update(), and update ifnet stats theirselves.

o bxe(4) was the most correct driver, it didn't call
drbr_stats_update(), thus it was the only driver accurate under
moderate load. Now it also maintains stats itself.

o ixgbe(4) had already taken stats from hardware, so just
- drop software stats updating.
- take multicast packet count from hardware as well.

o mxge(4) just no longer needs NO_SLOW_STATS define.

o cxgb(4), cxgbe(4) need no change, since they obtain stats
from hardware.

Reviewed by: jfv, gnn

Merge similar fixes from 223198 from igb to ixgbe:
- Use a dedicated task to handle deferred transmits from the if_transmit
method instead of reusing the existing per-queue interrupt task.
Reusing the per-queue interrupt task could result in both an interrupt
thread and the taskqueue thread trying to handle received packets on a
single queue resulting in out-of-order packet processing and lock
contention.
- Don't define ixgbe_start() at all where if_transmit is used.

Tested by: Vijay Singh
Reviewed by: jfv
MFC after: 2 weeks

Define missing DEBUGOUT# macros. DEBUGOUT[45] are not yet used but are
being defined pre-emptively to avoid future build breakage

PR: kern/168967
Submitted by: fuzhli <fuzl@arraynetworks.com.cn>
Approved by: cperciva
MFC after: 1 week

Remove a prefetch() directive that, after careful testing, does more harm
than good.

Submitted by: Fabien Thomas
Reviewed by: jfv

Add missing braces

Obtained from: DragonFly

Heavily optimize the case of small RX packets of 160 bytes or less. For
this case, allocate a plain mbuf and copy the frame into it, then send the
copy up the stack, leaving the original mbuf+cluster in place in the
receive ring for immediate re-use. This saves a trip through 2 of the
3 zones of the compound mbuf allocator, a trip through busdma, and a trip
through the 1 of the 3 mbuf destructors. For our load at Netflix, this can
lower CPU consumption by as much as 20%. The copy algorithm is based on
investigative work from Luigi Rizzo earlier in the year.

Reviewed by: jfv
Obtained from: Netflix

Update to the ixgbe driver:
- Add a couple of new devices
- Flow control changes in shared and core code
- Bug fix to Flow Director for 82598
- Shared code sync to internal with required core change

Thanks to those helping in the testing and improvements to this driver!

MFC after:5 days

Correct typo(?) and actually set PTHRESH to 32 and not 16 as per Intel
Linux driver 3.8.21.

MFC after: 1 week

Before it gets lost in the noise.

Put a bandaid to prevent ixgbe(4) from completely locking up the system
under high load. Our platform has a few CPU cores and a single active
ixgbe(4) port with 4 queues. Under high enough traffic load, at about
7.5GBs and 700,000 packets/sec (outbound), the entire system would
deadlock. What we found was that each CPU was in an endless loop on a
different ix taskqueue thread. The OACTIVE flag had gotten set on each
queue, and the ixgbe_handle_queue() function was continuously rescheduling
itself via the taskqueue_enqueue. Since all CPUs were busy with their
taskqueue threads, the ixgbe_local_timer() function couldn't run to clear
the OACTIVE flag.

Submitted by: scottl
MFC after: 1 week

MFp4 bz_ipv6_fast:

Add TSO6 and LRO/IPv6 support.
Fix the module Makefile to at least properly inlcude opt_inet6.h
and allow builds without INET or INET6.

Sponsored by: The FreeBSD Foundation
Sponsored by: iXsystems

Reviewed by: gnn (as part of the whole)
MFC After: 3 days

fix a typo in a comment

If we pass down 64k - L2 hdr size + 1 to 64K L3+ data adding an ether
header will make the data go over the 64k limits announced to busdma as
maxsize and the transaction will fail.

With TSO this can result in a TCP regression due to the lost packet.

According to the data sheets ixgbe(4) 82598 and 82599 can handle up to
256k so increase the maximum.

Reported by: Jon Kåre Hellan, UNINETT (jon.kare.hellan uninett.no)
Tested by: Jon Kåre Hellan, UNINETT (jon.kare.hellan uninett.no)
MFC after: 1 week

Properly disable crc stripping when operating in netmap mode.

Contrarily to what i wrote in my previous commit, the 82599
does include the CRC in the length. The operating mode is
reset in ixgbe_init_locked() and so we need to hook into
the places where the two registers (HLREG0 and RDRXCTL) are
modified.

Enable prefetching of descriptors on the TX ring, using the same
values as in the Intel driver 3.8.21 for linux. The fact that it
is standard in the above driver suggests that it has no bad side
effects.

But of course there must be a reason for enabling features, not
just "it does not harm", so here it is a good one:

Prefetching enables full line rate even using a single queue (14.88
Mpps, compared to ~12 Mpps without prefetch). This in turn is
terribly useful when one wants to schedule traffic.

For obvious reasons the difference is only visible with netmap
or other high speed solutions, but presumably the advantage
should be in the order of a fraction of a microsecond when
starting transmission on an empty queue.

Discussed with Jack Vogel.

MFC after: 1 week

More conversions of drivers to use the PCI parent DMA tag.

A bunch of netmap fixes:

USERSPACE:
1. add support for devices with different number of rx and tx queues;

2. add better support for zero-copy operation, adding an extra field
to the netmap ring to indicate how many buffers we have already processed
but not yet released (with help from Eddie Kohler);

3. The two changes above unfortunately require an API change, so while
at it add a version field and some spares to the ioctl() argument
to help detect mismatches.

4. update the manual page for the two changes above;

5. update sample applications in tools/tools/netmap

KERNEL:

1. simplify the internal structures moving the global wait queues
to the 'struct netmap_adapter';

2. simplify the functions that map kring<->nic ring indexes

3. normalize device-specific code, helps mainteinance;

4. start exploring the impact of micro-optimizations (prefetch etc.)
in the ixgbe driver.
Use 'legacy' descriptors on the tx ring and prefetch slots gives
about 20% speedup at 900 MHz. Another 7-10% would come from removing
the explict calls to bus_dmamap* in the core (they are effectively
NOPs in this case, but it takes expensive load of the per-buffer
dma maps to figure out that they are all NULL.

Rx performance not investigated.

I am postponing the MFC so i can import a few more improvements
before merging.

(This commit only touches code within the DEV_NETMAP blocks)

Introduce some functions to map NIC ring indexes into netmap ring
indexes and vice versa. This way we can implement the bound
checks only in one place (and hopefully in a correct way).

On passing, make the code and comments more uniform across the
various drivers.

Wrap the bool typedef

New hardware support: Intel X540 adapter support added.
Some shared code reorganization along with the new adapter.
Sync changes to OACTIVE in igb into this driver.
Misc small fixes.

ixgbe changes:
- remove experimental code for disabling CRC
- use the correct constant for conversion between interrupt rate
and EITR values (the previous values were off by a factor of 2)
- make dev.ix.N.queueM.interrupt_rate a RW sysctl variable.
Changing individual values affects the queue immediately,
and propagates to all interfaces at the next reinit.
- add dev.ix.N.queueM.irqs rdonly sysctl, to export the actual
interrupt counts

Netmap-related changes for ixgbe:
- use the "new" format for TX descriptors in netmap mode.
- pass interrupt mitigation delays to the user process doing poll()
on a netmap file descriptor.
On the RX side this means we will not check the ring more than once
per interrupt. This gives the process a chance to sleep and process
packets in larger batches, thus reducing CPU usage.
On the TX side we take this even further: completed transmissions are
reclaimed every half ring even if the NIC interrupts more often.
This saves even more CPU without any additional tx delays.

Generic Netmap-related changes:
- align the netmap_kring to cache lines so that there is no false sharing
(possibly useful for multiqueue NICs and MSIX interrupts, which are
handled by different cores). It's a minor improvement but it does not
cost anything.

Reviewed by: Jack Vogel
Approved by: Jack Vogel

netmap-related changes:
1. correct the initialization of RDT when there is an ixgbe_init()
while a netmap client is active. This code was previously
in ixgbe_initialize_receive_units() but RDT is overwritten
shortly afterwards in ixgbe_init_locked()

2. add code (not active yet) to disable CRCSTRIP while in netmap mode.
From all evidence i could gather, it seems that when the 82599 has to
write a data block that is not a full cache line, it first reads
the line (64 bytes) and then writes back the updated version.
This hurts reception of min-sized frames, which are only 60 bytes
if the CRC is stripped: i could never get above 11Mpps
(received from one queue) with CRCSTRIP enabled, whyle 64+4-byte
packets reach 14.2 Mpps (the theoretical maximum).
Leaving the CRC in gets us 14.88Mpps for 60+4 byte frames,
(and penalizes 64+4). The min-size case is important not just because
it looks good in benchmarks, but also because this is the size
of pure acks.
Note we cannot leave CRCSTRIP on by default because it is
incompatible with some other features (LRO etc.)

small code cleanup in preparation for future modifications in
the memory allocator used by netmap. No functional change,
two small bug fixes:
- in if_re.c add a missing bus_dmamap_sync()
- in netmap.c comment out a spurious free() in an error handling block

ether_ifattach() sets if_mtu to ETHERMTU, don't bother set it again

Reviewed by: yongari

Consistently use types in ixgbe driver code:

- {ixgbe,ixv}_header_split is passed to TUNABLE_INT, so delcare it
int, not bool.
- {ixgbe,ixv}_tx_ctx_setup() returns a boolean value, so declare it
bool, not int.
- {ixgbe,ixv}_tso_setup() returns a bool, so declare it bool, not boolean_t.
- {ixgbe,ixv}_txeof() returns a bool, so declare it bool, not boolean_t.
- Do not re-define bool if the symbol already exists.

MFC after: 2 weeks
Sponsored by: Isilon Systems, LLC

1. Fix the handling of link reset while in netmap more.
A link reset now is completely transparent for the netmap client:
even if the NIC resets its own ring (e.g. restarting from 0),
the client will not see any change in the current rx/tx positions,
because the driver will keep track of the offset between the two.

2. make the device-specific code more uniform across different drivers
There were some inconsistencies in the implementation of the netmap
support routines, now drivers have been aligned to a common
code structure.

3. import netmap support for ixgbe . This is implemented as a very
small patch for ixgbe.c (233 lines, 11 chunks, mostly comments:
in total the patch has only 54 lines of new code) , as most of
the code is in an external file sys/dev/netmap/ixgbe_netmap.h ,
following some initial comments from Jack Vogel about making
changes less intrusive.
(Note, i have emailed Jack multiple times asking if he had
comments on this structure of the code; i got no reply so
i assume he is fine with it).

Support for other drivers (em, lem, re, igb) will come later.

"ixgbe" is now the reference driver for netmap support. Both the
external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific
patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should
serve as a reference for other device drivers.

Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap,
the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz
on an i7-860 with 4 cores and 82599 card. Haven't tried yet more
aggressive optimizations such as adding 'prefetch' instructions
in the time-critical parts of the code.

The maximum read size of incoming packets is done in 1024-byte increments.
The current code was rounding down the maximum frame size instead of
routing up, resulting in a read size of 1024 bytes, in the non-jumbo
frame case, and splitting the packets across multiple mbufs.

Consequently the above problem exposed another issue, which is when
packets were splitted across multiple mbufs, and all of the mbufs in the
chain have the M_PKTHDR flag set.

Submitted by: original patch by Ray Ruvinskiy at BlueCoat dot com
Reviewed by: jfv, kmacy, rwatson
Approved by: re (rwatson)
MFC after: 5 days

Cut and paste mistake corrected.

First off: update the driver README, the old one was horribly
crusty, and this still isn't perfect, but its at least a bit
more recent.

Secondly, a few improvements to the driver from Andrew Boyer,
support hint to allow devices to not attach, add VLAN_HWTSO
capability so vlans can use TSO, fix in the interrupt handler
to make sure the stack TX queue is processed. Oh, and also
make sure IPv6 does not cause a re-init in the ioctl routine.
Thanks for your efforts Andrew!

Thanks to Claudio Jeker for noticing the ixgbe_xmit() routine
was not correctly swapping the dma map from the first to the
last descriptor in a multi-descriptor transmission, corrected
this.

Add a #define for driver portability to older OS

- Add the RX refresh changes from igb to ixgbe
- Also a couple minor tweaks to the TX code from the same source.
- Add the INET ioctl code which has been missing from this driver,
and which caused IP aliases to reset the interface.
- Last, some minor logic changes that just reflect upcoming
hardware support, but have no other functional effect now.

MFC after a week

Do a sweep of the tree replacing calls to pci_find_extcap() with calls to
pci_find_cap() instead.

Don't bother to run the flowcontrol code if there
is no change. Thanks to Andrew for the tweak.

Missing case for 82598DA type adapter, thanks Andrew.

Leftover bogus TX UNLOCK removed. Thanks to
Andrew Boyer.

Update driver to version 2.3.8:
CRITICAL FIX - with stats changes the older 82598 will panic
and trash the stack on driver load, FCOE registers ONLY exist
in 82599 and must not be read otherwise.

kern/153951 - to correct incorrect media type on adapters
with pluggable modules I have eliminated the old static
table in favor of a new dynamic shared code routine. This
also has the benefit of detecting changes when a different
module is inserted.

Performance/enhancement to the Flow Director code from my
linux coworker (the developer of the code).

Fixes from Michael Tuexen - a data corruption problem on the
82599 (CRITICAL), fix so the buf size correctly adjusts as
the cluster changes, and max descriptors are set properly.
Also added 16K clusters for those REALLY big jumbos :)

In the RX path, the RX LOCK was not being released, and this
causes LOR problems. Add the code that igb already has.

Sync with in house shared code, this was necessary for the
Flow Director fix.

MFC in 2 days

Specify a CTLTYPE_FOO so that a future sysctl(8) change does not need
to rely on the format string.

sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly.

Commit the Intel drivers.

CSUM flags need to be OS version sensitive in ixv code

MFC in 3 days

kern/150247 - virtualization code also needs fix for 7.X to be buildable...

MFC in 3 days

Fix to kern/150247 - make ixgbe buildable for 7.x

kern/153772 fix variable names.

Thank you Andrew Boyer for catching these

MFC in 3 days

This small little change is a bug that drove me nuts
finding. The test to compare the mbuf m_len against
a fixed value and then returning needs to be removed.

When using VLANS and doing HW_TAGGING, and IPV6, the
ICMP6 packets actually fail this condition, the constant
assumes that the tag is IN the frame, and its not, so
the length is actually tiny. Furthermore, I'm not sure
what the point was to just return??

MFC after: 3 days

Interrupt handler, and stats changes from Michael Tuexen,
thanks Michael!

and the header...

A couple fixes got clobbered, putting them back.

Update ixgbe driver to verion 2.3.6
- This adds a VM SRIOV interface, ixv, it is however
transparent to the user, it links with the ixgbe.ko,
but when ixgbe is loaded in a virtualized guest with
SRIOV configured this will be detected.
- Sync shared code to latest
- Many bug fixes and improvements, thanks to everyone
who has been using the driver and reporting issues.

Fix typos.

PR: bin/148894
Submitted by: olgeni

Do not allocate multicast array memory in multicast filter
configuration function. For failed memory allocations, em(4)/lem(4)
called panic(9) which is not acceptable on production box.
igb(4)/ixgb(4)/ix(4) allocated the required memory in stack which
consumed 768 bytes of stack memory which looks too big.

To address these issues, allocate multicast array memory in device
attach time and make multicast configuration success under any
conditions. This change also removes the excessive use of memory in
stack.

Reviewed by: jfv

Do not call voluntary panic(9) in case of if_alloc() failure.

Reviewed by: jfv

Fix build

Left out header change in last delta - new member
in adapter so that advertise changes can be done
to one port without the other changing.

Fix build.

BAH, I apologize, the wrong version of the code got
fat fingered in place, this is the correct version
that actually works... <sheepish grin>

MFC: in a week

Add a new sysctl option, this will allow one to
limit the advertised speed of an SFP+ to 1G, effectively
"forcing" link at that lower speed. It is off by default
and is enabled by sysctl dev.ix.0.force_gig=1, 0 will
set it back to the norm.

Change the mbuf memory calls back to NOWAIT as a
problem has been seen in one case with doing the
M_WAITOK

Remove a disable_queue from the beginning of the
interrupt handler, automask handles it.
Also, add in msix vector descriptions.

MFC for 8.1 asap

Fixes for panic experienced in test at Intel, when
doing bidirectional stress traffic on 82598.

Also a couple bug fixes from Michael Tuexen, thank you!!

Add a workaround into the header so that 8 REL can use
the driver (adds local copy of ALTQ fix).

MFC: in a few days

A few changes:
When not defining header split do not allocate mbufs,
this can be a BIG savings in the mbuf memory pool.

Also keep seperate dma maps for the header and
payload pieces when doing header split. The basis
of this code was a patch done a while ago by
yongari, thank you :)

A number of white space changes.

MFC: in a few days

A few minor fixes:
- add a moderation value to the Link vector
- allow disabling HW RSC on the 82599 if LRO
is not enabled.
- correct error in the stats code
- change optic type on the 82598 DA device

Thanks to Andrew Boyer for the changes.

Remove the tx queue selection based on the cpu whe
no flowid is present, this was causing some bad
reordering, now just use 0.

Also, add a few watchdog bits, and tx handler bits
that were corrected in igb.

fix my clobber of the copyright date :)

Thanks to Michael Tuexen for adding SCTP support for 82599,
also for finding a one character bug that kept TSO from working.

Sometimes with direct attach cables a failure can occur in init,
the old method of calling detach was broken, there is no way to
return an error to the system from init, so I have changed it to
return failure thru the ioctl.

And, have fixed the ALTQ code changes of Max Laier, sorry Max :)

Update the driver to Intel version 2.1.6
- add some new hardware support for 82599
- Big change to interrupt architecture, it now
uses a queue which contains an RX/TX pair as
the recipient of the interrupt. This will reduce
overall system interrupts/msix usage.
- Improved RX mbuf handling: the old get_buf routine
is no longer synchronized with rxeof, this allows
the elimination of packet discards due to mbuf
allocation failure.
- Much simplified and improved AIM code, it now
happens in the queue interrupt context and takes
into account both the traffic on the RX AND TX
side.
- variety of small tweaks, like ring size, that have
been seen as performance improvements.
- Thanks to those that provided feedback or suggested
changes, I hope I've caught all of them.

Fix drbr and altq interaction:
- introduce drbr_needs_enqueue that returns whether the interface/br needs
an enqueue operation: returns true if altq is enabled or there are
already packets in the ring (as we need to maintain packet order)
- update all drbr consumers
- fix drbr_flush
- avoid using the driver queue (IFQ_DRV_*) in the altq case as the
multiqueue consumer does not provide enough protection, serialize altq
interaction with the main queue lock
- make drbr_dequeue_cond work with altq

Discussed with: kmacy, yongari, jfv
MFC after: 4 weeks

Remove extraneous semicolons, no functional changes.

Submitted by: Marc Balmer <marc@msys.ch>
MFC after: 1 week

Update driver to Intel version 2.0.7:

This adds new feature support for the 82599, a hardware
assist to LRO, doing this required a large revamp to the
RX cleanup code because the descriptor ring may not be
processed out of order, this necessitated the elimination
of global pointers.

Additionally, the RX routine now does not refresh mbufs
on every descriptor, rather it will do a range, and then
update the hardware pointer at that time. These are
performance oriented changes.

The TX side now has a cleaner simpler watchdog algorithm
as well, in TX cleanup a read of ticks is stored, that
can then be compared in local_timer to determine if
there is a hang.

Various other cleanups along the way, thanks to all who
have provided input and testing.

Take a step towards removing if_watchdog/if_timer. Don't explicitly set
if_watchdog/if_timer to NULL/0 when initializing an ifnet. if_alloc()
sets those members to NULL/0 already.

Stats missed packet handling was still not quite
right, thanks to Dmitrij Tejblum for the correction,
need a variable with scope only within the for loop
for all queues.

MFC: 3 days

If an interface is brought up with no cable it will experience
watchdog resets, this is due to a missing check for link in the
new multiqueue start code.

MFC: 3 days

Stupid cut and paste error on a stats struct member, thanks
to Ryan at Small Tree for finding this one.

A couple of changes: first, make header split on by default
and remove from its dependency on LRO, my tests have shown
that its always beneficial, even when doing bridging.
Second, fix up a few problems in the statistics code, the
adapter dependencies had gotten lost so some code that should
only run on 82599 was always running, this resulted in bogus
flow control numbers on 82598.

Improvement on the last change, this gives a precise
way to tell the one and only interface that a vlan
event is for. Thanks to John Baldwin for the patch.

Approved by: re

This delta fixes two bugs:
- When a vlan event occurs a check was not made that
the event was actually for the interface, thus resulting
in a panic. All three drivers have this vulnerability. Add
a check for this condition.
- Secondly, there was a duplicate buf_ring free in the em
driver resulting in a panic on unload. Remove.

Approved by: re

Multiqueue RX is not correctly enabled on the new 82599
adapter, the SRRCTL register needs to be setup per queue.

Approved by: re

Use if_maddr_rlock()/if_maddr_runlock() rather than IF_ADDR_LOCK()/
IF_ADDR_UNLOCK() across network device drivers when accessing the
per-interface multicast address list, if_multiaddrs. This will
allow us to change the locking strategy without affecting our driver
programming interface or binary interface.

For two wireless drivers, remove unnecessary locking, since they
don't actually access the multicast address list.

Approved by: re (kib)
MFC after: 6 weeks

Decided to limit the interrupt bind to multiqueue
config as done in igb.

Change intr_bind to bus_bind_intr, thanks to John Baldwin
for pointing out this simplification.

Make CPU bind call architecture specific to satisfy LINT

Update for the Intel 10G driver, this adds support for
newest hardware, adds multiqueue tx interface, infrastructure
cleanup to allow up to 32 MSIX vectors on newer Nehalem systems.
Bug fixes, etc.

Add missing file, sorry bout that :)

Update the ixgbe driver to 1.7.4, this includes support
for the new 82599 adapter family, adds header split, and
many small fixes. The driver should now be added to the
GENERIC kernel.

MFC after: 2 weeks

Small nit I just noticed, a pre-decrement should be post.

Updated ixgbe driver - version 1.6.2

-This version has header split, and as a result a number of
aspects of the code have been improved/simplified.
- Interrupt handling refined for performance
- Many small bugs fixed along the way

MFC after: ASAP - in time for 7.1

This updates the ixgbe driver to Intel internal version 1.4.7
Shared code changes, core driver fixes, vlan event/filter support

Also Kip Macy's fix to allow any number of queues, thanks Kip!

Change include to reflect tcp_lro.h move.

Remove the tcp_lro.[ch] files as they are now in netinet.

This is driver version 1.4.4 of the Intel ixgbe driver.
-It has new hardware support
-It uses a new method of TX cleanup called Head Write Back
-It includes the provisional generic TCP LRO feature contributed
by Myricom and made general purpose by me. This should move into
the stack upon approval but for this driver drop its in here.
-Also bug fixes and etc...

MFC in a week if no serious issues arise.

This is an update to the new Intel 10G 82598 driver.
The first drop was Beta, this code is expected to be the release version.
Note that this driver code will build in either 6.2 or 7. If you
use the code in 6.2 you will not get TSO or MSI/X support but it will
function in a legacy mode.

Approved by: re

A couple late breaking bugs that testing have turned up.
- change include style so build in kernel try OR standalone work.
- Limit HWCSUM - I was led to believe that it would work with RSS,
but our testing had odd issues which suggests this is false.
- A fatfinger error in the ioctl code made ifconfig up not work.

Approved by: re

Add the actual source too :)

Approved by: re