sys: Remove $FreeBSD$: one-line .c comment pattern

Remove /^/[*/]\s*\$FreeBSD\$.*\n/

net: iflib: fix vlan processing in the drivers

The logic that sets iri_vtag and M_VLANTAG does not handle the
case where the 802.11q VLAN tag is 0. Fix this issue across
the iflib drivers. While there, also improve and align the
VLAN tag check extraction, by moving it outside the RX descriptor
loop, eliminating a local variable and additional checks.

PR: 260068
Reviewed by: kbowling, gallatin
Reported by: erj
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D33156

net: iflib: let the drivers use isc_capenable

Since isc_capenable (private copy of ifp->if_capenable) is
now synchronized to if_capenable, use it in the drivers
when checking the IFCAP_* bits.
This results in better cache usage and avoids indirection
through the ifp pointer.

PR: 260068
Reviewed by: kbowling, gallatin
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D33156

ixgbe: whitespace cleanup pass

Reviewed by: markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D32131

ixgbe: Rename 'struct adapter' to 'struct ixgbe_softc'

Rename the 'struct adapter' to 'struct ixgbe_softc' to avoid type
ambiguity in things like kgdb.

Reviewed by: markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D32131

ixgbe: Use C99 bool types

Approved by: imp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D31621

ixgbe: Clean up ix_txrx

The intention here is to reduce differences with D30072.
The only functional change is logical simplification in
ixgbe_rx_checksum.

Reviewed by: gallatin
Differential Revision: https://reviews.freebsd.org/D30074

iflib: Make if_shared_ctx_t a pointer to const

This structure is shared among multiple instances of a driver, so we
should ensure that it doesn't somehow get treated as if there's a
separate instance per interface. This is especially important for
software-only drivers like wg.

DEVICE_REGISTER() still returns a void * and so the per-driver sctx
structures are not yet defined with the const qualifier.

Reviewed by: gallatin, erj
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D29102

if_ixv: disable RSS configuration on 82599 and X540 VFs

It is reported that those VFs share their RSS configuration with PF and,
thus, they cannot be configured independently.

Also:
- add missing opt_rss.h to if_ixv.c, otherwise RSS kernel option could
not be seen
- do not enable IXGBE_FEATURE_RSS on the older VFs
- set flowid / hash type to M_HASHTYPE_NONE or M_HASHTYPE_OPAQUE_HASH
(based on what the hardware reports) if IXGBE_FEATURE_RSS is not set

Reviewed by: nobody
MFC after: 4 weeks
Sponsored by: Panzura
Differential Revision: https://reviews.freebsd.org/D21705

ix(4),ixv(4): Fix TSO offloads when TXCSUM is disabled

This patch and commit message are based on r340256 created by Jacob Keller:

The iflib stack does not disable TSO automatically when TXCSUM is
disabled, instead assuming that the driver will correctly handle TSOs
even when CSUM_IP is not set.

This results in iflib calling ixgbe_isc_txd_encap with packets which have
CSUM_IP_TSO, but do not have CSUM_IP or CSUM_IP_TCP set. Because of
this, ixgbe_tx_ctx_setup will not setup the IPv4 checksum offloading.

This results in bad TSO packets being sent if a user disables TXCSUM
without disabling TSO.

Fix this by updating the ixgbe_tx_ctx_setup function to check both
CSUM_IP and CSUM_IP_TSO when deciding whether to enable checksums.

Once this is corrected, another issue for TSO packets is revealed. The
driver sets IFLIB_NEED_ZERO_CSUM in order to enable a work around that
causes the ip->sum field to be zero'd. This is necessary for ix
hardware to correctly perform TSOs.

However, if TXCSUM is disabled, then the work around is not enabled, as
CSUM_IP will not be set when the iflib stack checks to see if it should
clear the sum field.

Fix this by adding IFLIB_TSO_INIT_IP to the iflib flags for the ix and
ixv interface files.

Once both of these changes are made, the ix and ixv drivers should
correctly offload TSO packets when TSO offload is enabled, regardless
of whether TXCSUM is enabled or disabled.

Submitted by: Piotr Pietruszewski <piotr.pietruszewski@intel.com>
Reviewed by: IntelNetworking
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D18470

intel iflib drivers: correct initialization of tx_cidx_processed

From Jake:

In r341156 ("Fix first-packet completion", 2018-11-28) a hack to work
around a delta calculation determining how many descriptors were used
was added to ixl_isc_tx_credits_update_dwb.

The same fix was also applied to the em and igb drivers in r340310, and
to ix in r341156.

The hack checked the case where prev and cur were equal, and then added
one. This works, because by the time we do the delta check, we already
know there is at least one packet available, so the delta should be at
least one.

However, it's not a complete fix, and as indicated by the comment is
really a hack to work around the real bug.

The real problem is that the first time that we transmit a packet,
tx_cidx_processed will be set to point to the start of the ring.
Ultimately, the credits_update function expects it to point to the
*last* descriptor that was processed. Since we haven't yet processed any
descriptors, pointing it to 0 results in this incorrect calculation.

Fix the initialization code to have it point to the end of the ring
instead. One way to think about this, is that we are setting the value
to be one prior to the first available descriptor.

Doing so, corrects the delta calculation in all cases. The original fix
only works if the first packet has exactly one descriptor. Otherwise, we
will report 1 less than the correct value.

As part of this fix, also update the MPASS assertions to match the real
expectations. First, ensure that prev is not equal to cur, since this
should never happen. Second, remove the assertion about prev==0 || delta
!= 0. It looks like that originated from when the em driver was
converted to iflib. It seems like it was supposed to ensure that delta
was non-zero. However, because we originally returned 0 delta for the
first calculation, the "prev == 0" was tacked on.

Instead, replace this with a check that delta is greater than zero,
after the correction necessary when the ring pointers wrap around.

This new solution should fix the same bug as r341156 did, but in a more
robust way.

Submitted by: Jacob Keller <jacob.e.keller@intel.com>
Reviewed by: shurd@
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D18545

Fix first-packet completion

The first packet after the ring is initialized was never
completed as isc_txd_credits_update() would not include it in the
count of completed packets. This caused netmap to never complete
a batch. See PR 233022 for more details.

This is the same fix as the r340310 for e1000

PR: 233607
Reported by: lev
Reviewed by: lev
MFC after: 3 days
Sponsored by: Limelight Networks
Differential Revision: https://reviews.freebsd.org/D18368

em/igb/ix(4): Port two Tx/Rx fixes made to ixl in r339338

- Fix assert/panic on receive when Jumbo Frames are enabled.

From the commit I made to ixl:
"It turns out that *_isc_rxd_available is supposed to return how many
packets are available to be cleaned on the rx ring. This patch removes
a section of code where if the budget argument is 1, the function would return
one if there was a descriptor available, not necessarily a packet.

This is okay in regular mtu 1500 traffic since the max frame size is less
than the configured receive buffer size (2048), but this doesn't work when
received packets can span more than one descriptor, as is the case when the
mtu is 9000 and the receive buffer size is 4096."

- Fix possible Tx hang because *_isc_txd_credits_update returns incorrect result

From the commit by Krzysztof Galazka to ixl: "Function isc_txd_update_credits
called with clear set to false should return 1 if there are TX descriptors
already handled by HW. It was always returning 0 causing troubles with UDP TX
traffic."

PR: 231659
Reported by: lev@
Approved by: re (gjb@)
Sponsored by: Intel Corporation

ix(4), ixv(4): VLAN tag stripping fixes for Amazon EC2 Enhanced Networking

From Piotr:

ix(4), ixv(4): Add VLAN tag strip check when receiving packets
ixv(4): Fix support for VLAN_HWTAGGING and VLAN_HWFILTER flags

This change will prevent driver from passing VLAN tags when
interface configuration is not expecting them. VF driver will
check for VLAN_HWTAGGING and VLAN_HWFILTER flags and act adequately.

This patch resolves problem occuring on EC2 platforms.

Submitted by: Piotr Pietruszewski <piotr.pietruszewski@intel.com>
Reported by: cperciva@
Reviewed by: cperciva@, Intel Networking
Approved by: re
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D17061

netmap and iflib drivers, silence unused var warnings

Use C99 initializers for iflib function tables.

Reviewed by: sbruno
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D15041

ixgbe(4): Convert driver to use iflib

Initial update to the ixgbe PF and VF drivers to support the iflib interface.

The PF driver version is bumped to 4.0.0, and the VF driver version is bumped to 2.0.0.

Special thanks to sbruno@ for the support in helping make this conversion happen.

Submitted by: Jeb Cramer <cramerj@intel.com>, Krzysztof Galazka (Chris) <krzysztof.galazka@intel.com>, Piotr Pietruszewski <piotr.pietruszewski@intel.com>
Reviewed by: sbruno@, shurd@, #IntelNetworking
Tested by: Jeffrey Pieper <jeffrey.e.pieper@intel.com>, Sergey Kozlov <kozlov.sergey.404@gmail.com>
Sponsored by: Limelight Networks, Intel Corporation
Differential Revision: https://reviews.freebsd.org/D11727

Drop ixgbe RX lock during TCP_LRO processing. This eliminates a "storm"
of LOR detection and a bit of lock release/acquire collision when using LRO.

Submitted by: Kevin Bowling <kevin.bowling@kev009.com>
MFC after: 2 days
Differential Revision: https://reviews.freebsd.org/D11712

ixgbe(4): Update HEAD (p3) to 3.2.12-k

Includes:

- Support for X550EM devices.
- Support for Bypass adapters.
- Flow Director code moved to separate files
- SR-IOV code moved to separate files
- Netmap code moved to separate files

Differential Revision: https://reviews.freebsd.org/D11232
Submitted by: Jeb Cramer <cramerj@intel.com>
Reviewed by: erj@
Tested by: Jeff Pieper <jeffrey.e.pieper@intel.com>
Sponsored by: Intel Corporation

Fix a double free in ixgbe_rxeof()

Submitted by: rstone
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D10255

net: Use M_HASHTYPE_OPAQUE_HASH if the mbuf flowid has hash properties

Reviewed by: hps, erj, tuexen
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6688

Correct possible underflow conditions when checking for available space
in the tx h/w ring buffer.

Reviewed by: gnn jeb.j.cramer@intel.com
MFC after: 1 week
Sponsored by: Limelight Networks
Differential Revision: https://reviews.freebsd.org/D5918

tcp/lro: Use tcp_lro_flush_all in device drivers to avoid code duplication

And factor out tcp_lro_rx_done, which deduplicates the same logic with
netinet/tcp_lro.c

Reviewed by: gallatin (1st version), hps, zbb, np, Dexuan Cui <decui microsoft com>
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D5725

Add missing #ifdef INET after r292674 to allow NOIP and NOINET kernels
to build.

Fix NO INET6 build broken at svn r292674

Reported by: ohartman@zedat.fu-berlin.de

ixgbe(4): Update to version 3.1.13-k

Add support for two new devices: X552 SFP+ 10 GbE, and the single port
version of X550T.

Submitted by: erj
Reviewed by: gnn
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D4186

Add support for sysctl knobs to live tune the per interrupt rx/tx packet
processing limits in ixgbe(4)

Differential Revision: https://reviews.freebsd.org/D3719
Submitted by: jason wolfe (j-nitrology.com)
MFC after: 2 weeks

Remove FreeBSD version check for deprecated M_FLOWID.

Reviewed by: erj
Sponsored by: Limelight Networks

Expose full 32bit RSS hash from card regardless of whether RSS is defined or
not. When doing multiqueue, we are all setup to have full 32bit RSS hash from
the card. We do not need to hide that under "ifdef RSS" and should expose that
by default so others like lagg(4) can use that and avoid hashing the traffic by
themselves.

While here, delete the FreeBSD version check and use of deprecated M_FLOWID.

Reviewed by: adrian, erj
MFC after: 1 week
Sponsored by: Limelight Networks

Delta D2489 - Add SRIOV support to the Intel 10G driver.

NOTE: This is a technology preview, while it has undergone
development testing, Intel has not yet completed full
validation of the feature. It is being integrated for
early access and customer testing.

Revert last commit, to remove added skeleton tree.

Delta D2489 - Add SRIOV support to the Intel 10G driver.

NOTE: This is a technology preview, while it has undergone development
tests, Intel has not yet completed full validation of the feature.
It is being integrated for early access and customer testing.

Add support for certain Intel X550 devices.

These include standalone X550 adapters, X552 10GbE backplane, and
X552/X557-AT 10GBASE-T; with the latter two being integrated into Xeon D SoCs.

As well, this bumps the ixgbe version number to 2.8.3, and includes updates
to shared code for support for the new devices.

Differential Revision: D2414
Reviewed by: gnn, adrian
Approved by: jfv (mentor), gnn (mentor)

Make changes to busdma code in tx/rx path similar to the ones made in r257541.

- bus_dmamap_create() does not take the BUS_DMA_NOWAIT flag
- properly unload maps
- do not assign NULL to dma map pointers

Submitted by: Konstantin Belousov <kostikbel@gmail.com>
Approved by: jfv (mentor)
MFC after: 1 week

Fix building ixgbe with gcc, it doesn't like nested extern declarations.
The fix is to move the extern declaration ix_crcstrip out of
ixgbe_setup_hw_rsc.

Fix ixgbe(4) to compile - with RSS; with ix+ixv in the kernel.

* Fix the multiple same-named devclasses; the duplicate name
trips up the linker.

* Re-do the taskqueue stuff to use the new cpuset API, not the old
pinned API.

* Add includes for the new location of the RSS configuration routines.

This allows ixgbe to compile as a module /and/ linked into the kernel,
along with RSS working.

Sponsored by: Norse Corp, Inc.

Resolve a few build issues, add module directories back into Makefile,
then correct a NETMAP problem resulting from the split, and finally
temporarily disable the X550 functionality.

Update to the Intel ixgbe driver:
- Split the driver into independent pf and vf loadables. This is
in preparation for SRIOV support which will be following shortly.
This also allows us to keep a seperate revision control over the
two parts, making for easier sustaining.
- Make the TX/RX code a shared/seperated file, in the old code base
the ixv code would miss fixes that went into ixgbe, this model
will eliminate that problem.
- The driver loadables will now match the device names, something that
has been requested for some time.
- Rather than a modules/ixgbe there is now modules/ix and modules/ixv
- It will also be possible to make your static kernel with only one
or the other for streamlined installs, or both.

Enjoy!

Submitted by: jfv and erj