Remove svn:mergeinfo carried over from stable/9.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFH: sync the netmap code with the one in HEAD
(enhanced VALE switch, netmap pipes, emulated netmap mode).
See details in the log for svn 261909.

MFC: 261291

The timestamp bit is number 17, and not number 9, in the stat error
field of the receive descriptor.

MFC revision 259907
use the correct netmap <-> nic slot mapping on the transmit ring for 'lem'.
This bug would manifest only in netmap mode and on packets transmitted after
a NIC reset while netmap mode is active.

Fix the outstanding mergeinfo part of r249374.

Merge from head: sync the netmap code with the one in HEAD

MFC: r256069

Expose system level ixgbe sysctls.
Device level sysctls are already exposed as dev.ix.<device>

Fixing the case where number of queues for igb is auto-tuned and
hw.igb.num_queues does not return current/updated value.

MFC r254264 Make the igb_mq_start() routine use TRYLOCK rather than simple
queuing, there were latency and other problems with the latter.

MFC r254262 Further improve the msix setup, make sure pci_alloc_msix() gives us
the vectors we requested, and fall back to MSI when not, also release
any allocated resources before the fallback.

MFC r254008 Make the fallback from MSIX to MSI interrupt usage more graceful.

Merge r254263:

Update PCI drivers to no longer look at the MEMIO-enabled bit in the PCI
command register. The lazy BAR allocation code in FreeBSD sometimes
disables this bit when it detects a range conflict, and will re-enable
it on demand when a driver allocates the BAR. Thus, the bit is no longer
a reliable indication of capability, and should not be checked. This
results in the elimination of a lot of code from drivers, and also gives
the opportunity to simplify a lot of drivers to use a helper API to set
the busmaster enable bit.

This changes fixes some recent reports of disk controllers and their
associated drives/enclosures disappearing during boot.

Candidate for 9.2

Submitted by: jhb
Reviewed by: jfv, marius, adrian, achim

When the igb driver is static there are cases when early interrupts occur,
resulting in a panic in refresh_mbufs, to prevent this add a check in the
interrupt handler for DRV_RUNNING.

MFC: r253284, r253285, r253303:

Correct the Intel network driver module builds. They were not
defining INET or INET6, and in the case of ixgbe this will cause
a panic in the TSO setup code, but in all cases the ioctl behavior
is different, this change makes the module and static consistent.

Approved by: re

MFC e1000 driver revisions: 248906,248908,249074,249339,249509
250108,250109,250168,250413,250414

MFC r250963, r251013, r251056: Driver 'aacraid' added to support Adaptec by PMC RAID controller families Series 6, 7, 8 and upcoming products.

Approved by: emaste (co-mentor)

MFC: sync the version of netmap with the one in HEAD, including device
drivers (mostly simplifying the code in the interrupt handlers).

On passing, also merge r250414, which is related to netmap
and the use of lem/em in virtual machines.

MFC r249070

Update man page for igb(4) with a little bit of information about
hw.igb.num_queues for those so inclined.

MFC r248933:

Use a shared lock for VOP_GETEXTATTR, as it is a read-like operation.

Improve error handling when unwrapping received data.

Submitted by: Rick Macklem

MFC of 246876 and 246877

MFC: 246876:

Add barrier write capability to the VFS buffer interface. A barrier
write is a disk write request that tells the disk that the buffer
being written must be committed to the media along with any writes
that preceeded it before any future blocks may be written to the drive.

Barrier writes are provided by adding the functions bbarrierwrite
(bwrite with barrier) and babarrierwrite (bawrite with barrier).

Following a bbarrierwrite the client knows that the requested buffer
is on the media. It does not ensure that buffers written before that
buffer are on the media. It only ensure that buffers written before
that buffer will get to the media before any buffers written after
that buffer. A flush command must be sent to the disk to ensure that
all earlier written buffers are on the media.

Reviewed by: kib
Tested by: Peter Holm

MFC 246877:

The UFS2 filesystem allocates new blocks of inodes as they are needed.
When a cylinder group runs short of inodes, a new block for inodes is
allocated, zero'ed, and written to the disk. The zero'ed inodes must
be on the disk before the cylinder group can be updated to claim them.
If the cylinder group claiming the new inodes were written before the
zero'ed block of inodes, the system could crash with the filesystem in
an unrecoverable state.

Rather than adding a soft updates dependency to ensure that the new
inode block is written before it is claimed by the cylinder group
map, we just do a barrier write of the zero'ed inode block to ensure
that it will get written before the updated cylinder group map can
be written. This change should only slow down bulk loading of newly
created filesystems since that is the primary time that new inode
blocks need to be created.

Reported by: Robert Watson
Reviewed by: kib
Tested by: Peter Holm

MFS of 246289:

For UFS2 i_blocks is unsigned. The current "sanity" check that it
has gone below zero after the blocks in its inode are freed is a
no-op which the compiler fails to warn about because of the use of
the DIP macro. Change the sanity check to compare the number of
blocks being freed against the value i_blocks. If the number of
blocks being freed exceeds i_blocks, just set i_blocks to zero.

Reported by: Pedro Giffuni (pfg@)

MFC of the E1000 drivers including revisions:
------------------------------------------------------------------------
r238765 | luigi | 2012-07-25 04:28:15 -0700 (Wed, 25 Jul 2012) | 7 lines
Use legacy interrupts as a default. This gives up to 10% speedup
when used in qemu (and this driver is for non-PCIe cards,
so probably its largest use is in virtualized environments).
------------------------------------------------------------------------
r238770 | luigi | 2012-07-25 05:51:33 -0700 (Wed, 25 Jul 2012) | 4 lines
remove some extra testing code that slipped into the previous commit
------------------------------------------------------------------------
r238953 | jfv | 2012-07-31 11:44:10 -0700 (Tue, 31 Jul 2012) | 6 lines
Clean up some unused leftover code from em
Make IRQ style a tuneable
Fix lock handling in the interrupt handler
------------------------------------------------------------------------
r238981 | sbruno | 2012-08-01 17:00:34 -0700 (Wed, 01 Aug 2012) | 9 lines
CPU_NEXT() already handles wrapping around to the beginning. Also, in a
system with sparse CPU IDs, you can have a valid CPU ID > mp_ncpus (e.g. if
you have two CPUs 0 and 4, with mp_maxid == 4 and mp_ncpus == 2).
------------------------------------------------------------------------
r239105 | jfv | 2012-08-06 13:44:05 -0700 (Mon, 06 Aug 2012) | 5 lines
Correct the mq_start routine to avoid out-of-order
packet delivery, always enqueue when possible. Also
correct the DEPLETED test as multiple bits might be
set. Thanks to Randall Stewart for the changes!
------------------------------------------------------------------------
r239109 | jfv | 2012-08-06 15:43:49 -0700 (Mon, 06 Aug 2012) | 6 lines
Make the polling interface in igb able to handle
multiqueue, and correct the rxdone handling. Update
the polling man page to include igb as well.
------------------------------------------------------------------------
r239304 | jfv | 2012-08-15 10:12:40 -0700 (Wed, 15 Aug 2012) | 10 lines
Customer report of a panic on boot due to the old
"m_getjcl:invalid cluster type" that occurred some
time back with the igb driver. This happens often when
booting over the net. I believe the NIC hardware is left
in a warm state when handed over to the driver, and a stray
RX interrupt happens earlier than the code is prepared for
it to happen. This change was verified to fix the problem,
its kind of a bandaid... but it is similar to what was done
in the igb code.
------------------------------------------------------------------------
r240693 | gavin | 2012-09-19 05:27:23 -0700 (Wed, 19 Sep 2012) | 5 lines
Switch some PCI register reads from using magic numbers to using the names
defined in pcireg.h
------------------------------------------------------------------------
r241856 | eadler | 2012-10-21 20:41:14 -0700 (Sun, 21 Oct 2012) | 7 lines
Now that device disabling is generic, remove extraneous code from the
device drivers that used to provide this feature.
------------------------------------------------------------------------
r241885 | eadler | 2012-10-22 06:06:09 -0700 (Mon, 22 Oct 2012) | 7 lines
This isn't functionally identical. In some cases a hint to disable
unit 0 would in fact disable all units. This reverts r241856
------------------------------------------------------------------------
r243570 | glebius | 2012-11-26 12:03:57 -0800 (Mon, 26 Nov 2012) | 14 lines
drbr_enqueue() awlays consumes mbuf, no matter did it
fail or not. The mbuf pointer is no longer valid, so
can't be reused after.
Fix igb_mq_start() where mbuf pointer was used after
drbr_enqueue().
This eventually leads us to all invocations of
igb_mq_start_locked() called with third argument as NULL.
This allows us to simplify this function.
------------------------------------------------------------------------
r245334 | smh | 2013-01-12 08:05:55 -0800 (Sat, 12 Jan 2013) | 9 lines
Fixed mbuf free when receive structures fail to allocate.
This prevents quad igb card on high core machines, without any nmbcluster or
igb queue tuning wedging the boot process if all nics are configured.
------------------------------------------------------------------------
r246128 | sbz | 2013-01-30 10:01:20 -0800 (Wed, 30 Jan 2013) | 5 lines
Use DEVMETHOD_END macro defined in sys/bus.h instead of {0, 0} sentinel on device_method_t arrays
------------------------------------------------------------------------
r246482 | rrs | 2013-02-07 07:20:54 -0800 (Thu, 07 Feb 2013) | 30 lines
This fixes a out-of-order problem with several
of the newer drivers. The basic problem was
that the driver was pulling the mbuf off the
drbr ring and then when sending with xmit(), encounting
a full transmit ring. Thus the lower layer
xmit() function would return an error, and the
drivers would then append the data back on to the ring.
For TCP this is a horrible scenario sure to bring
on a fast-retransmit.

The fix is to use drbr_peek() to pull the data pointer
but not remove it from the ring. If it fails then
we either call the new drbr_putback or drbr_advance
method. Advance moves it forward (we do this sometimes
when the xmit() function frees the mbuf). When
we succeed we always call advance. The
putback will always copy the mbuf back to the top
of the ring. Note that the putback *cannot* be used
with a drbr_dequeue() only with drbr_peek(). We most
of the time, in putback, would not need to copy it
back since most likey the mbuf is still the same, but
sometimes xmit() functions will change the mbuf via
a pullup or other call. So the optimial case for
the single consumer is to always copy it back. If
we ever do a multiple_consumer (for lagg?) we
will need a test and atomic in the put back possibly
a seperate putback_mc() in the ring buf.
------------------------------------------------------------------------
r247064 | jfv | 2013-02-20 16:25:45 -0800 (Wed, 20 Feb 2013) | 19 lines
Refresh on the shared code for the E1000 drivers.
- bear with me, there are lots of white space changes, I would not
do them, but I am a mere consumer of this stuff and if these drivers
are to stay in shape they need to be taken.

em driver changes: support for the new i217/i218 interfaces

igb driver changes:
- TX mq start has a quick turnaround to the stack
- Link/media handling improvement
- When link status changes happen the current flow control state
will now be displayed.
- A few white space/style changes.

lem driver changes:
- the shared code uncovered a bogus write to the RLPML register
(which does not exist in this hardware) in the vlan code,this
is removed.
------------------------------------------------------------------------

MFC: r243857 (partial)

Mechanically substitute flags from historic mbuf allocator with
malloc(9) flags in sys/dev.

MFC 240968:
Merge similar fixes from 223198 from igb to ixgbe:
- Use a dedicated task to handle deferred transmits from the if_transmit
method instead of reusing the existing per-queue interrupt task.
Reusing the per-queue interrupt task could result in both an interrupt
thread and the taskqueue thread trying to handle received packets on a
single queue resulting in out-of-order packet processing and lock
contention.
- Don't define ixgbe_start() at all when if_transmit is used.

MFC r230590 (by ken) except parts changing ABI:
Add CAM infrastructure to allow reporting when a drive's long read capacity
data changes.

MFC r244750:
Add net.link.stf.permit_rfc1918 sysctl variable. It can be used to allow
the use of private IPv4 addresses with stf(4).

MFC r244752:
Add an ability to set net.link.stf.permit_rfc1918 from the loader.

MFC: 234930

Fix so that ,usr and ,os work correctly with fixed function (IAF)
counters.

MFC: 238366, 240924

Initial commit of an I/O provider for DTrace on FreeBSD.

These probes are most useful when looking into the structures
they provide, which are listed in io.d. For example:

dtrace -n 'io:kernel::start { printf("%d\n", args[0]->bio_bcount); }'

Note that the I/O systems in FreeBSD and Solaris/Illumos are sufficiently
different that there is not a 1:1 mapping from scripts that work
with one to the other.

This commit includes the fix so that our probes use "kernel"
instead of the Solaris specific "genunix"

Merge r241037 from head:
The drbr(9) API appeared to be so unclear, that most drivers in
tree used it incorrectly, which lead to inaccurate overrated
if_obytes accounting. The drbr(9) used to update ifnet stats on
drbr_enqueue(), which is not accurate since enqueuing doesn't
imply successful processing by driver. Dequeuing neither mean
that. Most drivers also called drbr_stats_update() which did
accounting again, leading to doubled if_obytes statistics. And
in case of severe transmitting, when a packet could be several
times enqueued and dequeued it could have been accounted several
times.

o Thus, make drbr(9) API thinner. Now drbr(9) merely chooses between
ALTQ queueing or buf_ring(9) queueing.
- It doesn't touch the buf_ring stats any more.
- It doesn't touch ifnet stats anymore.
- drbr_stats_update() no longer exists.

o buf_ring(9) handles its stats itself:
- It handles br_drops itself.
- br_prod_bytes stats are dropped. Rationale: no one ever
reads them but update of a common counter on every packet
negatively affects performance due to excessive cache
invalidation.
- buf_ring_enqueue_bytes() reduced to buf_ring_enqueue(), since
we no longer account bytes.

o Drivers handle their stats theirselves: if_obytes, if_omcasts.

o mlx4(4), igb(4), em(4), vxge(4), oce(4) and ixv(4) no longer
use drbr_stats_update(), and update ifnet stats theirselves.

o bxe(4) was the most correct driver, it didn't call
drbr_stats_update(), thus it was the only driver accurate under
moderate load. Now it also maintains stats itself.

o ixgbe(4) had already taken stats from hardware, so just
- drop software stats updating.
- take multicast packet count from hardware as well.

o mxge(4) just no longer needs NO_SLOW_STATS define.

o cxgb(4), cxgbe(4) need no change, since they obtain stats
from hardware.

Reviewed by: jfv, gnn

MFC r241844:
remove duplicate semicolons where possible.

Approved by: cperciva (implicit)

MFC r241917:
Now that device disabling is generic, remove extraneous code from the
device drivers that used to provide this feature.

Approved by: cperciva (implicit)

Merge r240680 from head:

Align the PCI Express #defines with the style used for the PCI-X
#defines. This has the advantage that it makes the names more
compact, and also allows us to correct the non-uniform naming of
the PCIM_LINK_* defines, making them all consistent amongst themselves.

This is a mostly mechanical rename:
s/PCIR_EXPRESS_/PCIER_/g
s/PCIM_EXP_/PCIEM_/g
s/PCIM_LINK_/PCIEM_LINK_/g

In this MFC, #defines have been added for the old names to assist
out-of-tree drivers.

MFC: r239864

- Unlike cache invalidation and TLB demapping IPIs, reading registers from
other CPUs doesn't require locking so get rid of it. As the latter is used
for the timecounter on certain machine models, using a spin lock in this
case can lead to a deadlock with the upcoming callout(9) rework.
- Merge r134227/r167250 from x86:
Avoid cross-IPI SMP deadlock by using the smp_ipi_mtx spin lock not only
for smp_rendezvous_cpus() but also for the MD cache invalidation and TLB
demapping IPIs.
- Mark some unused function arguments as such.

MFC r240879

This patch fixes a nit in the em, lem, and igb driver statistics. Increment
adapter->dropped_pkts instead of if_ierrors because if_ierrors is
overwritten by hw stats collection.

Submitted by: Andrew Boyer <aboyer@averesystems.com>
Reviewed by: Jack F Vogel <jfv@freebsd.org>

MFC r239257:

Reserve room for the terminating NUL when setting or getting kernel
environment variables. KENV_MNAMELEN and KENV_MVALLEN doesn't include
space for the terminating NUL.

MFC: r240720
Modify the NFSv4 client so that it can handle owner
and owner_group strings that consist entirely of
digits, interpreting them as the uid/gid number.
This change was needed since new (>= 3.3) Linux
servers reply with these strings by default.
This change is mandated by the rfc3530bis draft.
Reported on freebsd-stable@ under the Subject
heading "Problem with Linux >= 3.3 as NFSv4 server"
by Norbert Aschendorff on Aug. 20, 2012.

MFC 239779:
Shorten the name of the fast SWI taskqueue to "fast taskq" so that
it fits.

MFC 239008,239020:
Improve the handling of static DMA buffers that use non-default memory
attributes (currently just BUS_DMA_NOCACHE):
- Don't call pmap_change_attr() on the returned address, instead use
kmem_alloc_contig() to ask the VM system for memory with the requested
attribute.
- As a result, always use kmem_alloc_contig() for non-default memory
attributes, even for sub-page allocations. This requires adjusting
bus_dmamem_free()'s logic for determining which free routine to use.
- For x86, add a new dummy bus_dmamap that is used for static DMA
buffers allocated via kmem_alloc_contig(). bus_dmamem_free() can then
use the map pointer to determine which free routine to use.
- For powerpc, add a new flag to the allocated map (bus_dmamem_alloc()
always creates a real map on powerpc) to indicate which free routine
should be used.

Note that the BUS_DMA_NOCACHE handling in powerpc is currently #ifdef'd out.
I have left it disabled but updated it to match x86.

MFC 239771: Fix misspelled "Infiniband".

MFC: r240289
Add a simple printf() based debug facility to the new nfs client.
Use it for a printf() that can be harmlessly generated for mmap()'d
files. It will be used extensively for the NFSv4.1 client.
Debugging printf()s are enabled by setting vfs.nfs.debuglevel to
a non-zero value. The higher the value, the more debugging printf()s.

MFC r240518:
Correct double "the the"

Approved by: cperciva (implicit)

Merge rev. 239673 from head to stable/9:
(g_multipath_rotate): Fix algorithm so that it does rotate over all good
providers, not just the last two.

MFC: merge rev. 239012 from head to stable/9
New command "gmultipath prefer" to force selection of a specified
provider in an Active/Passive configuration.

MFC 239128:
Don't try to stop the IPMI watchdog timer if it is not running.
Starting or stopping the IPMI watchdog is rather expensive with the
current implementation as all IPMI requests are bounced via thread.
This is not viable during shutdown or dumps, and this avoids headache
in the common case that the watchdog is not enabled. The IPMI watchdog
should probably be reworked to not use a separate thread to fix this
in the case when the watchdog timer is enabled.

MFC 239103:
Explicitly enable busmastering on PCI-PCI bridges. Transactions initiated
on the secondary side of a bridge will not be propagated to the primary
bus unless this is enabled. Busmastering is not enabled by default (we
have relied on firmware to set this bit to date). The OS needs to set it
for any bridges not configured by system firmware.

MFC 238424:
Make the interval timings for EVFILT_TIMER more accurate. tvtohz() always
adds an extra tick to account for the current partial clock tick. However,
that is not appropriate for a repeating timer when the exact tvtohz() value
should be used for subsequent intervals. Fix repeating callouts for
EVFILT_TIMER by subtracting 1 tick from the tvtohz() result similar to the
fix used in realitexpire() for interval timers.

While here, update a few comments to note that if the EVFILT_TIMER code
were to move out of kern_event.c, it should move to kern_time.c (where the
interval timer code it mimics lives) rather than kern_timeout.c.

MFC 238311:
Add a clts() wrapper around the 'clts' instruction to <machine/cpufunc.h>
on x86 and use that to implement stop_emulating() in the fpu/npx code.
Reimplement start_emulating() in the non-XEN case by using load_cr0() and
rcr0() instead of the 'lmsw' and 'smsw' instructions. Intel explicitly
discourages the use of 'lmsw' and 'smsw' on 80386 and later processors in
the description of these instructions in Volume 2 of the ADM.

MFC 238142,238179:
Now that our assembler supports the xsave family of instructions, use them
natively rather than hand-assembled versions. For xgetbv/xsetbv, add a
wrapper API to deal with xcr* registers: rxcr() and load_xcr().

MFC 238310:
Partially revert r217515 so that the mem_range_softc variable is always
present on x86 kernels. This fixes the build of kernels that include
'device acpi' but do not include 'device mem'.

MFC 238109,238166:
Several fixes to the amd64 disassembler:
- Decode the 'xsave', 'xrstor', 'xsaveopt', 'xgetbv', 'xsetbv', and
'rdtscp' instructions.
- Add generic support for opcodes that are escape bytes used for
multi-byte opcodes (such as the 0x0f prefix). Use this to replace
the hard-coded 0x0f special case and add support for three-byte
opcodes that use the 0x0f38 prefix.
- Decode all Intel VMX instructions. invept and invvpid in particular are
three-byte opcodes that use the 0x0f38 escape prefix.
- Rework how the special 'SDEP' size flag works such that the default
instruction name (i_name) is the instruction when the data size
prefix (0x66) is not specified, and the alternate name in i_extra is
used when the prefix is included.
- Add a new 'ADEP' size flag similar to 'SDEP' except that it chooses
between i_name and i_extra based on the address size prefix (0x67).
Use this to fix the decoding for jrcxz vs jecxz which is determined
by the address size prefix, not the operand size prefix. Also, jcxz
is not possible in 64-bit mode, but jrcxz is the default instruction
for that opcode.
- Add support for handling instructions that have a mandatory 'rep'
prefix (this means not outputting the 'repe ' prefix until determining
if it is used as part of an opcode). Make 'pause' less of a special
case this way.
- Decode 'cmpxchg16b' and 'cdqe' which are variants of other instructions
but with a REX.W prefix.

MFC 238000,239584:
Honor db_pager_quit in 'show malloc', 'show uma', and 'show witness'.

MFC 238077:
Fix panics triggered by older mfiutil binaries run on the new mfi(4) driver.
The new driver changed the size of the mfi_dcmd_frame structure in such a
way that a MFI_IOC_PASSTHRU ioctl from an old amd64 binary is treated as an
MFI_IOC_PASSTHRU32 ioctl in the new driver. As a result, the user pointer
is treated as the buffer length. mfi_user_command() doesn't have a bounds
check on the buffer length, so it passes a really big value to malloc()
which panics when it tries to exhaust the kmem_map. Fix this two ways:
- Only honor MFI_IOC_PASSTHRU32 if the binary has the SV_ILP32 flag set,
otherwise treat it as an unknown ioctl.
- Add a bounds check on the buffer length passed by the user. For now
it fails any user attempts to use a buffer larger than 1MB.

While here, fix a few other nits:
- Remove an unnecessary check for a NULL return from malloc(M_WAITOK).
- Use the ENOTTY errno for invalid ioctl commands instead of ENOENT.

MFC 237338:
Don't return an error if a kld does not contain any modules (e.g. a
kld that only contained a sysctl). The kernel linker allows such
modules, so the boot loader should not reject them.

MFC 230782,237274:
Refine the implementation of POSIX_FADV_NOREUSE to perform
POSIX_FADV_DONTNEED requests on the currently accessed portion of the
file on each read(2) or write(2) rather than using direct I/O. This
gives much better performance including read-ahead and write clustering
similar to normal read(2) and write(2) calls.

If subsequent read(2) and write(2) calls are sequential, then the
POSIX_FADV_DONTNEED requests will cover the entire sequentially-accessed
range.

MFC 233040,233198,233870,234183:
Add OFED and the associated options and drivers to x86 LINT builds:
- Fix build with INET6 disabled.
- Fix build of OFED bits with debugging options enabled.
- Fix build on i386.
- Mark 'sdp' as requiring 'inet'.
- Always include "opt_inet.h" and "opt_inet6.h" and modify the IB
driver Makefiles to honor WITH/WITHOUT_INET/INET6/_SUPPORT options
to determine what should be enabled during a module build.
- Fix the mlxen(4) driver and the core IB code to compile without
if INET is disabled (including when both INET and INET6 are disabled).

MFC 233872:
Add descriptions after the 'device' line for several NICs to match the
existing style.

MFC r238502:

Fix a bug with memguard(9) on 32-bit architectures without a
VM_KMEM_MAX_SIZE.

The code was not taking into account the size of the kernel_map, which
the kmem_map is allocated from, so it could produce a sub-map size too
large to fit. The simplest solution is to ignore VM_KMEM_MAX entirely
and base the memguard map's size off the kernel_map's size, since this
is always relevant and always smaller.

Found by: Justin Hibbits

MFC: r239079

Merge r236494 from x86:

Isolate the global TTE list lock from data and other locks to prevent false
sharing within the cache.

MFC 238164:
Add another PS/2 keyboard PNP ID. This ID is listed as
"Reserved by Microsoft" in the standard PNP ID table, but has been seen
in the wild on at least one laptop.

Approved by: re (kib)

Pull the tier-2 card and change the sparc64 ZFS loader to no longer probe
all diskN aliases for providers (which more or less corresponds to how the
x86 version behaves) but instead probe only those listed in the boot-device
OFW environment variable. This has the following advantages:
- avoids otherwise unavoidable OFW warnings about failures to open disks
for which aliases exist but no actual hardware is connected
- avoids issues due to different diskN naming schemes
- aligns us with Solaris

Approved by: re (kib)

MFC: r238621

Revert the use of BUS_DMA_ALLOCNOW when creating the DMA tag for user
data introduced in r236061 (MFC'ed to stable/9 in r237186). Using that
flag doesn't make that much sense on this case as the DMA maps using
it are also created during sym_pci_attach(). Moreover, due to the
maxsegsz parameter used, doing so may exhaust the bounce pages pool
on architectures requiring bounce pages. [1]
While at it, use a slightly more appropriate maxsegsz parameter.

PR: 169526
Submitted by: Mike Watters [1]
Approved by: re (kib)

MFC of the e1000 drivers: 236406,238148,238151,238181, and 238214

Approved by:re

MFC r237852

Add the same check as vlan(4) where we ignore the ifnet departure event if the
interface is just being renamed.

PR: kern/169557

MFC: r237547

Fix size of the bcopy when extracting ethernet address

Obtained from: DragonFly

MFC: r237546

Correct sizeof usage

Obtained from: DragonFly

MFC: r237842

Switch back to the 4BSD scheduler for now. There is some more or less
recent regression with ULE, causing processes to get stuck in getblk
as well as interrupt handler execution delays to rise above the command
timeout of mpt(4).

Merge r237736 from HEAD:
Kick the current-state report timer when a V1 group report would
be triggered.

Submitted by: rpaulo@

Merge r237735 from HEAD:
Fix a typo in MLD query exponent processing.

Submitted by: rpaulo@

Merge r237734 from HEAD:
In MLDv2 general query processing, do not enforce the strict check
on query origins.

Submitted by: Gu Yong

MFC r236503: free wdog_kern_pat calls in post-panic paths from under
SW_WATCHDOG

MFC 235024,235029,235556,235834,235845:
Use MADT to match ACPI Processor objects to CPUs. MADT and DSDT/SSDTs may
list CPUs in different orders, especially for disabled logical cores. Now
we match ACPI IDs from the MADT with Processor objects, strictly order CPUs
accordingly, and ignore disabled cores. This prevents us from executing
methods for other CPUs, e. g., _PSS for disabled logical core, which may not
exist. Unfortunately, it is known that there are a few systems with buggy
BIOSes that do not have unique ACPI IDs for MADT and Processor objects. To
work around these problems, 'debug.acpi.cpu_unordered' tunable is added.
Set this to a non-zero value to restore the old behavior.

MFC: r236581

The loaddev environment variable is not modifiable once set, so it is not
update for ZFS. It seems that this does not really affect anything except
the help command. Nevertheless, rearrange things so loaddev is set only
once in all cases in order to get it right.
Pointed out by: avg

MFC 233191:
Fix madvise(MADV_WILLNEED) to properly handle individual mappings larger
than 4GB. Specifically, the inlined version of 'ptoa' of the the 'int'
count of pages overflowed on 64-bit platforms. While here, change
vm_object_madvise() to accept two vm_pindex_t parameters (start and end)
rather than a (start, count) tuple to match other VM APIs as suggested
by alc@.

MFC 237334:
Move the per-thread deferred user map entries list into a private list
in vm_map_process_deferred() which is then iterated to release map entries.
This avoids having a nested vm map unlock operation called from the loop
body attempt to recuse into vm_map_process_deferred(). This can happen if
the vm_map_remove() triggers the OOM killer.

MFC 237008,237271,237272,237673:
- Fix a couple of bugs that prevented windows in PCI-PCI bridges from
growing "downward" (moving the start address down). First, an off by
one error caused the end address to be moved down an extra alignment
chunk unnecessarily. Second, when aligning the new candidate starting
address, the wrong bits were masked off.
- Add a 'wmask' variable to hold the expression '(1ul << w->step) - 1' in
pcib_grow_window().
- For subtractively decoding bridges, don't try to grow windows but pass
the request up the tree in order to be on the safe side. Growing windows
in this case would mean to switch resources to positive decoding and
it's unclear how to correctly handle this. At least with ALi/ULi M5249
PCI-PCI bridges, this also just doesn't work out of the box.

MFC r235390: zfs boot code: mark spa_t arguments as const where they are
used as such

MFC r235392,235394,235395: fixes and cleanups for zfs boot MFC

MFC r235330: zfs boot: try to set vfs.root.mountfrom from currdev as a
fallback

MFC r235329,235343,235361,235364: zfsboot/zfsloader: support accessing
filesystems within a pool

MFC r235264: MFi386: improve argument passing via btxldr

MFC r235219: cdboot, pxeldr: make use of bootargs.h instead of
redefining flag constants

MFC r235156: i386 zfsloader: rename LIBZFS to LIBZFSBOOT

MFC r235155: i386 boot: consolidate MAXBDDEV definition

MFC r235153: sys/boot: add common CTASSERT definition

MFC 228161,230774,230822,236415:
Add a new -e flag to pciconf(8)'s list mode to display PCI error details.
Currently this dumps the status of any error bits in the PCI status register
and PCI-express device status register. It also lists any errors indicated
by version 1 of PCI-express Advanced Error Reporting (AER).

MFC 236405:
Remove unnecessary initializations. The BSS of boot2 is in fact
zero'd when boot2 begins execution by the _start() routine in btxcsu.S.

MFC 236404:
Extend VERBOSE_SYSINIT to also print out the name of variables passed
to SYSINIT routines if they can be resolved via symbol look up in DDB.
To avoid false positives, only honor a name if the symbol resolves
exactly to the pointer value (no offset).

MFC 234494:
Include the associated wait channel message for context switch ktrace
records. kdump supports both the old and new messages.

MFC r236178

if_lagg: allow to invoke SIOCSLAGGPORT multiple times in a row

MFC r236062

Turn LACP debugging from a compile time option to a sysctl, it is very handy to
be able to turn it on when negotiation to a switch misbehaves.

MFC 233925,236357:
Add new ktrace records for the start and end of VM faults. This gives
a pair of records similar to syscall entry and return that a user can
use to determine how long page faults take. The new ktrace records are
enabled via the 'p' trace type, but are not enabled in the default set of
trace points.

MFC: r237244
Fix the NFSv4 client for the case where mmap'd files are
written, but not msync'd by a process. A VOP_PUTPAGES()
called when VOP_RECLAIM() happens will usually fail, since
the NFSv4 Open has already been closed by VOP_INACTIVE().
Add a vm_object_page_clean() call to the NFSv4 client's
VOP_INACTIVE(), so that the write happens before the NFSv4
Open is closed. kib@ suggested using vgone() instead and
I will explore this, but this patch fixes things in the
meantime. For some reason, the VOP_PUTPAGES() is still
attaempted in VOP_RECLAIM(), but having this fail doesn't
cause any problems except a "stateid0 in write" being logged.

MFC: r237200
Move the nfsrpc_close() call in ncl_reclaim() for the NFSv4 client
to below the vnode_destroy_vobject() call, since that is where
writes are flushed.

MFC: r231616, r232497, r234337

Add __aeabi_read_tp function required for thread-local storage.

MFC 230063
Clean up a switch statement for uncore events on Westmere processors.

Submitted by: Davide Italiano
Reviewed by: gnn

MFC: r231617

Add ARM relocations types used for thread-local storage

Reviewed by: cognet

MFC: r235348, r236191, r236497

Add glue/support for the SAM9XE512-based Ethernut 5 boards. Currently,
all integrated and on-board peripherals except NAND Flash (missing NAND
framework/integration) are working.

MFC: r236496

- Loop up to 3 seconds when waiting for a device to get ready. [1]
- Make the device description match the driver name.
- Identify the chip variant based on the JEDEC and use that information
to use the proper values for page count, offset and size instead of
hardcoding a AT45DB642x with 2^N byte page support disabled.
- Take advantage of bioq_takefirst().
- Given that CONTINUOUS_ARRAY_READ_HF (0x0b) command isn't even mentioned
in Atmel's DataFlash Application Note, as suggested by the previous
comment may not work on all all devices and actually doesn't properly
on at least AT45DB321D (JEDEC 0x1f2701), rewrite at45d_task() to use
CONTINUOUS_ARRAY_READ (0xe8) for reading instead. This rewrite is laid
out in a way allowing to easily add support for BIO_DELETE later on.
- Add support for reads and writes not starting on a page boundary.
- Verify the flash content after writing.
- Let at45d_task() gracefully handle errors on SPI transfers and the
device not becoming ready afterwards again. [1]
- Use DEVMETHOD_END. [1]
- Use NULL instead of 0 for pointers. [1]

Additional testing by: Ian Lepore

Submitted by: Ian Lepore [1]

MFC: r237239

Revert the part of r236495 (MFC'ed to stable/9 in r237095) that
introduced checking of SPI_SR_TXEMPTY for TX transfer completion as
for reasons unknown this occasionally causes SPI_SR_RXBUFF and
SPI_SR_ENDRX to not rise.
In any case, once the RX part of the transfer is done it's obvious
that the preceding TX part had finished and checking of SPI_SR_TXEMPTY
was introduced to rule out a possible cause for the data corruption
mentioned in r236495 but which didn't turn out to be the problem
anyway.

MFC: r230242, r237102, r237236

- Add support for the FT2232 based egnite Turtelizer 2 JTAG/RS232 Adapter.
This includes adding support for skipping FTDI interfaces used for JTAG
leaving them for userland and just attaching to the RS232 half, similarly
to how the corresponding Linux drivers handles these kind of adapters.
While at it, sort uftdi_devs and return BUS_PROBE_SPECIFIC (because
uftdi_probe() alters the instance variables for better or worse as do
other probe routines of USB drivers) instead of 0.
- Remove duplicated entries for BeagleBone.
- Use DEVMETHOD_END.
- Use NULL instead of 0 for pointers.
- Remove some stray lines.

MFC of 236937

In softdep_setup_inomapdep() we may have to allocate both inodedep
and bmsafemap dependency structures in inodedep_lookup() and
bmsafemap_lookup() respectively. The setup of these structures must
be done while holding the soft-dependency mutex. If the inodedep is
allocated first, it may be freed in the I/O completion callback when
the mutex is released to allocate the bmsafemap. If the bmsafemap is
allocated first, it may be freed in the I/O completion callback when
the mutex is released to allocate the inodedep.

To resolve this problem, bmsafemap_lookup has had a parameter added
that allows a pre-malloc'ed bmsafemap to be passed in so that it does
not need to release the mutex to create a new bmsafemap. The
softdep_setup_inomapdep() routine pre-malloc's a bmsafemap dependency
before acquiring the mutex and starting to build the inodedep with a
call to inodedep_lookup(). The subsequent call to bmsafemap_lookup()
is passed this pre-allocated bmsafemap entry so that it need not
release the mutex if it needs to create a new one.

Reported by: Peter Holm
Tested by: Peter Holm

MFC of 236825

When synchronously syncing a device (MNT_WAIT), wait for buffers
to become available. Otherwise we may excessively spin and fail
with ``fsync: giving up on dirty''.

Reviewed by: kib
Tested by: Peter Holm

MFC r236246

Add device ids for the Winbond 83627DHG-P chip and set the registers to
trigger the keyboard reset line on timeout.

MFC r236245

Fix a typo in wbwd so that CRF5 is actually written to the data register
rather than the index register.

MFC: r237107

- As a baind-aid, disable ATAPI DMA when using ATA_CAM for these controllers
as well as it causes the kernel to hang during boot.
Reported and tested by: Kevin Oberman
- Use NULL instead of 0 for a pointer.

MFC: r230179

BeagleBone uses an FTDI chip with
an altered Product ID.

MFC: r237101

Fix a braino in r236469 (MFC'ed to stable/9 in r236468); the number of
DMA tags required for handling MAXPHYS should be based on PAGE_SIZE rather
than SYM_CONF_DMA_BOUNDARY.
While at it, reuse the SYM_CONF_MAX_SG macro for specifying the maximum
number of DMA tags so sym(4) itself doesn't size memory beyond what's
required for handling MAXPHYS.

PR: 168928

MFC r236916

Fix a panic I introduced in r234487, the bridge softc pointer is set to null
early in the detach so rearrange things not to explode.

Reported by: David Roffiaen, Gustau Perez Querol

MFC: r236495

- Prepend the device description with "AT91" to reflect its nature. [1]
- Move DMA tag and map creature to at91_spi_activate() where the other
resource allocation also lives. [1]
- Flesh out at91_spi_deactivate(). [1]
- Work around the "Software Reset must be Written Twice" erratum.
- For now, run the bus at the slowest speed possible in order to work
around data corruption on transit even seen with 9 MHz on ETHERNUT5
(15 MHz maximum) and AT45DB321D (20 MHz maximum). This also serves as
a poor man's work-around for the "NPCSx rises if no data data is to be
transmitted" erratum of RM9200. Being able to use the appropriate bus
speed would require:
1) Adding a proper work-around for the RM9200 bug consisting of taking
the chip select control away from the SPI peripheral and managing it
directly as a GPIO line.
2) Taking the maximum frequencies supported by the actual board and the
slave devices into account and basing the whole thing on the master
clock instead of hardcoding a divisor as previously done.
3) Fixing the above mentioned data corruption.
- KASSERT that TX/RX command and data sizes match on transfers.
- Introduce a mutex ensuring that only one child device is running a SPI
transfer at a time. [1]
- Add preliminary, #ifdef'ed out support for setting the chip select. [1]
- Use the RX instead of the TX commando size when setting up the RX side
of a transfer.
- For controllers having SPI_SR_TXEMPTY, i.e. !RM9200, also wait for the
completion of the TX part of transfers before stopping the whole thing
again.
- Use DEVMETHOD_END. [1]
- Use NULL instead of 0 for pointers. [1, partially]

Additional testing by: Ian Lepore

Submitted by: Ian Lepore [1]

MFC: r225882

Remove pointless semicolons after label

MFC: r236579

The workaround added in r151650 for handling firmwares that don't allow
a single device to be opened multiple times concurrently unfortunately
isn't sufficient with ZFS. This is due to the fact, that ZFS may open
different partitions of a single device simultaneously. So the best we
can do in this case is to cache the lastly used device path and close
and open devices in ofwd_strategy() as needed.

PR: 165025
Submitted by: Gavin Mu

MFC: r235381
Fix two cases in the new NFS server where a tsleep() is
used, when the code should actually protect the tested
variable with a mutex. Since the tsleep()s had a 10sec
timeout, the race would have only delayed the allocation
of a new clientid for a client. The sleeps will also
rarely occur, since having a callback in progress when
a client acquires a new clientid, is unlikely.
in practice, since having a callback in progress when
a fresh clientid is being acquired by a client is unlikely.

MFC r236371:
Remove unnecessary device_printfs.

MFC 233760:
Export some more useful info about shared memory objects to userland
via procstat(1) and fstat(1):
- Change shm file descriptors to track the pathname they are associated
with and add a shm_path() method to copy the path out to a caller-supplied
buffer.
- Use the fo_stat() method of shared memory objects and shm_path() to
export the path, mode, and size of a shared memory object via
struct kinfo_file.
- Add a struct shmstat to the libprocstat(3) interface along with a
procstat_get_shm_info() to export the mode and size of a shared memory
object.
- Change procstat to always print out the path for a given object if it
is valid.
- Teach fstat about shared memory objects and to display their path,
mode, and size.

MFC 228509,228620,228533:
Add a helper API to allow in-kernel code to map portions of shared memory
objects created by shm_open(2) into the kernel's address space. This
provides a convenient way for creating shared memory buffers between
userland and the kernel without requiring custom character devices.

MFC: r236491

Add missing prototypes. While at it, sort them alphabetically.

MFC: r236488

Take advantage of nitems().

MFC: r236486

Add nitems(), a macro for determining the number of elements in a
statically-allocated array.

Obtained from: OpenBSD (in principle)

MFC: r236489

MFC: r233666

Fix build after changes to trap headers.

MFC: r233635

Allow multiple inclusion of trap.h. This has always been broken, but
until recently never caused problems.

MFC: r236328

Try to finally get the point in time at which bge_add_sysctls() is called
right; it needs to be called before bge_can_use_msi() but in turn requires
bge_flags to be properly set.

Submitted by: yongari

MFC r235829: vm_pager_object_lookup: small performance optimization

MFC: r236156

- Fix some typos in mmc_acquire_bus() and mmc_send_csd().
- Fix some math errors in mmc_decode_csd_sd().
- Fix incorrect arguments to mmc_send_app_op_cond() in mmc_go_discovery().
- Add reporting of CSD for debug purposes.
- Add detection (and skipping) of password-locked cards.
- Add setting of block length on card if necessary.

Submitted by: Patrick Kelsey

MFC: r236070

Consistently use USB_PAGE_SIZE. Currently, this is cosmetic.

MFC: r236069, r236073

Make the VIA workaround actually do its intended job.

MFC: r236063

Remove extraneous empty lines.

Revert changes accidentally committed as part of r236468.

MFC: r236061

- When creating the DMA tag for user data, don't ask for more segments
than required for handling MAXPHYS and report the resulting maximum
I/O size to CAM instead of implicitly limiting it to DFLTPHYS.
- Move the variables of sym_action2() out of nested scope as required
by style(9) and remove extraneous curly braces.
- Replace a magic value for PCIR_COMMAND with the appropriate macro.
- Use DEVMETHOD_END.
- Use NULL instead of 0 for pointers.

Tested with a HBA donated by wilko.

MFC r235391: i386 bootinfo: re-arrange EFI fields for natural alignment
and packing

MFC 236212:

Update mxge(4) firmware to the latest version available from
Myricom (1.4.55).

Sponored by: Myricom, Inc.

MFC 235833:
Only check to see if a memory resource is a PCI ROM BAR when activating
and deactivating PCI resources. Previously, if a device had more than
48 MSI interrupts, then activating message 48 (which has a rid == PCIR_BIOS)
would incorrectly try to enable the PCI ROM BAR.

MFC 234501:
The amr(4) firmware contains a rather dubious "feature" where it
assumes for small buffers (< 64k) that the OS driver is actually using
a buffer rounded up to the next power of 2. It also assumes that the
buffer is at least 4k in size. Furthermore, there is at least one
known instance of megarc sending a request with a 12k buffer where the
firmware writes out a 24k-ish reply.

To workaround the data corruption triggered by this "feature", ensure
that buffers for user commands use a minimum size of 32k, and that
buffers between 32k and 64k use a 64k buffer.

MFC 234099:
Properly parse 40G media types from newer Mellanox adapters that are
40G capable. For now, map all 40G links to 40GBase-CR4.

MFC r235821:
Don't force max payload size to 128. Root complex and Endpoint will
negotiate with each other on the TLP payload size so blindly
forcing the size to 128 can cause a completion error which in turn
will stop device.

Reported by: Geans Pin < geanspin <> broadcom dot com >

MFC r235816:
Make IPMI work in the bce driver even when the interface is
configured down. Formerly, IPMI communication was lost whenever the
interface was not up. The reason was that the BCE_EMAC_MODE
register was not configured with the correct media settings. There
are two parts to the fix.

First, resetting the chip in bce_reset() causes the BCE_EMAC_MODE
register to be initialized to a default value that does not
necessarily correspond to the actual media settings. The fix
implemented here is a bit of a hack. Ideally, at the end of
bce_reset() we would poll the PHY to determine the negotiated media,
and then we would set the BCE_EMAC_MODE register accordingly. That
is difficult, since the PHY is abstracted behind the MII layer and is
not supposed to be queried directly from the MAC driver. Instead,
we read the BCE_EMAC_MODE register at the beginning of bce_reset()
and then restore its media bits to their original values before
returning. If IPMI is up and running, then the link is already
established and the BCE_EMAC_MODE register is already set appropriately
when bce_reset() is called. If IPMI is not running, no harm is
done by preserving the BCE_EMAC_MODE settings. The driver will set
the register properly once the interface is configured up and link
is established.

Second, bce_miibus_statchg() is sometimes called when the link is
down. In that case, the reported media settings are invalid.
Formerly, the driver used them anyway to setup the BCE_EMAC_MODE
register. We now avoid changing any MAC registers unless link is
active and the reported media settings are valid.

Submitted by: jdp
Tested by: jdp

MFC of 235610

Add missing `continue' statement at end of case.

Found by: Kevin Lo (kevlo@)

MFC: r234740
Fix a leak of namei lookup path buffers that occurs when a
ZFS volume is exported via the new NFS server. The leak occurred
because the new NFS server code didn't handle the case where
a file system sets the SAVENAME flag in its VOP_LOOKUP() and
ZFS does this for the DELETE case.

MFC: r235332
PR# 165923 reported intermittent write failures for dirty
memory mapped pages being written back on an NFS mount.
Since any thread can call VOP_PUTPAGES() to write back a
dirty page, the credentials of that thread may not have
write access to the file on an NFS server. (Often the uid
is 0, which may be mapped to "nobody" in the NFS server.)
Although there is no completely correct fix for this
(NFS servers check access on every write RPC instead of at
open/mmap time), this patch avoids the common cases by
holding onto a credential that recently opened the file
for writing and uses that credential for the write RPCs
being done by VOP_PUTPAGES() for both NFS clients.

MFC: r234524

o Fixes:
- When switching to 4-bit operation, send a SET_CLR_CARD_DETECT command
to disconnect the card-detect pull-up resistor from the DAT3 line before
sending the SET_BUS_WIDTH command.
- Add the missing "reserved" zero entry to the mantissa table used to
decode various CSD fields. This was causing SD cards to report that they
could run at 30 MHz instead of the maximum 25 MHz mandated in the spec.
o Enhancements:
- At the MMC layer, format various info from the CID into a string that
uniquely identifies the card instance (manufacturer number, serial
number, product name and revision, etc). Export it as an instance
variable.
- At the MMCSD layer, display the formatted card ID string, and also
report the clock speed of the hardware (not the card's max speed), and
the number of bits and number of blocks per transfer. It comes out like
this now:
mmcsd0: 968MB <SD SD01G 8.0 SN 276886905 MFG 08/2008 by 3 SD> at mmc0
22.5MHz/4bit/128-block
o Use DEVMETHOD_END.
o Use NULL instead of 0 for pointers.

PR: 156496
Submitted by: Ian Lepore

MFC: r234901

- Add missing locking in at91_usart_getc().
- Align the RX buffers on the cache line size, otherwise the requirement
of partial cache line flushes on every are pretty much guaranteed. [1]
- Make the code setting the RX timeout match its comment (apparently,
start and stop bits were missed in the previous calculation). [1]
- Cover the busdma operations in at91_usart_bus_{ipend,transmit}() with
the hardware mutex, too, so these don't race against each other.
- In at91_usart_bus_ipend(), reduce duplication in the code dealing with
TX interrupts.
- In at91_usart_bus_ipend(), turn the code dealing with RX interrupts
into an else-if cascade in order reduce its complexity and to improve
its run-time behavior.
- In at91_usart_bus_ipend(), add missing BUS_DMASYNC_PREREAD calls on
the RX buffer map before handing things over to the hardware again. [1]
- In at91_usart_bus_getsig(), used a variable of sufficient width for
storing the contents of USART_CSR.
- Use KOBJMETHOD_END.
- Remove an unused header.

Submitted by: Ian Lepore [1]
Reviewed by: Ian Lepore

MFC: r234561

Interrupts must be disabled while handling a partial cache line flush,
as otherwise the interrupt handling code may modify data in the non-DMA
part of the cache line while we have it stashed away in the temporary
stack buffer, then we end up restoring a stale value.

PR: 160431
Submitted by: Ian Lepore

MFC: r234560

- Add support for MCI1 revision 2xx controllers and a work-around for their
"Data Write Operation and number of bytes" erratum.
- Use DEVMETHOD_END.
- Use NULL instead of 0 for pointers.

MFC: r234293

Generate an obviously missing STOP when having finished transmitting data.
This fixes communication with PCF8563.

MFC: r234291, r234292

Add support for the Atmel SAM9XE family of microcontrollers, which
consist of a ARM926EJ-S processor core with up to 512 Kbytes of on-chip
flash. Tested with SAM9XE512.

MFC: r234281

- Try to bring these files closer to style(9).
- Use DEVMETHOD_END.
- Use NULL instead of 0 for pointers.

MFC: r234248

Add a driver for the NXP (Philips) PCF8563 RTC.

Obtained from: NetBSD (pcf8563reg.h)

MFC: r234898, r235207

Add initial support for booting from ZFS on sparc64. At least on Sun Fire
V100, the firmware is known to be broken and not allowing to simultaneously
open disk devices, causing attempts to boot from a mirror or RAIDZ to cause
a crash. This will be worked around later. The firmwares of newer sun4u models
don't seem to exhibit this problem though.

PR: 165025
Submitted by: Gavin Mu

MFC r234936 (emaste)

Relax restriction on direct tx to child ports

Lagg(4) restricts the type of packet that may be sent directly to a child
port, to avoid undesired output from accidental misconfiguration.
Previously only ETHERTYPE_PAE was permitted.

BPF writes to a lagg(4) child port are presumably intentional, so just
allow them, while still blocking other packets that should take the
aggregation path.

PR: kern/138620

MFC r231130 (pjd)

Allow to set if_bridge(4) sysctls from /boot/loader.conf.

MFC r235147

Do not reinitialise the interface if it is already running, this prevents the
bootp+nfs code from working as it calls init on each dhcp send and rx fails to
start in time.

MFC r235144

The DEVICE_POLLING dereference of sc->tsec_ifp needs to be checked for null
first or this will panic. Condense three blocks that check sc->tsec_ifp into
one while I am here.

MFC r234488

Move the interface media check to a taskqueue, some interfaces (usb) sleep
during SIOCGIFMEDIA and we were holding locks.

MFC r234487

Add linkstate to bridge(4), set the link to up when at least one underlying
interface is up, otherwise the link is down.

This, among other things, allows carp to work on a bridge.

MFC r234163

Set the proto to LAGG_PROTO_NONE before calling the detach routine so packets
are discarded, this is an issue because lacp drops the lock which may allow
network threads to access freed memory. Expand the lock coverage so the
detach/attach happen atomically.

Submitted by: Andrew Boyer (earlier version)

MFC r232629,r232640

Add the ability to set which packet layers are used for the load balance hash
calculation.

MFC r232118

Only look for a usable MAC address for the bridge ID from ports within our
bridge, this allows us to have more than one independent bridge in the same
STP domain.

PR: kern/164369
Submitted by: Nikos Vassiliadis (earlier version)

MFC r232014,r232030,r232070

- bstp_input() always consumes the packet so remove the mbuf handling dance
around it.
- Now that network interfaces advertise if they support linkstate notifications
we do not need to perform a media ioctl every 15 seconds.
- Indicate this function decrements the timer as well as testing for expiry.

MFC: r235681

Rewrite nd6_sysctl_{d,p}rlist() to avoid misaligned accesses to char arrays
casted to structs by getting rid of these buffers entirely. In r169832, it
was tried to paper over this issue by 32-bit aligning the buffers. Depending
on compiler optimizations that still was insufficient for 64-bit architectures
with strong alignment requirements though.
While at it, add comments regarding the total lack of locking in this area.

Tested by: bz
Reviewed by: bz (slightly earlier version), yongari (earlier version)

MFC: r234897

Add a command for showing the heap usage.

PR: 165025
Submitted by: Gavin Mu

MFC: r234789

Add multiple inclusion protection.

PR: 165025
Submitted by: Gavin Mu

MFC: r234348

Turn on PREEMPTION by default. After fixing several bugs over time, the
last show-stopper keeping PREEMPTION from being usable on sparc64 should
have been dealt with in r230662 (MFC'ed to stable/9 in r230662).
At least on 2-way systems, PREEMPTION causes a little bit of a degradation
in worldstone performance. However, FreeBSD seems to have started building
up regressions in !PREEMPTION cases so sparc64 better should not be an
oddball in this regard.

MFC: r235568
A problem with the NFSv4 server was reported by Andrew Leonard
to freebsd-fs@, where the setfacl of an NFSv4 acl would fail.
This was caused by the VOP_ACLCHECK() call for ZFS replying
EOPNOTSUPP. After discussion with rwatson@, it was determined
that a call to VOP_ACLCHECK() before doing VOP_SETACL() is not
required. This patch fixes the problem by deleting the
VOP_ACLCHECK() call.

MFC r235151:
Implement basic remote PHY support. Remote PHY allows the
controller to perform MDIO type accesses to a remote transceiver
using message pages defined through MRBE(multirate backplane
ethernet). It's used in blade systems(e.g Dell Blade m610) which
are connected to pass-through blades rather than traditional
switches.
This change directly manipulates firmware's mailboxes to control
remote PHY such that it does not use mii(4). Alternatively, as
David said, it could be implemented in brgphy(4) by creating a fake
PHY and let brgphy(4) do necessary mii accesses and bce(4) can
implement mailbox accesses based on the type of brgphy(4)'s mii
accesses. Personally, I think it would make brgphy(4) hard to
maintain since it would have to access many bce(4) registers in
brgphy(4). Given that there are users who are suffering from lack
of remote PHY support, it would be better to get working system
rather than waiting for complete/perfect implementation.

MFC 234098:
Add media types for 40G media that might be used with FreeBSD.

Repair mergeinfo from merges done into a sparse checkout.

MFC 234190,234196,234280:
- Extend the KDB interface to add a per-debugger callback to print a
backtrace for an arbitrary thread (rather than the calling thread).
A kdb_backtrace_thread() wrapper function uses the configured debugger
if possible, otherwise it falls back to using stack(9) if that is
available.
- Replace a direct call to db_trace_thread() in propagate_priority()
with a call to kdb_backtrace_thread() instead.

Move mergeinfo from sys/kern/subr_witness.c up to sys/.

MFC of 234386, 234400, 234441, 234443, 234482, 234483, 235052, 235241,
235246, and 235619

MFC: 234386

Replace the MNT_VNODE_FOREACH interface with MNT_VNODE_FOREACH_ALL.
The primary changes are that the user of the interface no longer
needs to manage the mount-mutex locking and that the vnode that
is returned has its mutex locked (thus avoiding the need to check
to see if its is DOOMED or other possible end of life senarios).

To minimize compatibility issues for third-party developers, the
old MNT_VNODE_FOREACH interface will remain available so that this
change can be MFC'ed to 9. Following the MFC to 9, MNT_VNODE_FOREACH
will be removed in head.

The reason for this update is to prepare for the addition of the
MNT_VNODE_FOREACH_ACTIVE interface that will loop over just the
active vnodes associated with a mount point (typically less than
1% of the vnodes associated with the mount point).

Reviewed by: kib
Tested by: Peter Holm
MFC after: 2 weeks

MFC: 234400

Drop export of vdestroy() function from kern/vfs_subr.c as it is
used only as a helper function in that file. Replace sole call to
vbusy() with inline code in vholdl(). Replace sole calls to vfree()
and vdestroy() with inline code in vdropl().

The Clang compiler already inlines these functions, so they do not
show up in a kernel backtrace which is confusing. Also you cannot
set their frame in kgdb which means that it is impossible to view
their local variables. So, while the produced code is unchanged,
the debugging should be easier.

Discussed with: kib
MFC after: 2 weeks

MFC: 234441

Fix a memory leak of M_VNODE_MARKER introduced in 234386.

Found by: Peter Holm

MFC: 234443

Delete a no longer useful VNASSERT missed during changes in 234400.

Suggested by: kib

MFC: 234482

This change creates a new list of active vnodes associated with
a mount point. Active vnodes are those with a non-zero use or hold
count, e.g., those vnodes that are not on the free list. Note that
this list is in addition to the list of all the vnodes associated
with a mount point.

To avoid adding another set of linkage pointers to the vnode
structure, the active list uses the existing linkage pointers
used by the free list (previously named v_freelist, now renamed
v_actfreelist).

This update adds the MNT_VNODE_FOREACH_ACTIVE interface that loops
over just the active vnodes associated with a mount point (typically
less than 1% of the vnodes associated with the mount point).

Reviewed by: kib
Tested by: Peter Holm
MFC after: 2 weeks

MFC: 234483

This update uses the MNT_VNODE_FOREACH_ACTIVE interface that loops
over just the active vnodes associated with a mount point to replace
MNT_VNODE_FOREACH_ALL in the vfs_msync, ffs_sync_lazy, and qsync
routines.

The vfs_msync routine is run every 30 seconds for every writably
mounted filesystem. It ensures that any files mmap'ed from the
filesystem with modified pages have those pages queued to be
written back to the file from which they are mapped.

The ffs_lazy_sync and qsync routines are run every 30 seconds for
every writably mounted UFS/FFS filesystem. The ffs_lazy_sync routine
ensures that any files that have been accessed in the previous
30 seconds have had their access times queued for updating in the
filesystem. The qsync routine ensures that any files with modified
quotas have those quotas queued to be written back to their
associated quota file.

In a system configured with 250,000 vnodes, less than 1000 are
typically active at any point in time. Prior to this change all
250,000 vnodes would be locked and inspected twice every minute
by the syncer. For UFS/FFS filesystems they would be locked and
inspected six times every minute (twice by each of these three
routines since each of these routines does its own pass over the
vnodes associated with a mount point). With this change the syncer
now locks and inspects only the tiny set of vnodes that are active.

Reviewed by: kib
Tested by: Peter Holm
MFC after: 2 weeks

MFC: 235052 (by pluknet)

Fix mount mutex handling missed in r234386.

MFC: 235241 (by pluknet)

Fix mount interlock oversights from the previous change in r234386.

Reported by: dougb
Submitted by: Mateusz Guzik <mjguzik at gmail com>
Reviewed by: Kirk McKusick
Tested by: pho

MFC: 235246

Fix mount mutex handling missed in r234386.

MFC: 235619

Update comment to document that the vnode free-list mutex needs to be
held when updating mnt_activevnodelist and mnt_activevnodelistsize.

MFC 234186
If a linker file contains at least one module, but all of the modules
fail to load (the MOD_LOAD event fails) during a kldload(2), unload the
linker file and fail the kldload(2) with ENOEXEC.

MFC 234182:
Don't update if_obytes when transmitting packets. That is already done
in IFQ_HANDOFF() when the packet is passed to the start routine, so doing
it here resulted in double counting.

MFC 235563:
Don't expose i386-only ptrace constants on amd64. This broke gdb with
libthread_db on amd64.

MFC of the e1000 drivers: revisions include
227309,228281,228386,228387,228393,228405,
228415,228788,228803,229606,229767,229939,
230023,230024,230742,231796,232238,233708,
234154,234665,235256

MFC 235210

Modify the binding of queues to attach to as many CPUs
as possible when using more than one igb(4) adapter. This
means that queues will not be bound to the same CPUs if
there are more CPUs availble.

This is only applicable to a system that has multiple interfaces.

MFC: r233423

Initialize the mutexes used for the NVM and the swflag as MTX_DUPOK in
order to avoid otherwise harmless witness warnings when these are acquired
at the same time and due to both using MTX_NETWORK_LOCK as their type.
The right fix actually would be to use different, descriptive types for
these. However, the latter would require undesirable changes to the shared
code base. Another approach would be to just supply NULL as the type, which
was deemed as less desirable though as it would cause the unique but cryptic
name also to be used for the type and to diverge from the type used by other
network device drivers.

MFC 232367:
Properly handle failures in igb_setup_msix() by returning 0 if MSI or MSI-X
allocation fails.

MFC r227458, r226436:

- change "is is" to "is" or "it is"
- change "the the" to "the"
- other typo fixes

Approved by: lstewart

MFC r228441:

Consistently use types in e1000 driver code:

- Two struct members eee_disable are used in a function that expects
an int *, so declare them int, not bool.
- igb_tx_ctx_setup() returns a boolean value, so declare it bool, not int.
- igb_header_split is passed to TUNABLE_INT, so delcare it int, not bool.
- igb_tso_setup() returns a bool, so declare it bool, not boolean_t.
- Do not re-define bool/true/false if the symbols already exist.

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

Clear transmit checksum offload context state upon lem(4) interface
initialization. Prior to this change packets may be transmitted with an
incorrect checksum.

Em(4) already has an equivalent change in r213234.

Obtained From: Sandvine
MFC After: 1 week
Approved by: re (bz)

A fix to make the LINT-NOINET build happy, if this
works out the ixgbe driver should be changed as well.

- Add read-only sysctls for all of the tunables supported by the igb and
em drivers.
- Make the per-instance 'enable_aim' sysctl truly per-instance by having it
change a per-instance variable (which is used to control AIM) rather
than having all of the per-instance sysctls operate on a single global
variable.

Reviewed by: jfv (earlier version)
MFC after: 1 week

Put back the global for rx processing due to popular demand.

Eliminate some global tuneables in favor of adapter-specific,
particular flow control and dma coalesce. Also improve the
sysctl operation on those too.

Add IPv6 detection in the ioctl code, this was done for
ixgbe first, carrying that over.

Add resource ability to disable particular adapter.

Add HW TSO capability so vlans can make use of TSO

- Use a dedicated task to handle deferred transmits from the if_transmit
method instead of reusing the existing per-queue interrupt task.
Reusing the per-queue interrupt task could result in both an interrupt
thread and the taskqueue thread trying to handle received packets on a
single queue resulting in out-of-order packet processing.
- Don't define igb_start() at all on 8.0 and where if_transmit is used.
Replace last remaining call to igb_start() with a loop to kick off
transmit on each queue instead.
- Call ether_ifdetach() earlier in igb_detach().
- Drain tasks and free taskqueues during igb_detach().

Reviewed by: jfv
MFC after: 1 week

Add an initialization to the error variable, without
this there is a rare return path that bogusly appears
to fail when it should not. Also white space correction.

Thanks to Arnaud Lacombe for noticing the problem.

Small change to make backporting to stable/7,
thanks to Arnaud Lacombe for suggesting it.

Important update for the igb driver:
- Add the change made in em to the actual unrefreshed number
of descriptors is used as a basis in rxeof on the way out
to determine if more refresh is needed. NOTE: there is a
difference in the ring setup in igb, this is not accidental,
it is necessitated by hardware behavior, when you reset the
newer adapters it will not let you write RDH, it ALWAYS sets
it to 0. Thus the way em does it is not possible.
- Change the sysctl handling of flow control, it will now make
the change dynamically when the variable setting changes rather
than requiring a reset.
- Change the eee sysctl naming, validation found the old unintuitive :)
- Last but not least, some important performance tweaks in the TX
path, I found that UDP behavior could be drastically hindered or
improved with just small changes in the start loop. What I have
here is what testing has shown to be the best overall. Its interesting
to note that changing the clean threshold to start at a full half of
the ring, made a BIG difference in performance. I hope that this
will prove to be advantageous for most workloads.

MFC in a week.

Fix to an error condition case, when an mbuf chain
get's defragged due to a mapping failure the header
pointers will be invalidated and can result in a
TSO or other failure down the line. So, when the
remapping occurs force a retry thru the offload
calculation code. Thanks to Andrew Boyer for discovering
this and cooking up the fix!!

Change the refresh_mbuf logic slightly, add an inline
to calculate the outstanding descriptors that need to be
refreshed at any time, and use THAT in rxeof to determine
if refreshing needs to be done. Also change the local_timer
to simply fire off the appropriate interrupt rather than
schedule a tasklet, its simpler.

MFC in two weeks

Do a sweep of the tree replacing calls to pci_find_extcap() with calls to
pci_find_cap() instead.

A cut and paste here was wrong also.

Correct broken define

This delta updates the em driver to version 7.2.2 which has
been undergoing test for some weeks. This improves the RX
mbuf handling to avoid system hang due to depletion. Thanks
to all those who have been testing the code, and to Beezar
Liu for the design changes.

Next the igb driver is updated for similar RX changes, but
also to add new features support for our upcoming i350 family
of adapters.

MFC after a week

Fix typos - remove duplicate "the".

PR: bin/154928
Submitted by: Eitan Adler <lists at eitanadler.com>
MFC after: 3 days

Fix the shared code to be consistent with Intel-internal,
and now build.

Restore 2 prototypes that seem to have been mistakenly removed in r218582.

I've manually twiddled the whitespace for e1000_commit_fc_settings_generic
to match the others in the file.

Submitted by: dim
Tested by: me

Somehow the RX ring depletion fix got partially removed,
replace the missing pieces.

Revert changes made here, they will cause a conflict
later on with our shared code.

Inconsistencies in the updated igb shared code and the older
em/lem, breaking the build, correcting that.

After r218530 export several functions which are no longer private to
e1000_mac.c but part of the e1000_api.

X-MFC with: 218530 by jfv

Add support for the new I350 family of 1G interfaces.
- this also includes virtualization support on these devices

Correct some vlan issues we were seeing in test, jumbo frames on vlans
did not work correctly, this was all due to confused logic around HW
filters, the new code should now work for all uses.

Important fix: when mbuf resources are depeleted, it was possible to
completely empty the RX ring, and then the RX engine would stall
forever. This is fixed by a flag being set whenever the refresh code
fails due to an mbuf shortage, also the local timer now makes sure
that all queues get an interrupt when it runs, the interrupt code
will then always call rxeof, and in that routine the first thing done
is now to check the refresh flag and call refresh_mbufs. This has been
verified to fix this type 'hang'. Similar code will follow in the other
drivers.

Finally, sync up shared code for the I350 support.

Thanks to everyone that has been reporting issues, and helping in the
debug/test process!!

Fix for kern/152853, pullup at the wrong point
is breaking UDP. Thanks to Petr Lampa for the
patch.

Specify a CTLTYPE_FOO so that a future sysctl(8) change does not need
to rely on the format string.

sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly.

Commit the Intel drivers.

A couple problems discovered by Andrew Boyer:
- failure code in em_xmit got mangled along the way
and was not properly handling errors.
- local timer code had a leftover UNLOCK call that
should be removed.

MFC after 3 days

Correct build error.

Remove the bogus test in the TX context setup for IPV6,
the size can be smaller than the constant when you are
doing HW TAGGING, and you still need to process this
packet in a normal way. I'm not sure where the notion
to just return came from, but its wrong.

MFC after: 3 days

Small cut and paste bug in flow control string fixed.
Second, correct the discard/refresh_mbufs code to behave
more like igb, there have been panics due to discards and
this should fix them.

MFC after: 3 days

Unwanted extra call to set_vlan_hw added back by mistake.

The purpose of this change is to add a routine to
disable ASPM L0S and L1 LINK states on 82573, 82574,
and 82583. The theory is that this is behind certain
hangs being experienced by some customers.

Also included a small optimization in the rxeof routine
that was in my internal code.

Change the PBA size for pchlan, it was incorrect.

MFC after: 3 days

Add shared code glue for new 82580 devices.

- New 82580 devices supported
- Fixes from John Baldwin: vlan shadow tables made per/interface,
make vlan hw setup only happen when capability enabled, and
finally, make a tuneable interrupt rate. Thanks John!
- Tweaked watchdog handling to avoid any false positives, now
detection is in the TX clean path, with only the final check
and init happening in the local timer.
- limit queues to 8 for all devices, with 82576 or 82580 on
larger machines it can get greater than this, and it seems
mostly a resource waste to do so. Even 8 might be high but
it can be manually reduced.
- use 2k, 4k and now 9k clusters based on the MTU size.
- rework the igb_refresh_mbuf() code, its important to
make sure the descriptor is rewritten even when reusing
mbufs since writeback clobbers things.

MFC: in a few days, this delta needs to get to 8.2

Sync the lem code up with the vlan and other fixes in em.
Delete a unneeded test from the beginning of em_xmit.
CRITICAL: shared code fix for 82574, a mutex might not be
released, this can cause hangs.

In the data setup code for doing offloads the
ip and tcp pointers were not reset after some
pullups. In practice this led to an NFS mount
failure when using UDP reported by Kevin Lo,
thanks Kevin. Fix from yongari, thank you!

Bug fix delta to the em driver:
- Chasin down bogus watchdogs has led to an improved
design to this handling, the hang decision takes
place in the tx cleanup, with only a simple report
check in local_timer. Our tests have shown no false
watchdogs with this code.
- VLAN fixes from jhb, the shadow vfta should be per
interface, but as global it was not. Thanks John.
- Bug fixes in the support for new PCH2 hardware.
- Thanks for all the help and feedback on the driver,
changes to lem with be coming shortly as well.

Update code from Intel:
- Sync shared code with Intel internal
- New client chipset support added
- em driver - fixes to 82574, limit queues to 1 but use MSIX
- em driver - large changes in TX checksum offload and tso
code, thanks to yongari.
- some small changes for watchdog issues.
- igb driver - local timer watchdog code was missing locking
this and a couple other watchdog related fixes.
- bug in rx discard found by Andrew Boyer, check for null pointer

MFC: a week

Tweak the stats exported by the e1000 drivers:
- Add a single sysctl procedure to all three drivers to read an arbitrary
register (the register is passed as arg2). Use it to replace existing
routines in igb(4) that used a separate routine for each register, and
to add support for missing stats in em(4) and lem(4).
- Move the 'rx_overruns' and 'watchdog_timeouts' stats out of the MAC stats
section as they are driver stats, not MAC counters.
- Simplify the code that creates per-queue stats in igb(4) to use a single
loop and remove duplicated code.
- Properly read all 64 bits of the 'good octets received/transmitted' in
em(4) and lem(4).
- Actually read the interrupt count registers in em(4), and drop the
'host to card' sysctl stats from em(4) as they are not implemented in
any of the hardware this driver supports.
- Restore several stats to em(4) that were lost in the earlier stats
conversion including per-queue stats.
- Export several MAC stats in em(4) that were exported in igb(4) but not
in em(4).
- Export stats in lem(4) using individual sysctls as in em(4) and igb(4).

Reviewed by: jfv
MFC after: 1 week

Code correction in refresh_mbufs, just continuing
without index recalc was wrong.

Tighten up the rx mbuf refresh code, there were some
discrepencies from the igb version which was the target.

Change the message when neither MSI or MSIX are enabled
and a fallback to Legacy interrupts happen, the existing
message was confusing.

Do not allocate multicast array memory in multicast filter
configuration function. For failed memory allocations, em(4)/lem(4)
called panic(9) which is not acceptable on production box.
igb(4)/ixgb(4)/ix(4) allocated the required memory in stack which
consumed 768 bytes of stack memory which looks too big.

To address these issues, allocate multicast array memory in device
attach time and make multicast configuration success under any
conditions. This change also removes the excessive use of memory in
stack.

Reviewed by: jfv

If em(4) failed to allocate RX buffers, do not call panic(9).
Just showing some buffer allocation error is more appropriate
action for drivers. This should fix occasional panic reported on
em(4) when driver encountered resource shortage.

Reviewed by: jfv

Do not call voluntary panic(9) in case of if_alloc() failure.

Reviewed by: jfv

Make sure not to access unallocated stats memory.

Reviewed by: jfv

Eliminate the ambiguous queue setting logic for
the VF, it made it possible to have 2 queues which
we don't want, the HOST is unable to handle it.

Put the early setting of the MAC type back, its
removal resulted in broken code in MSIX setup.

Fix clang warning on empty statement.

Reviewed by: rdivacky, zml
Approved by: zml (mentor)

style(9) fix

MFC after: 1 week

Fix a bug in the statistics code for tracking the head and
tail pointers of the tx and rx queues. We needed a SYSCTL_PROC
to correctly get the values at run time.

Submitted by: Andrew Boyer aboyer at averesystems.com
MFC after: 1 week

Fix for a panic when TX checksum offload is done and
a packet has only a header in the first mbuf, the
checksum code will dereference a pointer into the
non-existing IP header. Do a check for the size and
pullup if needed. Thanks to Michael Tuexen for this
fix.

MFC: asap - should be in 8.1 IMHO

Fix of a VLAN problem by jhb, the checksum capability
got lost along the way.

MFC: asap

OK, I was a bit sleep this morning and checked in
the core changes but left out the shared code, lol.
Well, and a couple fixes to the core... hopefully
this will all be complete now.

Happy happy joy joy :)

SR-IOV support added to igb

What this provides is support for the 'virtual function'
interface that a FreeBSD VM may be assigned from a host
like KVM on Linux, or newer versions of Xen with such
support.

When the guest is set up with the capability, a special
limited function 82576 PCI device is present in its virtual
PCI space, so with this driver installed in the guest that
device will be detected and function nearly like the bare
metal, as it were.

The interface is only allowed a single queue in this configuration
however initial performance tests have looked very good.

Enjoy!!

Make sure that all the exposed counters and variables are actually
being updated.

Pointed out by: jfv

Two stats were duplicated, thanks to Andrew Boyer
for pointing this out.

Move statistics into the sysctl tree making it easier to find
and use them.
Add previously hidden statistics, some of which include interrupt
and host/card communication counters.

Move statistics into the sysctl tree making it easier to find
and use them.
Add previously hidden statistics, some of which include interrupt
and host/card communication counters.

Changes from John Baldwin adding to last commit,
change rxeof api for poll friendliness, and
eliminate unnecessary link tasklet use. Thanks John!

Change to have legacy interrupts use the same
handler had a flaw, thanks to John Baldwin for
finding it. Change which queue legacy tasks are
enqueued on.

MFC: soonest

Put back the lost bus_describe_intr() calls.

Add a couple fixes from Michael Tuexen.
Remove unneeded rxtx handler, make que handler generic.
Do not allocate header mbufs in rx ring if not doing hdr split.
Release the lock in rxeof call to stack.

MFC for 8.1 asap

Restore part of 200671 which was lost in previous driver changes:
- Add interrupt descriptions when using mulitple MSI-X interrupts.

Fix a mismerge in r206001.

PR: 146614
Approved by: jfv (implicit)
MFC afer: 3 days

Small changes preparing for MFC, need to conditionalize
the buf_ring_free call, and lem is missing the WOL change
put into em.

Address the LOD that some are seeing, put the RX lock
back in rxeof (I could see little point in taking it out),
and now release it before the stack entry.

Also, make it so the 82574 does not configure for multiqueue
when its not used in the stack.

Change default WOL back to MAGIC only, having
multicast enabled causes problems in man environments.

Add a missing fragment in the tx msix handler to invoke
another if all work is not done.

Sync the igb driver with changes suggested by yongari and
made in em, these made sense to be in both drivers.

Remove multiqueue stack related stuff form lem, it is
unneeded for legacy hardware.
Also remove some TSO related cruft.
Add some watchdog_time setting that was missing, thanks
to Mikolaj Golub for pointing that out.

The lock move in rxeof necessitated a couple
more places to do the locking, fixes a panic.

Correct broken build.

A few more changes from yongari:
- code flow in handler could let interrupt be
reenabled when not wanted.
- change where the RX lock is taken to improve
performance.
- adapter->msix is true for MSI systems also,
it needs to explicitly test for 82574, good one :)

DUH, must be tired, I missed the second instance...
time for the weekend :)

Thanks to Michael Tuexen for catching this, bit set that
keeps the clock from being reset when writing to EITR was
incorrect, also there is a shared code #define for it anyway.

Incorporate suggested improvements from yongari.

Also, from feedback, make the multiqueue code an
option (EM_MULTIQUEUE) that is off by default.
Problems have been seen with UDP when its on.

Three changes:
- add CRC stripping to the RX side, this was handled
by some obscure code in rxeof previously, its easier
to simply have the hardware strip it now.
- Add back an ALTQ change that slipped between the cracks
- Add an update to the watchdog_time in the xmit code, not
doing this in ixgbe caused problems, think its needed here
as well.

Important fix got clobbered in the em driver, keeping
VLAN HWFILTER from being used by default, this breaks
stacked pseudo devices, and as it turns out, also breaks
virtual machines that happen to use VLANS (didn't know that
before :). Put the fix back into the em driver, and for good
measure add the same code to the igb driver where it should
have been anyway.

The POLL code was missed in the queue conversion,
change the argument type to igb_rxeof() to the
correct type. Note, any users of POLLING must
be sure and set the number of queues to 1 for
things to work correctly.

Hook the identification LEDs of igb(4), lem(4) and em(4) devices up with
led(4) so they can be lit or f.e. made blink via `echo f2 > /dev/led/em0`
for localization purposes.

Approved by: jfv
MFC afer: 1 week (after r205869)

Fix poll handler declaration.

Fix lint build problem.

Update to igb and em:

em revision 7.0.0:
- Using driver devclass, seperate legacy (pre-pcie) code
into a seperate source file. This will at least help
protect against regression issues. It compiles along
with em, and is transparent to end use, devices in each
appear to be 'emX'. When using em in a modular form this
also allows the legacy stuff to be defined out.
- Add tx and rx rings as in igb, in the 82574 this becomes
actual multiqueue for the first time (2 queues) while in
other PCIE adapters its just make code cleaner.
- Add RX mbuf handling logic that matches igb, this will
eliminate packet drops due to temporary mbuf shortage.

igb revision 1.9.3:
- Following the ixgbe code, use a new approach in what
was called 'get_buf', the routine now has been made
independent of rxeof, it now does the update to the
engine TDT register, this design allows temporary
mbuf resources to become non-critical, not requiring
a packet to be discarded, instead it just returns and
does not increment the tail pointer.
- With the above change it was also unnecessary to keep
'spare' maps around, since we do not have the discard
issue.
- Performance tweaks and improvements to the code also.

MFC in a week

Fix drbr and altq interaction:
- introduce drbr_needs_enqueue that returns whether the interface/br needs
an enqueue operation: returns true if altq is enabled or there are
already packets in the ring (as we need to maintain packet order)
- update all drbr consumers
- fix drbr_flush
- avoid using the driver queue (IFQ_DRV_*) in the altq case as the
multiqueue consumer does not provide enough protection, serialize altq
interaction with the main queue lock
- make drbr_dequeue_cond work with altq

Discussed with: kmacy, yongari, jfv
MFC after: 4 weeks

A few minor changes: add altq option header, add missing conditional
around a buf_ring call that will break 7.3, and thanks to Fabien Thomas
add POLLING support for igb and a minor related fix in the em driver.

Fix for kern/141646: when stacking pseudo drivers like
lagg and vlan the vlan attach/detach event is not being
handed down to em, this caused some init code not to run,
and thus VLANs did not work. Ultimately having the event
get propagated would be nice, but for now the solution is
to have HWFILTER off by default, when this is the case
VLANs will work, ifconfig can be used to turn it on and
then get HW tag filtering.

Add a link tasklet so updates can be sleepable.

Two more build problems, missing includes and semicolon.

Opps, completely wrong version of if_em.h got into
the checkin, sorry all :(

Missing a fix for the new watchdog handling.

Remove some internal conditional defines that will
fail in kernel tree.

Update the 1G drivers, shared code sync with Intel,
igb now has a queue notion that has a single interrupt
with an RX/TX pair, this will reduce the total interrupts
seen on a system. Both em and igb have a new watchdog
method. igb has fixes from Pyun Yong-Hyeon that have
improved stability, thank you :)

I wish to MFC this for 7.3 asap, please test if able.

Remove extraneous semicolons, no functional changes.

Submitted by: Marc Balmer <marc@msys.ch>
MFC after: 1 week

- Add missing newlines to some error messages.
- Add interrupt descriptions when using mulitple MSI-X interrupts.

Reviewed by: jfv

Add old read_mac_addr routine to this module since
a customer report of an Invalid MAC Address has occurred.

Remove the MTX_SPIN flag to the shared code MUTEX
as it was causing a panic, also took the opportunity
to rename the lock for clarity.

Remove phantom line of code that somehow slipped
into the checkin.

Resync with Intel versions of both the em and igb
drivers. These add new hardware support, most importantly
the pch (i5 chipset) in the em driver. Also, both drivers
now have the simplified (and I hope improved) watchdog
code. The igb driver uses the new RX cleanup that I
first implemented in ixgbe.

em - version 6.9.24
igb - version 1.8.4

With an i386 kernel the igb driver can cause a
page fault panic on initialization due to a large
number of bounce pages being allocated. This is due
to the dma tag requiring page alignment on mbuf mapping.
This was removed some time back from the ixgbe driver
and is not needed here either.

Fix build issue with last commit.

Fix build complaint from previous checkin

Fix an xmit mbuf leak, when transmit failed but you
still have an mbuf it was not being requeued.

MFC: 3 days

Fix for pr 138516
An mbuf is not requeued when a xmit fails.

MFC: 3 days

Revert previous commit and add myself to the list of people who should
know better than to commit with a cat in the area.

Add necessary include.

Temporarily enhance em(4) and igb(4) hack to take account for IFF_NOARP.
Without this changeset there will be no way to prevent these NICs from
sending ARP, which is harmful in server farms that is configured as
"Direct Server Return" behind a load balancer.

A better fix would remove the whole hack completely but it would be
later than 8.0-RELEASE.

Reviewed by: jfv, yongari
Approved by: re (kib)

Improvement on the last change, this gives a precise
way to tell the one and only interface that a vlan
event is for. Thanks to John Baldwin for the patch.

Approved by: re

This delta fixes two bugs:
- When a vlan event occurs a check was not made that
the event was actually for the interface, thus resulting
in a panic. All three drivers have this vulnerability. Add
a check for this condition.
- Secondly, there was a duplicate buf_ring free in the em
driver resulting in a panic on unload. Remove.

Approved by: re

A small number of systems in the ICH9/10 family have a flash
part that is made up of 8K banks rather than 4K, if these
systems are using bank 1 then the last change in this code
breaks the bank read, resulting in an invalid checksum of
the eeprom during driver load. This change fixes this.

Approved by: re

Re-add opt_inet.h, as we did in r193862 and lost yet again.

Approved by: re (kib)

The new method of reading the mac address from the
RAR(0) register does not work on this old adapter,
provide a local routine that does it the older way.

Approved by: re

Type problem when FreeBSD is in a virtualized environment, the
result was when the RX index wrapped it was converted into some
sort of gibberish and written into the RDT register, effectively
killing the RX side of the thing :)

Approved by: re

Use if_maddr_rlock()/if_maddr_runlock() rather than IF_ADDR_LOCK()/
IF_ADDR_UNLOCK() across network device drivers when accessing the
per-interface multicast address list, if_multiaddrs. This will
allow us to change the locking strategy without affecting our driver
programming interface or binary interface.

For two wireless drivers, remove unnecessary locking, since they
don't actually access the multicast address list.

Approved by: re (kib)
MFC after: 6 weeks

Change intr_bind to bus_bind_intr, also limit this to
multiqueue setup which is not the shipping default for
igb (its set to 1).

need to make intr_bind call architecture specific for
global builds (failing sun4v lint build)

Fix lint issue.

Updates for both the em and igb drivers, add support
for multiqueue tx, shared code updates, new device
support, and some bug fixes.

Add opt_inet.h back lost with r190872.
This will bring back improved IPv4 SIOCSIFADDR ioctl handling
not re-initializing the interface if avoidable.

When user_frac in the polling subsystem is low it is going to busy the
CPU for too long period than necessary. Additively, interfaces are kept
polled (in the tick) even if no more packets are available.
In order to avoid such situations a new generic mechanism can be
implemented in proactive way, keeping track of the time spent on any
packet and fragmenting the time for any tick, stopping the processing
as soon as possible.

In order to implement such mechanism, the polling handler needs to
change, returning the number of packets processed.
While the intended logic is not part of this patch, the polling KPI is
broken by this commit, adding an int return value and the new flag
IFCAP_POLLING_NOCOUNT (which will signal that the return value is
meaningless for the installed handler and checking should be skipped).

Bump __FreeBSD_version in order to signal such situation.

Reviewed by: emaste
Sponsored by: Sandvine Incorporated

Call drbr_stats_update to update ifp stats directly when we bypass the buf_ring on transmit

fix typo in conditional

collapse the two em_start_locked routines in to one

Correct fat finger mistake

igb_txeof also has a case where the watchdog may not
get reset when it should be

MFC after: 2 weeks

Thanks for Michael Tuexen for tracking down a path where
the watchdog timer was not being rearmed in txeof, and also
a missing case in the new code.

MFC after: 2 weeks

fix typo

fix panic when using msix

Pointed out by Nate Whitehorn

Make sure the ALTQ case is handle correctly by using drbr_dequeue

call base if_qflush routine to flush if_snd

Thanks to Michael Tuexen and Randall Scott for providing a
few important bug fixes found while they were doing SCTP
development, and that I somehow lost during the scramble.

Thanks guys!!

- define em_transmit and em_qflush
- make buF_ring usage conditional but enabled by default

Reviewed by: jfv

Fix build problem with data format.

This delta syncs the em and igb drivers with Intel,
adds header split and SCTP support into the igb driver.
Various small improvements and fixes.

MFC after: 2 weeks

Fix a cut/paste bug which prevents us from setting the average
latency tunable.

Reviewed by: jfv
MFC after: 1 day

Restore opt_inet.h include which was lost in the last commit.

Thanks to the reminder from Ganbold, small fix in the RX failure
path for an infinite loop. Problem originally noticed in ixgbe
by Jeff Roberson and fixed there. Thanks to everyone involved.

This delta is primarily a fix for es2lan devices that
will sometimes fail to initialize problem due to a lock
contention with management hardware. However, in order to
deliver that fix it was necessary to take a shared code
update as a whole, and this required scattered changes in
the core code to be compatible.

The em driver now has VLAN HW support added as the igb
driver had previously.

MFC after: ASAP - in time for 7.1 RELEASE

Hide AF_INET specific ioctl handling under #ifdef INET.

MFC after: 2 months

Hide AF_INET specific ioctl handling under #ifdef INET.

MFC after: 2 months

Clean out some empty mergeinfo records, presumably by people doing local
cp/mv operations. The full repo-relative URL should be specified for the
source in these cases.

Update to igb driver:

- changes in support of the VLAN filter fix to 126850
- removal of a bunch of legacy code that was cruft, if not
possibly harmful.
- removal of POLLING from this driver, with multiqueue and
MSIX it just makes no sense here.
- Fix an LRO bug that I've been working on internally, intermittent
panics under stress, the problem was releasing the RX ring lock
before the LRO flushing.
- Following the above fix I now enable LRO by default
- For performance reasons increase the default number of RX queues
to 4.
- Add AIM - "Adaptive Interrupt Moderation", a fancy way of saying
that the EITR value is dynamically changed based on the size of
packets in the last interrupt interval.

- Much goodness to try, enjoy!!

Data type fix

Include netinet/tcp_lro.h, unbreak the build

Merge of the source for igb and em into dev/e1000, this
proved to be necessary to make the static drivers work
in EITHER/OR or BOTH configurations. Modules will still
build in sys/modules/igb or em as before.

This also updates the igb driver for support for the 82576
adapter, adds shared code fixes, and etc....

MFC after: ASAP

Thanks to report from Neil Hoggarth I found a missing UNLOCK in
the watchdog code. This delta also incorporates some missing PCI
IDs that got added.

PR 122928 - might be fixed by this, no verification at this point.

This small change will allow this driver in HEAD to build
on 6.3 as well as 7 :)

This delta has a few important items:

PR 122839 is fixed in both em and in igb

Second, the issue on building modules since the static kernel
build changes is now resolved. I was not able to get the fancier
directory hierarchy working, but this works, both em and igb
build as modules now.

Third, there is now support in em for two new NICs, Hartwell
(or 82574) is a low cost PCIE dual port adapter that has MSIX,
for this release it uses 3 vectors only, RX, TX, and LINK. In
the next release I will add a second TX and RX queue. Also, there
is support here for ICH10, the followon to ICH9. Both of these are
early releases, general availability will follow soon.

Fourth: On Hartwell and ICH10 we now have IEEE 1588 PTP support,
I have implemented this in a provisional way so that early adopters
may try and comment on the functionality. The IOCTL structure may
change. This feature is off by default, you need to edit the Makefile
and add the EM_TIMESYNC define to get the code.

Enjoy all!!

This update primarily addresses the ability to have both the em
and the igb driver static in the kernel. But it also reflects
some other bug fixes in my development stream at Intel.
PR 122373 is also fixed in this code.

Need to add define of FAST interrupts as default

This change introduces a split to the Intel E1000 driver, now rather than
just em, there is an igb driver (this follows behavior with our Linux drivers).
All adapters up to the 82575 are supported in em, and new client/desktop support
will continue to be in that adapter.

The igb driver is for new server NICs like the 82575 and its followons.
Advanced features for virtualization and performance will be in this driver.

Also, both drivers now have shared code that is up to the latest we have
released. Some stylistic changes as well.

Enjoy :)

Add COHERENT to descriptor mem allocation for the
benefit of ARM (request from Olivier Houchard), its
a noop on most architectures and goodness on those
that use it.

A minor issue with lock names, WITNESS complains
when the two locks are the same...

Fix for a reported panic in certain circumstances. When
calling em_stop() now make sure the TX lock is held as
well as CORE.

Take out em_poll() prototype from under EM_FAST_IRQ control.

Reported by: tindebox compiling a LINT kernel

One nit, FAST handling is now in #ifdef's for compatibility
between RELEASES, but we want it on by default in 7 and later,
add that define, and take out a fragment left from a workaround
being removed.

Driver version 6.7.3
- Bring HEAD up to the latest shared code
- Fix TSO problem using limited MSS and forwarding
- Dual lock implementation
- New device support
- For my ease, this code can compile in either 6.x or later
- brings this driver in sync with the 6.3

A number of small fixes:
- duplicate #define in header, thanks to Kevin Lo for pointing out.
- incorrect BUSMASTER enable logic, thanks Patrick Oeschger
- 82543 fails due to bogus IO BAR logic
- Allow 82571 to use MSI interrupts
- Checksum Offload for UDP not working on 82575

Approved by:re

Remove the now-unused NET_{LOCK,UNLOCK,ASSERT}_GIANT() macros, which
previously conditionally acquired Giant based on debug.mpsafenet. As that
has now been removed, they are no longer required. Removing them
significantly simplifies error-handling in the socket layer, eliminated
quite a bit of unwinding of locking in error cases.

While here clean up the now unneeded opt_net.h, which previously was used
for the NET_WITH_GIANT kernel option. Clean up some related gotos for
consistency.

Reviewed by: bz, csjp
Tested by: kris
Approved by: re (kensmith)

Use coherent mapping for DMA on arm. This is propably suitable for the
other archs, but I can't test it so I made it conditionnal on __arm__
for now.

Approved by: re (blanket)

Couple of the fixes needed revising. The ICH8 autoneg was still broken,
this change both simplifies the code and plugs a hole where the devise
was reset without keeping the management controller at bay :) Second,
the 82571 LAA reset problem was incomplete, this addition is necessary.
Just one of those days :)

A few small but significant fixes:
- Coverity Prevent(tm) CID 1906 a bogus use of bzero where unneeded.
- ICH8 systems autoneg to 100 rather than 1000, this can also be
seen in 82573, the logic was backwards.
- On new 82575 quadports half duplex tx speed is slow... this was due
to overwriting TCTL reg rather than adding bits.

Fix for PR 112937, thanks to Ruslan Ermilov. I am still
a bit confused how the 'link flap' was connected to the
'get' rather than 'set' address, but this seems the right
thing to do here.

Two minor fixes, keep old 82542 from using jumbo frames, and add
missing htole64 in encap code.

Reviewed by:Pdeuskar
Approved by:Pdeuskar

Couple of changes, back down on last TSO change, instead make old
adapter list still capable, but only PCI-E adapters are now enabled.
The user can enable older PCI-X or PCI adapters using ifconfig.
Secondly, Arthur Hartwig pointed out my MSI change was not working
correctly, changed to something that now does. Thanks Arthur.
There was also a fundamental bug in the 82575 MSIX code, the MSIX
registers had to be mapped, opps :)

Rubber-stamped by: Pdeuskar

This delta adds two bug fixes: one that makes HW Offload logic in
legacy codepath match the 82575, without this we were seeing bridging
fail on 82546 adapters. Secondly, I have limited TSO to PCI Express
adapters, I meant to do this and it got dropped in the earlier delta.
Next, I am dropping in the latest shared code from our development
team, consensus was that this should be done frequently, so I am :)

Approved by: pdeuskar

Mistake in the logic deciding what adapters need
to map the IO BAR. Causing the driver to fail on
th 82542.

Reviewed by:pdeuskar
Approved by:pdeuskar

A couple bug fixes that I've had internally at Intel. First is a long
time workaround for problems with 82571 adapters and LAAs, one port
getting reset can cause the other to have its RAR[0] also reset,
thus overwriting an LAA. This fix works around it by also keeping
the address in the last array member.

The other bug is specific to the new 575 adapter, its transmit code
logic in handling hwassists was too crude, it broken when doing
bridges. I am much happier with the new logic,we may want to change
the legacy path at some point to something similar.

Reviewed by: pdeuskar
Approved by: pdeuskar

$FreeBSD$ tags are not compilable C code; wrap in either __FBSDID() or
in comments for .c and .h files respectively. Jack may want to clean up
style or other aspects once he's up and about again, but this gets the
kernel compiling.

Merge in the new driver (6.5.0) of Intel. This has a new
shared code infrastructure that is family specific and
modular. There is also support for our latest gigabit
nic, the 82575 that is MSI/X and multiqueue capable.

The new shared code changes some interfaces to the core
code but testing at Intel has been going on for months,
it is fairly stable.

I have attempted to be careful in retaining any fixes that
CURRENT had and we did not, I apologize in advance if any
thing gets clobbered, I'm sure I'll hear about it :)

Approved by pdeuskar

Revert previous change and take back a pointy hat.

Fix panic on boot caused by setting up a NULL interrupt handler.

Submitted by: Goran Gajic
Pointy hat to: piso

o break newbus api: add a new argument of type driver_filter_t to
bus_setup_intr()

o add an int return code to all fast handlers

o retire INTR_FAST/IH_FAST

For more info: http://docs.freebsd.org/cgi/getmsg.cgi?fetch=465712+0+current/freebsd-current

Reviewed by: many
Approved by: re@

remove no longer correct comment above em_read_pcie_cap_reg

Move magic PCIe workaround constant to header - add appropriate comment

Suggested by: jfvogel

Fix TSO support on sun4v

- incorporate csjp's fix for a mishandled endian conversion
- convert PAGE_SIZE to 4096 for PCIe adapter workaround (my page size is not 4k)
- implement em_read_pcie_cap_reg where we set the max read size on pcie to 4k (taken from mxge)

Reviewed by: scottl and jfvogel

Implement new ETHER_BPF_MTAP macro. Roll back the various changes
made to accommodate the chip being in promiscuous mode while
offloading VLAN tag processing to the hardware. We can now
properly handle the absence of VLAN tags from hardware stripping.

Reviewed by: rwatson, andre
MFC after: 1 month

Add MSI support to em(4), bce(4), and mpt(4). For now, we only support
devices that support a maximum of 1 message, and we use that 1 message
instead of the INTx rid 0 IRQ with the same interrupt handler, etc.

Instead of using the legacy if_timer/if_watchdog interface create
our own watchdog that piggybacks on the em_local_timer() routine.

We suppose that the if_timer/if_watchdog interface should be
obsoleted, since it doesn't fit the modern SMP network stack.
NIC drivers should create their own watchdogs, that check and
clear the timers always holding driver's lock.

In collaboration with: jfv, scottl

Fix compile botch in the last panic botch fix. :(

Pointy hat: jhb
Reported by: brueffer

Fix botch in last commit (I tested on 6.x which doesn't have TSO):
- Test the mac_type rather than if_hwassist (since ifp doesn't exist yet)
to determine if the adapter supports TSO and thus to change the sizes
for the bus_dma tag.

Reviewed by: glebius

Allocate receive and transmit data structures during attach() and free them
during detach() similar to other NIC drivers rather than allocating them
during init() and freeing them during stop():
- Move creation of tx bus_dma tag amd maps and tx_buffer_area from
em_setup_transmit_structures() to em_allocate_transmit_structures().
- Call em_allocate_xxx_structures() in em_attach().
- Only call em_free_xxx_structures() in em_detach().
- Change em_setup_xxx_structures() to free any existing tx or rx buffers
and in the case of rx repopulate the ring with newer buffers.

Reviewed by: jfv

- Use callout_init_mtx() to close various callout-related races.
- Drain the two timers in detach.
- Check IFF_DRV_RUNNING in the link task and bail w/o doing anything if
it is clear.

Reviewed by: jfv, scottl

Rework the transmit register handling. In em_encap() store index of
the EOP descriptor in the first descriptor of the packet. And then
in em_txeof() search for DD bits set only in the EOP descriptors,
embedding the cleanup of all packet's descriptors into inner loop.

This change is important for future chips, where DD bit is going
to be set only on the EOP descriptors.

Submitted by: jfv

Merge new vendor release - 6.2.9.

Details:
o if_em.c changes:
- Added several new PCI ids.
- Check em_check_phy_reset_block() before doing SIOCSIFMEDIA ioctl.
- Don't touch TARC registers, they are now handled in shared
code in if_em_hw.c.
- Move RDH and RDT setting to the end of
em_initialize_receive_unit().
- Declare em_read_pcie_cap_reg(), now empty.
o if_em_hw.c dropped in from vendor, then restored rev. 1.15.
o if_em_hw.h dropped in from vendor, then modified:
- Added RX overrun interrupt flag to interrupt enable mask.
- Remove declarations of em_io_read(), em_io_write().

Approved by: jfv

Backout bogus checkin to HEAD
Approved by: scottl

This is the merge of the Intel 6.2.9 driver. It provides all new shared code,
new device support, and it is hoped a more stable driver for 6.2. RELEASE.
This checkin was discussed and approved today by RE, scottl, jhb, and pdeuskar

Back out rev. 1.152 as it was breaking vlan tag insertion when vlan tag
stripping was disabled due to being in promisc mode. This is a hardware
bug. Update comment to explicitly state the reason the manual vlan tag
insertion in this case. See rev. 1.53 for further information as well.

Noticed by: jhb

Small style and comment adjustments.

Reviewed by: jfv

Remove manual vlan header insertion in em_encap(). It is unnecessary as the
generic vlan_start() takes care of it when vlan hardware insertion is disabled.

In em_set_promisc() add a note that BPF may also be enabled without going into
promisc mode.

Reviewed by: jfv

Change em_transmit_checksum_setup() to deal with already inserted vlan headers,
IP options and add skeleton IPv6 support. The new code structure can also be
easily enhanced to support new/more protocols (SCTP) in the future.

Reviewed by: jfv

Change em_tso_setup() to deal with already inserted vlan headers, IP options
and add skeleton IPv6 support. The new code structure can also be easily
enhanced to support new/more protocols (SCTP) and IP fragmentation in the
future.

In em_encap() only try to do TSO if 'dotso' is true.

Reviewed by: jfv

Only advertize IFCAP_TSO4 capabilities. IPv6 is not yet supported.

Reviewed by: jfv

Handle all error cases from bus_dmamap_load_mbuf_sg(). Those are:

- EFBIG means the mbuf chain was too long and bus_dma ran out of segments.
Defragment the mbuf chain and try again. (Already existed, not changed.)
- ENOMEM means bus_dma could not obtain enough bounce buffers at this point
in time. Defer sending and try again later.
- All other errors, in particular EINVAL, are fatal and prevent the mbuf
chain from ever going through. Drop it and report error.
- Checking (nsegs == 0) is unnecessary as bus_dmamap_load_mbuf_sg() always
reports an error if it is < 1.

This prevents broken packets from clogging the interface queue indefinately.

Discussed with: scottl
Reviewed by: jfv

Move the initialization of the hardware capabilities in em_init_locked()
before em_setup_transmit_structures() as it needs this information to
properly set up TSO parameters.

Reviewed by: jfv

Don't forget to add curly braces when doing more than one line of actions
after a 'if' statement.

Pointy hat to: andre

Move ethernet VLAN tags from mtags to its own mbuf packet header field
m_pkthdr.ether_vlan. The presence of the M_VLANTAG flag on the mbuf
signifies the presence and validity of its content.

Drivers that support hardware VLAN tag stripping fill in the received
VLAN tag (containing both vlan and priority information) into the
ether_vtag mbuf packet header field:

m->m_pkthdr.ether_vtag = vlan_id; /* ntohs()? */
m->m_flags |= M_VLANTAG;

to mark the packet m with the specified VLAN tag.

On output the driver should check the mbuf for the M_VLANTAG flag to
see if a VLAN tag is present and valid:

if (m->m_flags & M_VLANTAG) {
... = m->m_pkthdr.ether_vtag; /* htons()? */
... pass tag to hardware ...
}

VLAN tags are stored in host byte order. Byte swapping may be necessary.

(Note: This driver conversion was mechanic and did not add or remove any
byte swapping in the drivers.)

Remove zone_mtag_vlan UMA zone and MTAG_VLAN definition. No more tag
memory allocation have to be done.

Reviewed by: thompsa, yar
Sponsored by: TCP/IP Optimization Fundraise 2005

Fix issues found by Coverity (223392, 223393) due to TSO additions

Submitted by: Matthew Jacob

Fix style(9) issues in the TSO specific changes.

Pointed out by: jmallett

Second attempt at fixing module build

Pointyhat: pdeuskar

Fix build breakage while compiling em as a module.

Add support for TSO. Thanks to Andre for adding support in the stack
and Jack Vogel for driver changes.

Submitted by: Jack Vogel

add a newbus method for obtaining the bus's bus_dma_tag_t... This is
required by arches like sparc64 (not yet implemented) and sun4v where there
are seperate IOMMU's for each PCI bus... For all other arches, it will
end up returning NULL, which makes it a no-op...

Convert a few drivers (the ones we've been working w/ on sun4v) to the
new convection... Eventually all drivers will need to replace the parent
tag of NULL, w/ bus_get_dma_tag(dev), though dev is usually different for
each driver, and will require hand inspection...

Reviewed by: scottl (earlier version)

Comment tweaks.

- Use pci_enable_busmaster() and pci_enable_io() to update the command
register. This really shouldn't be using pci_enable_io() directly as
bus_alloc_resource() does it already, but the cached copy of the
command word needs to be correct so the enable/disable mwi functions
work properly.
- Use pci bus accessors to read revision ID and subvendor IDs.

Reviewed by: jvogel

Add locking to the ifmedia callouts.

Reviewed by: jvogel, yongari

Fix my error in rev. 1.109.

Submitted by: jhb
Pointy hat to: glebius

Just foward declare 'struct adapter' instead of declaring an actual
'adapter' structure.

Compare the correct field against NULL when determining whether or not to
do bus_teardown_intr().

It seems that em(4) misses Tx completion interrupts under certain
conditions. The cause of missing Tx completion interrupts comes from
Tx interrupt moderation mechanism(delayed interrupts) or chipset bug.
If Tx interrupt moderation mechanism is the cause of false watchdog
timeout error we should have to fix all device drivers that have Tx
interrupt moderation capability. We may need more investigation
for this issue. Anyway, the fix is the same for both cases.

This should fix occasional watchdog timeout errors seen on a few
systems.

Reported by: -net, Patrick M. Hausen < hausen AT punkt DOT de >
Tested by: Patrick M. Hausen < hausen AT punkt DOT de >

Don't update Rx descriptor status in two different functions.

Suggested by: pdeuskar
Reviewed by: pdeuskar

Change hardcoded and incorrect number with correct define. This change is a
nop, since E1000_FDX_COLLISION_DISTANCE == E1000_HDX_COLLISION_DISTANCE.

PR: kern/101000
Submitted by: Doug Havir

Make em(4) handle too many fragmented frame with m_defrag(9).
Previously em(4) requeued the failed mbuf chains from
bus_dmamap_load_mbuf_sg(9) failure to resend it later. However,
bus_dmamap_load_mbuf_sg(9) may never complete its request as the
fragmented frames can have more than EM_MAX_SCATTER segments.
To handle the above EFBIG case, defragment the frame with m_defrag(9)
and free the mbuf chain if it can't deframent the chain due to
resource shortage.

Reviewed by glebius (with improvements)

Overhaul Rx path to recover from mbuf cluster allocation failure.
o Create one more spare DMA map for Rx handler to recover from
bus_dmamap_load_mbuf_sg(9) failure.
o Make sure to update status bit in Rx descriptors even if we failed
to allocate a new buffer. Previously it resulted in stuck condition
and em_handle_rxtx task took up all available CPU cycles.
o Don't blindly unload DMA map. Reuse loaded DMA map if received
packet has errors. This would speed up Rx processing a bit under
heavy load as it does not need to reload DMA map in case of error.
(bus_dmamap_load_mbuf_sg(9) is the most expensive call in driver
context.)
o Update if_iqdrops counter if it can't allocate a mbuf cluster.
With this change it's now possible to see queue dropped packets
with netstat(1).
o Update mbuf_cluster_failed counter if fixup code failed to
allocate mbuf header.
o Return ENOBUFS instead of ENOMEM in case of Rx fixup failure.
o Make adapter->lmp NULL in case of Rx fixup failure. Strictly
specking it's not necessary for correct operation but it makes
the intention clear.
o Remove now unused dropped_pkts member in softc.

With these changes em(4) should survive mbuf cluster allocation
failure on Rx path.

Reviewed by: pdeuskar, glebius (with improvements)

Apply alignment fixup only when programmed frame size is greater than
MCLBYTES - ETHER_ALIGN. Previously it applied the alignment fixup code
for oversized frames which would result in reduced performance on
strict alignment archs.

Merge in new driver from Intel, version 6.1.4. It adds support for
82571EB quad port copper NIC and has few minor fixes.

Details:
- if_em.c. Merged manually, viewing diff between new vendor
driver and previous one.
- if_em_hw.c. Dropped in from vendor, and then restored
revision 1.15.

10/100 PHY shouldn't support gigabit media types.

Submitted by: brad (brad@comstyle.com)
Obtained from: OpenBSD
MFC after: 1 week

Commit the results of the typo hunt by Darren Pilgrim.
This change affects documentation and comments only,
no real code involved.

PR: misc/101245
Submitted by: Darren Pilgrim <darren pilgrim bitfreak org>
Tested by: md5(1)
MFC after: 1 week

Revert back changes to made in rev 1.109 of if_em.c which were unnecessary.
This makes it easier for us to get the changes into -current and to -stable quickly.

Merge in new driver from Intel, version 6.0.5. It adds support for
80003 NICs and NICs found on ICH8 mobos, and improves support for
already known chips.

Details:
- if_em.c. Merged manually, viewing diff between new vendor
driver and previous one. This was an easy task, because
most changes between 5.1.5 and 6.0.5 are bugfixes taken
from FreeBSD.
- if_em_hw.h. Dropped in from vendor, and then restored
revisions 1.16, 1.17, 1.18.
- if_em_hw.c. Dropped in from vendor, and then restored
revision 1.15.
- if_em_osdep.h. Added new required macros from vendor file
and add a hack against define namespace mangling in
if_em_hw.h. Intel made another hack, but I prefer mine.

Prepending an mbuf after loading a DMA map results in unexpected
result. So, modify mbuf chains before loading a DMA map.

Nuke invalid use of BUS_DMA_ALLOCNOW.

Make sure to use the same DMA map in DMA map load/unload operations
by remembering a map used in bus_dmamap_load_mbuf_sg(9). I have
no idea how it could ever worked before.
This fixes a warning generated by a diagnostic check in sun4v
iommu driver.

Reported by: jb
Tested by: jb(sun4v)

Since resetting hardware takes a very long time and results in link
renegotiation, we only initialize the hardware only when it is
absolutely required. Process SIOCGIFADDR ioctl in em(4) when we know
an IPv4 address is added. Handling SIOCGIFADDR in a driver is
layering violation but it seems that there is no easy way without
rewritting hardware initialization code to reduce settle time after
reset.

This should fix a long standing bug which didn't send ARP packet when
interface address is changed or an alias address is added. Another
effect of this fix is it doesn't need additional delays anymore when
adding an alias address to the interface.
While I'm here add a new if_flags into softc which remembers current
prgroammed interface flags and make use of it when we have to program
promiscuous mode.

Tested by: Atanas <atanas AT asd DOT aplus DOT net>
Analyzed by: rwatson
Discussed with: -stable

Protect EEPROM access with the driver lock.

Honor IFF_DRV_OACTIVE in em_start_locked().

The procedure of raceless switching between polling mode and
taskqueued interrupt mode is going to be quite complex. Since
the polling mode is considered legacy feature for em(4) driver,
the decision is made to make polling and new interrupt handler
mutually exclusive, selected at compile time.

If kernel is compiled with DEVICE_POLLING, the fast taskqueued
interrupt handler code is disabled and the em_poll() and legacy
em_intr() functions are enabled. Otherwise, legacy functions
are disabled and only em_intr_fast() code is compiled.

Discussed with: scottl

Fix static array overrun.

(This will be also fixed in next vendor release.)

Coverity ID: 916
Reviewed by: Jack Vogel

Bring back arm-specific workaround from rev 1.15:
Do not use the IO-mapping to issue the reset on the 82546 on arm. For some
reason, it results in corrupted descriptors.

Restore accidentially removed rev. 1.3

Merge in new driver from Intel, version 5.1.5. Adds support for some
new chips and improves support for already supported ones.

Some details, important for future merges:
- if_em.c merged manually, viewing diff between new vendor
driver and previous one.
- if_em_hw.h dropped in from vendor, and then restored revisions
1.16, 1.17, 1.18.
- if_em_hw.c dropped in from vendor, and then two liner change made,
that restores support for two rare chips.

Back out 1.112,1.113. I don't have enough resources to fix breakages
introduced by this change.

Fix fallout from last commit - we need to program the MAC address in em_init().

em_hardware_init() in em_init() is not needed, and leads to annoying
link flap.

Submitted by: ru, Mike Tancsa

Set ifp->if_baudrate according to current speed.

- Rename em_print_link_status() to em_update_link_status().
- In em_attach() remove em_check_for_link(). Not needed here, since
already done in em_hardware_init().
- In em_attach() replace the printing block with call to
em_update_link_status().
- Remove modification of sc->link_state from em_hardware_init() and
from em_media_status(). This makes em_update_link_status() a
single point of change. Call em_update_link_status() where needed.

- Second style(9) megacleanup.
- Rename "adapter" to "sc"/"softc", to be like other drivers.

(-13 Kb less source code)

Move includes from if_em.h to if_em.c and sort them.

Fix two important typos in watchdog handling:

- Restart watchdog if we *did* processed any descriptors. [1]
- Log the watchdog event if the link is *up*. [2]

PR: kern/92948 [1]
Submitted by: Mihail Balikov <mihail.balikov interbgc.com> [1]
PR: kern/92895 [2]
Submitted by: Vladimir Ivanov <wawa yandex-team.ru> [2]

Since em(4) taskqueue is a new network context, we need to conditionally
lock Giant here.

Submitted by: Andrey V. Elsukov <bu7cher yandex.ru>

Now that the em driver no longer needs to directly touch the scheduler, remove some
unneeded headers.