uma: Deduplicate uma_small_alloc

This commit refactors the UMA small alloc code and
removes most UMA machine-dependent code.
The existing machine-dependent uma_small_alloc code is almost identical
across all architectures, except for powerpc where using the direct
map addresses involved extra steps in some cases.

The MI/MD split was replaced by a default uma_small_alloc
implementation that can be overridden by architecture-specific code by
defining the UMA_MD_SMALL_ALLOC symbol. Furthermore, UMA_USE_DMAP was
introduced to replace most UMA_MD_SMALL_ALLOC uses.

Reviewed by: markj, kib
Approved by: markj (mentor)
Differential Revision: https://reviews.freebsd.org/D45084

tpm: Refactor TIS and add a SPI attachment

Summary:
Though mostly used in x86 devices, TPM can be used on others, with a
direct SPI attachment. Refactor the TPM 2.0 driver set to use an
attachment interface, and implement a SPI bus interface.

Test Plan:
Tested on a Raspberry Pi 4, with a GeeekPi TPM2.0 module (SLB9670
TPM) using security/tpm2-tools tpm2_getcaps for very light testing against the
spibus attachment.

Reviewed by: kd
Obtained from: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D45069

files.x86: Pull in some more duplicate lines from files.{amd64,i386}

Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D44759

ice_ddp: Update package to 1.3.36.0

This is intended to be used with the upcoming ice 1.39.13-k
driver update, but is still backwards compatible with
previous versions of the driver.

Signed-off-by: Eric Joyner <erj@FreeBSD.org>

MFC after: 3 days
Sponsored by: Intel Corporation

build: Do not run ctfconvert on VDSO files

Reviewed by: emaste
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D43877

sanitizers: Avoid building genassym.c and genoffset.c with sanitizers

Some, particularly KASAN, may insert redzones around global symbols,
resulting in incorrect offset definitions because genassym.sh (ab)uses
symbol sizes to assign semantic meaning.

(Ideally I would be able to define this pattern in one place, but I
haven't found a way to define a GENSYM_CFLAGS that actually works for
all of the consumers (kern.post.mk, kmod.mk, sys/conf/files*).)

MFC after: 1 week
Sponsored by: Klara, Inc.
Sponsored by: Juniper Networks, Inc.

sys: ${CFLAGS:N-flto} -> ${CFLAGS:N-flto*}

For the same reason as the original https://reviews.freebsd.org/D9659:
-flto=<N>, -flto=full, and -flto=thin also produce the GIMPLE/bitcode
which is not supported by genassym, so filter those out as well.

Signed-off-by: Alex Xu (Hello71) <alex_y_xu@yahoo.ca>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/898

ice_ddp: Update to 1.3.35.0

This is intended to be used with the upcoming updated ice(4) version
1.38.16-k.

Signed-off-by: Eric Joyner <erj@FreeBSD.org>

MFC after: 3 days
Sponsored by: Intel Corporation

smartpqi: update to version 4410.0.2005

This updates the smartpqi driver to Microsemi's latest code. This will
be the driver for FreeBSD 14 (with updates), but no MFC is planned.

Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D41550

sys: Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

superio+ncthwm: Add hardware monitoring support

Add support for the following chips:
- Nuvoton NCT6779D
- Nuvoton NCT6796D-E

Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/719

qlnxe: Fix building as in-kernel component

Add some extra files for building the driver as part of the kernel.
Change some #defines to match those used when building as a module.

PR: 268354

Signed-off-by: Mark O'Donovan <shiftee@posteo.net>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/779

ossl: Compile newly added files into the kernel if so requested

Fixes: 9a3444d91c70 ("ossl: Add a VAES-based AES-GCM implementation for amd64")
Fixes: 9b1d87286c78 ("ossl: Add a fallback AES-GCM implementation using AES-NI")

zfs: merge openzfs/zfs@431083f75

Notable upstream pull request merges:
#12194 Fix short-lived txg caused by autotrim
#13368 ZFS_IOC_COUNT_FILLED does unnecessary txg_wait_synced()
#13392 Implementation of block cloning for ZFS
#13741 SHA2 reworking and API for iterating over multiple implementations
#14282 Sync thread should avoid holding the spa config write lock
when possible
#14283 txg_sync should handle write errors in ZIL
#14359 More adaptive ARC eviction
#14469 Fix NULL pointer dereference in zio_ready()
#14479 zfs redact fails when dnodesize=auto
#14496 improve error message of zfs redact
#14500 Skip memory allocation when compressing holes
#14501 FreeBSD: don't verify recycled vnode for zfs control directory
#14502 partially revert PR 14304 (eee9362a7)
#14509 Fix per-jail zfs.mount_snapshot setting
#14514 Fix data race between zil_commit() and zil_suspend()
#14516 System-wide speculative prefetch limit
#14517 Use rw_tryupgrade() in dmu_bonus_hold_by_dnode()
#14519 Do not hold spa_config in ZIL while blocked on IO
#14523 Move dmu_buf_rele() after dsl_dataset_sync_done()
#14524 Ignore too large stack in case of dsl_deadlist_merge
#14526 Use .section .rodata instead of .rodata on FreeBSD
#14528 ICP: AES-GCM: Refactor gcm_clear_ctx()
#14529 ICP: AES-GCM: Unify gcm_init_ctx() and gmac_init_ctx()
#14532 Handle unexpected errors in zil_lwb_commit() without ASSERT()
#14544 icp: Prevent compilers from optimizing away memset()
in gcm_clear_ctx()
#14546 Revert zfeature_active() to static
#14556 Remove bad kmem_free() oversight from previous zfsdev_state_list
patch
#14563 Optimize the is_l2cacheable functions
#14565 FreeBSD: zfs_znode_alloc: lock the vnode earlier
#14566 FreeBSD: fix false assert in cache_vop_rmdir when replaying ZIL
#14567 spl: Add cmn_err_once() to log a message only on the first call
#14568 Fix incremental receive silently failing for recursive sends
#14569 Restore ASMABI and other Unify work
#14576 Fix detection of IBM Power8 machines (ISA 2.07)
#14577 Better handling for future crypto parameters
#14600 zcommon: Refactor FPU state handling in fletcher4
#14603 Fix prefetching of indirect blocks while destroying
#14633 Fixes in persistent error log
#14639 FreeBSD: Remove extra arc_reduce_target_size() call
#14641 Additional limits on hole reporting
#14649 Drop lying to the compiler in the fletcher4 code
#14652 panic loop when removing slog device
#14653 Update vdev state for spare vdev
#14655 Fix cloning into already dirty dbufs
#14678 Revert "Do not hold spa_config in ZIL while blocked on IO"

Obtained from: OpenZFS
OpenZFS commit: 431083f75bdd3efaee992bdd672625ec7240d252

superio,ftgpio: Add support for Fintek F81865 GPIO

Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/674
Differential Revision: https://reviews.freebsd.org/D37893

ice(4): Update to 1.37.7-k

Notable changes include:

- DSCP QoS Support (leveraging support added in
rG9c950139051298831ce19d01ea5fb33ec6ea7f89)
- Improved PFC handling and TC queue assignments (now all remaining
queues are assigned to TC 0 when more than one TC is enabled and the
number of available queues does not evenly divide between them)
- Support for dumping the internal FW state for additional debugging by
Intel support
- Support for allowing "No FEC" to be a valid state for the LESM to
negotiate when using non-standard compliant modules

Also includes various bug fixes and smaller enhancements, too.

Signed-off-by: Eric Joyner <erj@FreeBSD.org>

Reviewed by: erj@
Tested by: Jeff Pieper <jeffrey.pieper@intel.com>
MFC after: 3 days
Relnotes: yes
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D38109

amd64: record a dependency of vdso on its linker script

Reviewed by: emaste, val_packett.cool
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D38392

enic: Cisco VIC driver

This driver is based of the enic (Cisco VIC) DPDK driver. It provides
basic ethernet functionality. Has been run with various VIC cards to
do UEFI PXE boot with NFS root.

ice_ddp: Update package to 1.3.30.0

This updated DDP is intended to be used with the forthcoming ice(4)
driver update to 1.37.7-k. (But it will still work with the current
version.)

Co-authored-by: Piotr Kubaj <pkubaj@FreeBSD.org>
Signed-off-by: Eric Joyner <erj@FreeBSD.org>

MFC after: 1 week
Sponsored by: Intel Corporation

wdatwd: Add support for ACPI WDAT based watchdog timer.

Simply said, WDAT is an abstraction for the real WDT hardware. For
instance, to add a newer generation WDT to ichwd(4), one must know the
detailed hardware registers, etc..

With WDAT, the necessary IO accesses to operate the WDT are comprehensively
described in it and no hardware knowledge is required.

With this driver, the WDT on Advantech ARK-1124C, Dell R210 and Dell R240 are
detected and operated flawlessly.
* While R210 is also supported by ichwd(4), others are not supported yet.

The unfortunate thing is that not all systems have WDAT defined.

Submitted by: t_uemura at macome.co.jp
Reviewed by: hrs
Differential Revision: https://reviews.freebsd.org/D37493

zfs: fix static module build broken in 1f1e2261e

ice(4): Add RDMA Client Interface

This allows the "irdma" driver to communicate with the ice(4)
driver to allow it access to the underlying device's hardware
resources as well as synchronize access to shared resources.

This interface already existed in the standalone out-of-tree
1.34.2 driver; this commit adds and enables it in the in-kernel
driver.

Note:

Adds hack to module Makefile to compile interface/.m files

These are required for the RDMA client interface, but they don't
build as-is like the normal .c files. The source directory doesn't
seem to be included by default, so add lines that specifically
add them as libraries so that ice_rdma.h can be found and the
interface files will compile.

Signed-off-by: Eric Joyner <erj@FreeBSD.org>

MFC after: 1 week
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D30889

ice(4): Update to 1.34.2-k

- Adds FW logging support
- Once enabled, this lets the firmware print event and error messages
to the log, increasing the visibility into what the hardware is
doing; this is useful for debugging
- General bug fixes
- Adds inital DCB support to the driver
- Notably, this adds support for DCBX to the driver; now with the
fw_lldp sysctl set to 1, the driver and adapter will adopt a DCBX
configuration sent from a link partner
- Adds statistcs sysctls for priority flow control frames
- Adds new configuration sysctls for DCB-related features: (VLAN) user
priority to TC mapping; ETS bandwidth allocation; priority flow
control
- Remove unused SR-IOV files (until support gets added)

Signed-off-by: Eric Joyner <erj@FreeBSD.org>

Tested by: jeffrey.e.pieper@intel.com
MFC after: 3 days
MFC with: 213e91399b, e438f0a975
Relnotes: yes
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D34024

ice_ddp: Update to 1.3.27.0

This is intended to be used with forthcoming ice(4) driver version 1.34.2.

Signed-off-by: Eric Joyner <erj@FreeBSD.org>

Sponsored by: Intel Corporation

x86: Speed up clock calibration

Prior to this commit, the TSC and local APIC frequencies were calibrated
at boot time by measuring the clocks before and after a one-second sleep.
This was simple and effective, but had the disadvantage of *requiring a
one-second sleep*.

Rather than making two clock measurements (before and after sleeping) we
now perform many measurements; and rather than simply subtracting the
starting count from the ending count, we calculate a best-fit regression
between the target clock and the reference clock (for which the current
best available timecounter is used). While we do this, we keep track
of an estimate of the uncertainty in the regression slope (aka. the ratio
of clock speeds), and stop measuring when we believe the uncertainty is
less than 1 PPM.

In order to avoid the risk of aliasing resulting from the data-gathering
loop synchronizing with (a multiple of) the frequency of the reference
clock, we add some additional spinning depending upon the iteration number.

For numerical stability and simplicity of implementation, we make use of
floating-point arithmetic for the statistical calculations.

On the author's Dell laptop, this reduces the time spent in calibration
from 2000 ms to 29 ms; on an EC2 c5.xlarge instance, it is reduced from
2000 ms to 2.5 ms.

Reviewed by: bde (previous version), kib
MFC after: 1 month
Sponsored by: https://www.patreon.com/cperciva
Differential Revision: https://reviews.freebsd.org/D33802

amd64: Pass DEBUG when building VDSO wrapper

Not doing so results in ctfconvert failing with

ERROR: ctfconvert: elf-vdso.so.o doesn't have type data to convert

On FreeBSD this is non-fatal, since the ctf tools have a hack to make
such errors not fail the build and instead just silently continue
without CTF data (which is a bad idea these days and should probably be
removed; they date back to the original import). However, those are
under #ifdef __FreeBSD__ so do not apply when cross-building from
non-FreeBSD, causing the build to fail.

Fix this by forwarding DEBUG on to the compiler invocation for the VDSO
wrapper. It's assembly so it's not hugely useful, but there is a
non-zero amount of information preserved, and other assembly files are
built with -g by default too so this matches them; the alternative would
be to tag the files.amd64 entries with no-ctfmerge. Note that the VDSO
itself is still compiled without debug info, this only affects the
wrapper linked into the kernel.

Fixes: 98c8b6252496 ("vdso for ia32 on amd64"), ab4524b3d7fb ("amd64: wrap 64bit sigtramp into vdso")
MFC after: 1 month

Enforce that vdso does not contain runtime relocations

Reviewed by: emaste
Discussed with: jrtc27
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 month
Differential revision: https://reviews.freebsd.org/D32960

vdso for ia32 on amd64

Reviewed by: emaste
Discussed with: jrtc27
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 month
Differential revision: https://reviews.freebsd.org/D32960

amd64: wrap 64bit sigtramp into vdso

Reviewed by: emaste
Discussed with: jrtc27
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 month
Differential revision: https://reviews.freebsd.org/D32960

iavf(4): Split source and update to 3.0.26-k

The iavf(4) driver now uses a different source base from ixl(4), since
it will be the standard VF driver for new Intel Ethernet products going
forward, including ice(4). It continues to use the iflib framework
for network drivers.

Since it now uses a different source code base, this commit adds a new
sys/dev/iavf entry, but it re-uses the existing module name so no
configuration changes are necessary.

Signed-off-by: Eric Joyner <erj@FreeBSD.org>

Reviewed by: kbowling@
Tested by: lukasz.szczepaniak@intel.com
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D28636

netinet: Deduplicate most in_cksum() implementations

in_cksum() and related routines are implemented separately for each
platform, but only i386 and arm have optimized versions. Other
platforms' copies of in_cksum.c are identical except for style
differences and support for big-endian CPUs.

Deduplicate the implementations for the rest of the platforms. This
will make it easier to implement in_cksum() for unmapped mbufs. On arm
and i386, define HAVE_MD_IN_CKSUM to mean that the MI implementation is
not to be compiled.

No functional change intended.

Reviewed by: kp, glebius
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33095

ossl: Add support for AES-CBC cipher

AES-CBC OpenSSL assembly is used underneath.
The glue layer(ossl_aes.c) is based on CHACHA20 implementation.
Contrary to the SHA and CHACHA20, AES OpenSSL assembly logic
does not have a fallback implementation in case CPU doesn't
support required instructions.
Because of that CPU caps are checked during initialization and AES
support is advertised only if available.
The feature is available on all architectures that ossl supports:
i386, amd64, arm64.

The biggest advantage of this patch over existing solutions
(aesni(4) and armv8crypto(4)) is that it supports SHA,
allowing for ETA operations.

Sponsored by: Stormshield
Obtained from: Semihalf
Reviewed by: jhb (previous version)
Differential revision: https://reviews.freebsd.org/D32099

Revert "ossl: Add support for AES-CBC cipher"

This reverts commit 849faf4e0ba9a8b8f24ff34da93a0fd46c14eda9.

ossl: Add support for AES-CBC cipher

AES-CBC OpenSSL assembly is used underneath.
The glue layer(ossl_aes.c) is based on CHACHA20 implementation.
Contrary to the SHA and CHACHA20, AES OpenSSL assembly logic
does not have a fallback implementation in case CPU doesn't
support required instructions.
Because of that CPU caps are checked during initialization and AES
support is advertised only if available.
The feature is available on all architectures that ossl supports:
i386, amd64, arm64.

The biggest advantage of this patch over existing solutions
(aesni(4) and armv8crypto(4)) is that it supports SHA,
allowing for ETA operations.

Sponsored by: Stormshield
Obtained from: Semihalf
Reviewed by: jhb
Differential revision: https://reviews.freebsd.org/D32099

Drop cloudabi

According to https://github.com/NuxiNL/cloudlibc:
CloudABI is no longer being maintained. It was an awesome experiment,
but it never got enough traction to be sustainable.

There is no reason to keep it in FreeBSD.

Approved by: ed (private mail)
Reviewed by: emaste
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D31923

amd64: move signal handling and register structures manipulations into exec_machdep.c

from machdep.c which is too large pile of unrelated things.
Some ptrace functions are moved from machdep.c to ptrace_machdep.c.

Now machdep.c contains code mostly related to the low level initialization
and regular low level operation of the architecture, while signal MD code
and registers handling is placed in exec_machdep.c.

Reviewed by: jhb, markj
Discussed with: jrtc27
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31954

x86: duplicate acpi_wakeup.c per i386 and amd64

The file as is is the maze of #ifdef passages, all slightly different.
Divorcing i386 and amd64 version actually makes changing the code
easier, also no changes for i386 are planned.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31931

vmd(4): Major driver refactoring

- Re-implement pcib interface to use standard pci bus driver on top of
vmd(4) instead of custom one.
- Re-implement memory/bus resource allocation to properly handle even
complicated configurations.
- Re-implement interrupt handling to evenly distribute children's MSI/
MSI-X interrupts between available vmd(4) MSI-X vectors and setup them
to be handled by standard OS mechanisms with minimal overhead, except
sharing when unavoidable.

Successfully tested on Dell XPS 13 laptop with Core i7-1185G7 CPU (VMD
device ID 0x9a0b) and single NVMe SSD, dual-booting with Windows 10.

Successfully tested on Supermicro X11DPI-NT motherboard with Xeon(R)
Gold 6242R CPUs (VMD device ID 0x201d), simultaneously handling NVMe
SSD on one PCIe port and PLX bridge with 3 NVMe and 1 AHCI SSDs on
another. Handles SSD hot-plug (except Optane 905p for some reason,
which are not detected until manual bus rescan) and enabled IOMMU
(directly connected SSDs work, but ones connected to the PLX fail
without errors from IOMMU).

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.
Differential revision: https://reviews.freebsd.org/D31762

Add support for Lewisburg GPIO through P2SB

Obtained from: Semihalf
Sponsored by: Stormshield
Differential revision: https://reviews.freebsd.org/D31269

LinuxKPI: Make FPU sections thread-safe and use the NOCTX flag.

Reviewed by: kib
Submitted by: greg@unrelenting.technology
Differential Revision: https://reviews.freebsd.org/D29921
MFC after: 1 week
Sponsored by: NVIDIA Networking

ice_ddp: Update to 1.3.24.0

This version is intended to be used with the 0.29.4 version of the
ice(4) driver, which will be be committed afterwards.

Signed-off-by: Eric Joyner <erj@FreeBSD.org>

Reviewed by: stallamr_netapp.com
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D30887

linux(4): Get rid of Linuxulator kernel build options.

Stop confusing people, retire COMPAT_LINUX and COMPAT_LINUX32 kernel
build options. Since we have 32 and 64 bit Linux emulators, we can't build both
emulators together into the kernel. I don't think it matters, Linux emulation
depends on loadable modules (via rc).

Cut LINPROCFS and LINSYSFS for consistency.

PR: 215061
Reviewed by: bcr (manpages), trasz
Differential Revision: https://reviews.freebsd.org/D30751
MFC after: 2 weeks

x86/xen: remove PVHv1 code

PVHv1 was officially removed from Xen in 4.9, so just axe the related
code from FreeBSD.

Note FreeBSD supports PVHv2, which is the replacement for PVHv1.

Sponsored by: Citrix Systems R&D
Reviewed by: kib, Elliott Mitchell
Differential Revision: https://reviews.freebsd.org/D30228

x86: remove gcov kernel support

Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D29529

ice(4): Update to version 0.28.1-k

This updates the driver to align with the version included in
the "Intel Ethernet Adapter Complete Driver Pack", version 25.6.

There are no major functional changes; this mostly contains
bug fixes and changes to prepare for new features. This version
of the driver uses the previously committed ice_ddp package
1.3.19.0.

Signed-off-by: Eric Joyner <erj@FreeBSD.org>

Tested by: jeffrey.e.pieper@intel.com
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D28640

ossl: Add ChaCha20 cipher support.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D28756

ossl: Add Poly1305 digest support.

Reviewed by: cem
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D28754

ice_ddp: Update package file to 1.3.19.0

This package is intended to be used with ice(4) version 0.28.1-k.
That update will happen in a forthcoming commit.

Signed-off-by: Eric Joyner <erj@FreeBSD.org>
Sponsored by: Intel Corporation

amd64: implement strlen in assembly, take 2

Tested with glibc test suite.

The C variant in libkern performs excessive branching to find the zero
byte instead of using the bsfq instruction. The same code patched to use
it is still slower than the routine implemented here as the compiler
keeps neglecting to perform certain optimizations (like using leaq).

On top of that the routine can be used as a starting point for copyinstr
which operates on words intead of bytes.

The previous attempt had an instance of swapped operands to andq when
dealing with fully aligned case, which had a side effect of breaking the
code for certain corner cases. Noted by jrtc27.

Sample results:

$(perl -e "print 'A' x 3"):
stock: 211198039
patched:338626619
asm: 465609618

$(perl -e "print 'A' x 100"):
stock: 83151997
patched: 98285919
asm: 120719888

Reviewed by: jhb, kib
Differential Revision: https://reviews.freebsd.org/D28779

Revert "amd64: implement strlen in assembly"

This reverts commit af366d353b84bdc4e730f0fc563853abc338271c.

Trips over '\xa4' byte and terminates early, as found in
lib/libc/gen/setdomainname_test:setdomainname_basic testcase

However, keep moving libkern/strlen.c out of conf/files.

Reported by: lwhsu

ndis(4): remove as previous announced

nids(4) was a clever idea in the early 2000's when the market was
flooded with 10/100 NICs with Windows-only drivers, but that hasn't been
the case for ages and the driver has had no meaningful maintenance in
ages. It only supports Windows-XP era drivers.

Also remove:
- ndis support from wpa_supplicant
- ndiscvt(8)

Reviewed By: emaste, bcr (manpages)
Differential Revision: https://reviews.freebsd.org/D27609

Add missing no-ctfconvert to C generation rules

Stop running ctfconvert over generated C files in the kernel by marking
them with no-ctfconvert.

This fixes warnings from ctfconvert trying to parse C files:

ctfconvert: file.c: Couldn't read ehdr: Invalid argument

Reviewed by: emaste, mmel
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D28156

linuxkpi: add kernel_fpu_begin/kernel_fpu_end

With newer AMD GPUs (>=Navi,Renoir) there is FPU context usage in the
amdgpu driver.
The `kernel_fpu_begin/end` implementations in drm did not even allow nested
begin-end blocks.

Submitted by: Greg V
Reviewed By: manu, hselasky
Differential Revision: https://reviews.freebsd.org/D28061

Remove bvmconsole and bvmdebug.

Now that bhyve(8) supports UART, bvmconsole and bvmdebug are no longer needed.

This also removes the '-b' and '-g' flag from bhyve(8). These two flags were
marked deprecated in r368519.

Reviewed by: grehan, kevans
Approved by: kevans (mentor)
Differential Revision: https://reviews.freebsd.org/D27490

Fix i386 linux module after r367395.

In r367395 parts of machine dependent linux_dummy.c were moved to a new
machine independent file sys/compat/linux/linux_dummy.c and the existing
linux_dummy.c was renamed to linux_dummy_machdep.c.

Add linux_dummy_machdep.c to the linux module for i386.
Rename sys/amd64/linux32/linux_dummy.c for consistency.
Add the new linux_dummy.c to the linux module for i386.

Move kern_clocksource.c to sys/conf/files

Sponsored by: The FreeBSD Foundation

linux(4): Deduplicate unimpl/dummy syscall handlers

No functional change.

Reviewed by: emaste, trasz
Differential Revision: https://reviews.freebsd.org/D27099

Add a kernel crypto driver using assembly routines from OpenSSL.

Currently, this supports SHA1 and SHA2-{224,256,384,512} both as plain
hashes and in HMAC mode on both amd64 and i386. It uses the SHA
intrinsics when present similar to aesni(4), but uses SSE/AVX
instructions when they are not.

Note that some files from OpenSSL that normally wrap the assembly
routines have been adapted to export methods usable by 'struct
auth_xform' as is used by existing software crypto routines.

Reviewed by: gallatin, jkim, delphij, gnn
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D26821

move vmware pv drivers to sys/conf/files

VMware now has arm64 support; move these to MI files in advance of
building them on arm64.

PR: 250308
Reported by: Vincent Milum Jr
MFC after: 1 week
Sponsored by: The FreeBSD Foundation

10Gigabit Ethernet driver for AMD SoC

This patch has the driver for 10Gigabit Ethernet controller in AMD
SoC. This driver is written compatible to the Iflib framework. The
existing driver is for the old version of hardware. The submitted
driver here is for the recent versions of the hardware where the Ethernet
controller is PCI-E based.

Submitted by: Rajesh Kumar <rajesh1.kumar@amd.com>
MFC after: 1 month
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D25793

ice_ddp: Update package file to 1.3.16.0

This package is intended to be used with ice(4) version 0.26.16. That
update will happen in a forthcoming commit.

MFC after: 3 days
Sponsored by: Intel Corporation

Driver for 4x10Gb Ethernet reference NIC FPGA design for NetFPGA SUME
development board.

Submitted by: Denis Salopek <denis.salopek AT fer.hr>
Reported by: zec, bz (src); rgrimes, bcr (manpages)
MFC after: 7 days
Sponsored by: Google Summer of Code 2020
Differential Revision: https://reviews.freebsd.org/D26074

Merge OpenZFS support in to HEAD.

The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
project quotas, encrypted datasets,
allocation classes, vectorized raidz,
vectorized checksums, various command line
improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D25872

ixl(4): Add FW recovery mode support and other things

Update the iflib version of ixl driver based on the OOT version ixl-1.11.29.

Major changes:

- Extract iflib specific functions from ixl_pf_main.c to ixl_pf_iflib.c
to simplify code sharing between legacy and iflib version of driver

- Add support for most recent FW API version (1.10), which extends FW
LLDP Agent control by user to X722 devices

- Improve handling of device global reset

- Add support for the FW recovery mode

- Use virtchnl function to validate virtual channel messages instead of
using separate checks

- Fix MAC/VLAN filters accounting

Submitted by: Krzysztof Galazka <krzysztof.galazka@intel.com>
Reviewed by: erj@
Tested by: Jeffrey Pieper <jeffrey.e.pieper@intel.com>
MFC after: 1 week
Relnotes: yes
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D24564

ice(4): Introduce new driver for Intel E800 Ethernet controllers

The ice(4) driver is the driver for the Intel E8xx series Ethernet
controllers; currently with codenames Columbiaville and
Columbia Park.

These new controllers support 100G speeds, as well as introducing
more queues, better virtualization support, and more offload
capabilities. Future work will enable virtual functions (like
in ixl(4)) and the other functionality outlined above.

For full functionality, the kernel should be compiled with
"device ice_ddp" like in the amd64 NOTES file, and/or
ice_ddp_load="YES" should be added to /boot/loader.conf so that
the DDP package file included in this commit can be downloaded
to the adapter. Otherwise, the adapter will fall back to a single
queue mode with limited functionality.

A man page for this driver will be forthcoming.

MFC after: 1 month
Relnotes: yes
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D21959

Remove support for DES and Triple DES from OCF.

It no longer has any in-kernel consumers via OCF. smbfs still uses
single DES directly, so sys/crypto/des remains for that use case.

Reviewed by: cem
Relnotes: yes
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D24773

Remove support for the Blowfish algorithm from OCF.

It no longer has any in-kernel consumers.

Reviewed by: cem
Relnotes: yes
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D24772

Don't try ctfconvert on file without debug info.

This was currently an ignored error but will change to a hard error
eventually.

Differential Revision: https://reviews.freebsd.org/D24536

More fixes to build the kernel with a compiler that defaults to -fno-common

Using the same approach as the last commit for the files used by genassym.sh.

Obtained from: CheriBSD

xen-locore: Silence DWARF2 section warning

Silence the "DWARF2 can only represent one section per compilation unit"
warning in amd64 GENERIC builds by disabling Clang's debuginfo generation for
this assembler file (-g0). The message is replaced by a warning from
ctfconvert that there is no debuginfo to convert (future work).

The file contains some metadata (several ELF notes) and some code. The code
does not appear to have anything that debuginfo would aid.

I looked at the generated debuginfo (readelf -w xen-locore.o) prior to this
change, and the metadata that would be disabled are things like associated
between binary offset and code line number (not especially useful with a
disassembler), and label metadata for the entry points (not especially useful
as this is already in the symbol table).

Reviewed by: royger
Differential Revision: https://reviews.freebsd.org/D24384

Move font.h generation to conf/files from conf/files.*

Use ${SRCTOP} instead of /usr/share.
Prefer to depend on option sc_dflt_fnt instead of sc.
gc the 4 otherwise identical instances in the tree.
Platforms that don't need this won't included it.

Fix old-style build

Fix the old-style build by using ${SRCTOP} instead of a weird
construct that only works for new-style build.
Simplify the building of keymap files by using macros
Move atkbdmap.h in files.x86
This has been broken since r296899 which removed the implicit
dependency on /usr/share.

Revert r357349, since the clang 10.0.0 warning was actually correct, and
the ! operator should have been a ~ instead:

Merge r357348 from the clang 10.0.0 import branch:

Disable new clang 10.0.0 warnings about converting the result of
shift operations to a boolean in tpm(4):

sys/dev/tpm/tpm_crb.c:301:32: error: converting the result of '<<' to a boolean; did you mean '(1 << (0)) != 0'? [-Werror,-Wint-in-bool-context]
WR4(sc, TPM_CRB_CTRL_CANCEL, !TPM_CRB_CTRL_CANCEL_CMD);
^
sys/dev/tpm/tpm_crb.c:73:34: note: expanded from macro 'TPM_CRB_CTRL_CANCEL_CMD'
#define TPM_CRB_CTRL_CANCEL_CMD BIT(0)
^
sys/dev/tpm/tpm20.h:60:19: note: expanded from macro 'BIT'
#define BIT(x) (1 << (x))
^

Such warnings can be useful in C++ contexts, but not so much in kernel
drivers, where this type of bit twiddling is commonplace. So disable
it for this case.

Noticed by: cem
MFC after: 3 days

Disable new clang 10.0.0 warnings about converting the result of shift
operations to a boolean in tpm(4):

sys/dev/tpm/tpm_crb.c:301:32: error: converting the result of '<<' to a boolean; did you mean '(1 << (0)) != 0'? [-Werror,-Wint-in-bool-context]
WR4(sc, TPM_CRB_CTRL_CANCEL, !TPM_CRB_CTRL_CANCEL_CMD);
^
sys/dev/tpm/tpm_crb.c:73:34: note: expanded from macro 'TPM_CRB_CTRL_CANCEL_CMD'
#define TPM_CRB_CTRL_CANCEL_CMD BIT(0)
^
sys/dev/tpm/tpm20.h:60:19: note: expanded from macro 'BIT'
#define BIT(x) (1 << (x))
^

Such warnings can be useful in C++ contexts, but not so much in kernel
drivers, where this type of bit twiddling is commonplace. So disable it
for this case.

MFC after: 3 days

Remove sio(4).
It had been disconnected from build in r181233 in 2008.

Reviewed by: imp

Add the pvscsi driver to the tree.

This driver allows to usage of the paravirt SCSI controller
in VMware products like ESXi. The pvscsi driver provides a
substantial performance improvement in block devices versus
the emulated mpt and mps SCSI/SAS controllers.

Error handling in this driver has not been extensively tested
yet.

Submitted by: vbhakta@vmware.com
Relnotes: yes
Sponsored by: VMware, Panzura
Differential Revision: D18613

Move all the sys/dev/[a-j]* that are common to files.x86

All these device entries are common between the two files. Move them to
files.x86. Also sort entries from this range into proper order in files.amd64.

emulate illumos membar_producer with atomic_thread_fence_rel

membar_producer is supposed to be a store-store barrier.
Also, in the code that FreeBSD has ported from illumos membar_producer
is used only with regular stores to regular memory (with respect to
caching).

We do not have an MI primitive for the store-store barrier, so
atomic_thread_fence_rel is the closest we have as it provides
(load | store) -> store barrier.

Previously, membar_producer was an empty function call on all 32-bit
arm-s, 32-bit powerpc, riscv and all mips variants. I think that it was
inadequate.
On other platforms, such as amd64, arm64, i386, powerpc64, sparc64,
membar_producer was implemented using stronger primitives than required
for a store-store barrier with respect to regular memory access.
For example, it used sfence on amd64 and lock-ed nop in i386 (despite TSO).
On powerpc64 we now use recommended lwsync instead of eieio.
On sparc64 FreeBSD uses TSO mode.
On arm64/aarch64 we now use dmb sy instead of dmb ish. Not sure if this
is an improvement, actually.

After this change we can drop opensolaris_atomic.S for aarch64, amd64,
powerpc64 and sparc64 as all required atomic operations have either
direct or light-weight mapping to FreeBSD native atomic operations.

Discussed with: kib
MFC after: 4 weeks

This driver attaches to the Intel VMD drive and connects a new PCI domain
starting at the max. domain, and then work down. Then existing FreeBSD
drivers will attach. Interrupt routing from the VMD MSI-X to the NVME
drive is not well known, so any interrupt is sent to all children that
register.

VROC used Intel meta data so graid(8) works with it. However, graid(8)
supports RAID 0,1,10 for read and write. I have some early code to
support writes with RAID 5. Note that RAID 5 can have life issues
with SSDs since it can cause write amplification from updating the parity
data.

Hot plug support needs a change to skip the following check to work:
if (pcib_request_feature(dev, PCI_FEATURE_HP) != 0) {
in sys/dev/pci/pci_pci.c.

Looked at by: imp, rpokala, bcr
Differential Revision: https://reviews.freebsd.org/D21383

Fix bogusly declared WERRORs in kernel build

Many arm kernel configs bogusly specified WERROR=-Werror. There's no
reason for this because the default is that and there's no reason to
override. These date from a time when we needed to add additional
warning->error suppression. They are obsolete and were cut and paste
propagated from file to file.

Comment out all the WERROR=.... lines in powerpc. They aren't bogus,
but were appropriate for the old defaults for gcc4.2.1. Now that we've
made the policy decision to suppress -Werror by default on these
platforms, it is appropriate to comment these out. People wishing to
fix these errors can still un-comment them out, or say WERROR=-Werror
on the command line.

Fix two instances (cut and paste propagation) of hard-coded -Werror
in x86 code. Replace with ${WERROR} instead. This is a no-op change
except for people who build WERROR=-Wno-error :).

This should fix tinderbox / CI breakage.

NTB Tool: Test driver for NTB hardware drivers.

NTB Tool driver is meant for testing NTB hardware driver functionalities,
such as doorbell interrupts, link events, scratchpad registers and memory
windows. This is a port of ntb_tool driver from Linux. It has been
verified on top of AMD and PLX NTB HW drivers.

Submitted by: Arpan Palit <arpan.palit@amd.com>
Cleaned up by: mav
MFC after: 2 weeks
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D18819

Move the common x86 ipmi files to files.x86. The powerpc file list is different
enough that unification will have to wait for the next pass.

Reviewed by: jhb (verbal OK on irc)
Differential Revision: https://reviews.freebsd.org/D21248

The x86 part of hwpmc is shared, so move it to files.x86.

Reviewed by: jhb (verbal OK on irc)
Differential Revision: https://reviews.freebsd.org/D21248

Windows ndis support is x86 only. Move the MI parts there.

Reviewed by: jhb (verbal OK on irc)
Differential Revision: https://reviews.freebsd.org/D21248

Intel's isci is part of the chipset, so it is x86 specific.

Reviewed by: jhb (verbal OK on irc)
Differential Revision: https://reviews.freebsd.org/D21248

Move hyperv to files.x86

Move the comomon part of hyperv to files.x86.

Reviewed by: jhb (verbal OK on irc)
Differential Revision: https://reviews.freebsd.org/D21248

The bxe driver, QLogic NetXtreme II Ethernet 10Gb PCIe adapter driver, is x86
specific, and only builds there. Likewise the module is built there. Move it to
the x86-only files.x86.

Reviewed by: jhb (verbal OK on irc)
Differential Revision: https://reviews.freebsd.org/D21248

The ACPI parts are identical between i386 and amd64

Apart from one MD file, ACPI is a x86 implementation, not specific to either
i386 or amd64, so put it into files.x86. Other architectures include fewer
files for the same options, so it can't move into the MI files file.

Reviewed by: jhb (verbal OK on irc)
Differential Revision: https://reviews.freebsd.org/D21248

Move via padlock files to files.x86.

VIA Padlock support is for VIA C3, C7 and Eden processors, which are 64bit x86
processors.

Reviewed by: jhb (verbal OK on irc)
Differential Revision: https://reviews.freebsd.org/D21248

Apart from one MD file, aesni is common to x86. Move it into files.x86.

Reviewed by: jhb (verbal OK on irc)
Differential Revision: https://reviews.freebsd.org/D21248

Move all the hp* drivers too files.x86

The HPT drivers are all x86 only. Move them to files.x86. Because of the way we
run uudecode, we can use $M instead of needing entries for them in separate
files.

Reviewed by: jhb (verbal OK on irc)
Differential Revision: https://reviews.freebsd.org/D21248

Move the identical x86 lines to files.x86

Move all the identical x86 lines to files.x86. The non-identical ones should be
unified and moved as well, but that would require additional changes that would
need a more careful review and may not be MFCable, so I'll do them
separately. I'll delete the mildly snarky comment when things are unified.

Reviewed by: jhb (verbal OK on irc)
Differential Revision: https://reviews.freebsd.org/D21248

nvme has been moved to 'files' so shouldn't be here anymore. It works on
powerpc64 and arm64 these days as well as amd64/i386.

ed(4) has been removed from the tree, but these were forgotten in r347911.

ukbdmap.h rule was identical on all platforms, so move them into sys/conf/files.

This allows us to remove 'nodevice ukbd' from the arm64 NOTES file.

Create files.x86

files.x86 is for the parts of the system that are common to both i386 and amd64
due too their nature. First up, to get the ball rolling, is fdc, the floppy disk
support. It works only on amd64 and i386 these days, and that's unlikely to
change.

Reviewed by: jhb, cem (earlier versrions)
Differential Revision: https://reviews.freebsd.org/D21210

linuxulator: rename linux_locore.s to .asm

It is assembled using "${CC} -x assembler-with-cpp", which by convention
(bsd.suffixes.mk) uses the .asm extension.

This is a portion of the review referenced below (D18344). That review
also renamed linux_support.s to .S, but that is a functional change
(using the compiler's integrated assembler instead of as) and will be
revisited separately.

MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D18344

Remove gzip'ed a.out support.

The current implementation of gzipped a.out support was based
on a very old version of InfoZIP which ships with an ancient
modified version of zlib, and was removed from the GENERIC
kernel in 1999 when we moved to an ELF world.

PR: 205822
Reviewed by: imp, kib, emaste, Yoshihiro Ota <ota at j.email.ne.jp>
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D21099

Add driver for NTB in AMD SoC.

This patch is the driver for NTB hardware in AMD SoCs (ported from Linux)
and enables the NTB infrastructure like Doorbells, Scratchpads and Memory
window in AMD SoC. This driver has been validated using ntb_transport and
if_ntb driver already available in FreeBSD.

Submitted by: Rajesh Kumar <rajesh1.kumar@amd.com>
MFC after: 1 month
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D18774

add superio driver

The goal of this driver is consolidate information about SuperIO chips
and to provide for peaceful coexistence of drivers that need to access
SuperIO configuration registers.

While SuperIO chips can host various functions most of them are
discoverable and accessible without any knowledge of the SuperIO.
Examples are: keyboard and mouse controllers, UARTs, floppy disk
controllers. SuperIO-s also provide non-standard functions such as
GPIO, watchdog timers and hardware monitoring. Such functions do
require drivers with a knowledge of a specific SuperIO.

At this time the driver supports a number of ITE and Nuvoton (fka
Winbond) SuperIO chips.
There is a single driver for all devices. So, I have not done the usual
split between the hardware driver and the bus functionality. Although,
superio does act as a bus for devices that represent known non-standard
functions of a SuperIO chip. The bus provides enumeration of child
devices based on the hardcoded knowledge of such functions. The
knowledge as extracted from datasheets and other drivers.
As there is a single driver, I have not defined a kobj interface for it.
So, its interface is currently made of simple functions.
I think that we can the flexibility (and complications) when we actually
need it.

I am planning to convert nctgpio and wbwd to superio bus very soon.
Also, I am working on itwd driver (watchdog in ITE SuperIO-s).
Additionally, there is ithwm driver based on the reverted sensors
import, but I am not sure how to integrate it given that we still lack
any sensors interface.

Discussed with: imp, jhb
MFC after: 7 weeks
Differential Revision: https://reviews.freebsd.org/D8175

Fix qlxgbe(4) static build.

MFC after: 2 weeks

random(4): Fix RANDOM_LOADABLE build

I introduced an obvious compiler error in r346282, so this change fixes
that.

Unfortunately, RANDOM_LOADABLE isn't covered by our existing tinderbox, and
it seems like there were existing latent linking problems. I believe these
were introduced on accident in r338324 during reduction of the boolean
expression(s) adjacent to randomdev.c and hash.c. It seems the
RANDOM_LOADABLE build breakage has gone unnoticed for nine months.

This change correctly annotates randomdev.c and hash.c with !random_loadable
to match the pre-r338324 logic; and additionally updates the HWRNG drivers
in MD 'files.*', which depend on random_device symbols, with
!random_loadable (it is invalid for the kernel to depend on symbols from a
module).

(The expression for both randomdev.c and hash.c was the same, prior to
r338324: "optional random random_yarrow | random !random_yarrow
!random_loadable". I.e., "random && (yarrow || !loadable)." When Yarrow
was removed ("yarrow := False"), the expression was incorrectly reduced to
"optional random" when it should have retained "random && !loadable".)

Additionally, I discovered that virtio_random was missing a MODULE_DEPEND on
random_device, which breaks kld load/link of the driver on RANDOM_LOADABLE
kernels. Address that issue as well.

PR: 238223
Reported by: Eir Nym <eirnym AT gmail.com>
Reviewed by: delphij, markm
Approved by: secteam(delphij)
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D20466

Add an AESNI-optimized version of the CCM/CBC cryptographic and authentication
code. The primary client of this is probably going to be ZFS encryption.

Reviewed by: jhb, cem
Sponsored by: iXsystems Inc, Kithrup Enterprises
Differential Revision: https://reviews.freebsd.org/D19298

Move scterm_teken.c from 6 MD files lists to the MI files list so that it
is easier to configure. It is MI, unlike some of the other syscons files
already in the MI list.

Move scvtb.c similarly. It is needed whenever sc is configured, and is
more MI than most of the files already in the MI list.

This only changes the combined list for arm64 and mips. These arches
already cannot build sc or even NOTES.

Introduce driver for TPM 2.0 in CRB and FIFO (TIS) modes

It was written basing on:
TCG PC Client Platform TPM Profile (PTP) Specification Version 22, Revision 1.03.
It only supports Locality 0. Interrupts are only supported in FIFO mode.

The driver in FIFO mode was tested on x86 with Infineon SLB9665 discrete TPM chip.
Driver in both modes was also tested on qemu with swtpm running on host.

Submitted by: Kornel Duleba <mindal@semihalf.com>
Obtained from: Semihalf
Sponsored by: Stormshield
Differential Revision: https://reviews.freebsd.org/D18048

acpica : move SRAT/SLIT parsing to sys/dev/acpica

This moves the architecture independent parts of sys/x86/acpica/srat.c
to sys/dev/acpica/acpi_pxm.c, to be used later on arm64. The function
declarations are moved to sys/dev/acpica/acpivar.h

We also need to update sys/conf/files.{i386,amd64} to use the new file.
No functional changes.

Reviewed by: markj, imp
Differential Revision: https://reviews.freebsd.org/D17941

sfxge(4): add firmware image layout option

Submitted by: Andy Moreton <amoreton at solarflare.com>
Sponsored by: Solarflare Communications, Inc.
Differential Revision: https://reviews.freebsd.org/D18214

sfxge(4): add Medford2 support to NIC module

Submitted by: Andy Moreton <amoreton at solarflare.com>
Sponsored by: Solarflare Communications, Inc.
Differential Revision: https://reviews.freebsd.org/D18142

sfxge(4): add API to control UDP tunnel ports

HW needs to know which UDP packets should be treated as tunnel
encapsulation to do inner packet recognition, classification and
offloads.

Submitted by: Ivan Malov <ivan.malov at oktetlabs.ru>
Sponsored by: Solarflare Communications, Inc.
Differential Revision: https://reviews.freebsd.org/D18134

Add pci_early function to detect Intel stolen memory.

On some Intel devices BIOS does not properly reserve memory (called
"stolen memory") for the GPU. If the stolen memory is claimed by the
OS, functions that depend on stolen memory (like frame buffer
compression) can't be used.

A function called pci_early_quirks that is called before the virtual
memory system is started was added. In Linux, this PCI early quirks
function iterates through all PCI slots to check for any device that
require quirks. While this more generic solution is preferable I only
ported the Intel graphics specific parts because I think my
implementation would be too similar to Linux GPL'd solution after
looking at the Linux code too much.

The code regarding Intel graphics stolen memory was ported from
Linux. In the case of Intel graphics stolen memory this
pci_early_quirks will read the stolen memory base and size from north
bridge registers. The values are stored in global variables that is
later read by linuxkpi_gplv2. Linuxkpi stores these values in a
Linux-specific structure that is read by the drm driver.

Relevant linuxkpi code is here:
https://github.com/FreeBSDDesktop/kms-drm/blob/drm-v4.16/linuxkpi/gplv2/src/linux_compat.c#L37

For now, only amd64 arch is suppor ted since that is the only arch
supported by the new drm drivers. I was told that Intel GPUs are
always located on 0:2:0 so these values are hard coded for now.

Note that the structure and early execution of the detection code is
not required in its current form, but we expect that the code will be
added shortly which fixes the potential BIOS bugs by reserving the
stolen range in phys_avail[]. This must be done as early as possible
to avoid conflicts with the potential usage of the memory in kernel.

Submitted by: Johannes Lundberg <johalun0@gmail.com>
Reviewed by: bwidawsk, imp
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D16719
Differential revision: https://reviews.freebsd.org/D17775

Add amdgpio, driver for GPIO controller on AMD-based x86_64 platforms

Submitted by: Rajesh Kumar <rajbsd@gmail.com>
Differential Revision: https://reviews.freebsd.org/D16865

iavf(4): Finish rename/rebrand internally

Rename functions and variables from ixlv to iavf to match the
user-facing name change. There shouldn't be any functional changes
with this change, but this may help with browsing the source code
and reducing diffs in the future.

Submitted by: kbowling@
Reviewed by: erj@, sbruno@
Approved by: re (gjb@)
Differential Revision: https://reviews.freebsd.org/D17544

ixl/iavf(4): Change ixlv to iavf and update it to use iflib(9)

Finishes the conversion of the 40Gb Intel Ethernet drivers to iflib(9) for
FreeBSD 12.0, and fixes numerous bugs in both ixl(4) and iavf(4).

This commit also re-adds the VF driver to GENERIC since it now compiles and
functions.

The VF driver name was changed from ixlv(4) to iavf(4) because the VF driver is
now intended to be used with future products, not just with Fortville/Fort Park
VFs.

A man page update that documents these drivers is forthcoming in a separate
commit.

Reviewed by: sbruno@, kbowling@
Tested by: jeffrey.e.pieper@intel.com
Approved by: re (gjb@)
Relnotes: yes
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D16429

Catch exceptions during EFI RT calls on amd64.

This appeared to be required to have EFI RT support and EFI RTC
enabled by default, because there are too many reports of faulting
calls on many different machines. The knob is added to leave the
exceptions unhandled to allow to debug the actual bugs.

Reviewed by: kevans
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Approved by: re (rgrimes)
Differential revision: https://reviews.freebsd.org/D16972

Implement kernel support for early loading of Intel microcode updates.

Updates in the format described in section 9.11 of the Intel SDM can
now be applied as one of the first steps in booting the kernel. Updates
that are loaded this way are automatically re-applied upon exit from
ACPI sleep states, in contrast with the existing cpucontrol(8)-based
method. For the time being only Intel updates are supported.

Microcode update files are passed to the kernel via loader(8). The
file type must be "cpu_microcode" in order for the file to be recognized
as a candidate microcode update. Updates for multiple CPU types may be
concatenated together into a single file, in which case the kernel
will select and apply a matching update. Memory used to store the
update file will be freed back to the system once the update is applied,
so this approach will not consume more memory than required.

Reviewed by: kib
MFC after: 6 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D16370

Use SMAP on amd64.

Ifuncs selectors dispatch copyin(9) family to the suitable variant, to
set rflags.AC around userspace access. Rflags.AC bit is cleared in
all kernel entry points unconditionally even on machines not
supporting SMAP.

Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D13838

All genassym.sh usage need offset.inc

ensure that offset.inc is generated before ia32_genassym.c is compiled

inline atomics and allow tied modules to inline locks

- inline atomics in modules on i386 and amd64 (they were always
inline on other arches)
- allow modules to opt in to inlining locks by specifying
MODULE_TIED=1 in the makefile

Reviewed by: kib
Sponsored by: Limelight Networks
Differential Revision: https://reviews.freebsd.org/D16079

ixl(4): Remove ixlvc.c from files.amd64

ixlvc.c is only used by the VF driver (ixlv), and that driver currently doesn't work.

It will be re-added once the VF driver is finished being ported to iflib.

ixl(4): Update to use iflib

Update the driver to use iflib in order to bring performance,
maintainability, and (hopefully) stability benefits to the driver.

The driver currently isn't completely ported; features that are missing:

- VF driver (ixlv)
- SR-IOV host support
- RDMA support

The plan is to have these re-added to the driver before the next FreeBSD release.

Reviewed by: gallatin@
Contributions by: gallatin@, mmacy@, krzysztof.galazka@intel.com
Tested by: jeffrey.e.pieper@intel.com
MFC after: 1 month
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D15577

Finish COMPAT_AOUT support for amd64. It wasn't in any amd64 or MI
file in /sys/conf, so was unavailable in configurations that don't use
modules, and was not testable or notable in NOTES. Its normal
configuration (not using a module) is still silently deprecated in
aout(4) by not mentioning it there.

Update i386 NOTES for COMPAT_AOUT. It is not i386-only, or even very MD.
Sort its entry better.

Finish gzip configuration (but not support) for amd64. gzip is really
gzipped aout. It is currently broken even for i386 (a call to vm fails).
amd64 has always attempted to configure and test it, but it depends on
COMPAT_AOUT (as noted). The bug that it depends on unconfigured files
was not detected since it is configured as a device. All other optional
image activators are configured properly using an option.

hwpmc: remove unused pre-table driven bits for intel

Intel now provides comprehensive tables for all performance counters
and the various valid configuration permutations as text .json files.
Libpmc has been converted to use these and hwpmc_core has been greatly
simplified by moving to passthrough of the table values.

The one gotcha is that said tables don't support pentium pro and and pentium
IV. There's very few users of hwpmc on _amd64_ kernels on new hardware. It is
unlikely that anyone is doing low level optimization on 15 year old Intel
hardware. Nonetheless, if someone feels strongly enough to populate the
corresponding tables for p4 and ppro I will reinstate the files in to the
build.

Code for the K8 counters and !x86 architectures remains unchanged.

amd64: replace libkern's memset and memmove with assembly variants

memmove is repurposed bcopy (arguments swapped, return value added)
The libkern variant is a wrapper around bcopy, so this is a big
improvement.

memset is repurposed memcpy. The librkern variant is doing fishy stuff,
including branching on 0 and calling bzero.

Both functions are rather crude and subject to partial depessimization.

This is a soft prerequisite to adding variants utilizing the
'Enhanced REP MOVSB/STOSB' bit and let the kernel patch at runtime.

ixl(4): Update to 1.9.9-k

Refresh upstream driver before impending conversion to iflib.

Major changes:

- Support for descriptor writeback mode (required by ixlv(4) for AVF support)
- Ability to disable firmware LLDP agent by user (PR 221530)
- Fix for TX queue hang when using TSO (PR 221919)
- Separate descriptor ring sizes for TX and RX rings

PR: 221530, 221919
Submitted by: Krzysztof Galazka <krzysztof.galazka@intel.com>
Reviewed by: #IntelNetworking
MFC after: 1 day
Relnotes: Yes
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D14985

martpqi(4):
- Microsemi SCSI driver for PQI controllers.
- Found on newer model HP servers.
- Restrict to AMD64 only as per developer request.

The driver provides support for the new generation of PQI controllers
from Microsemi. This driver is the first SCSI driver to implement the PQI
queuing model and it will replace the aacraid driver for Adaptec Series 9
controllers. HARDWARE Controllers supported by the driver include:

HPE Gen10 Smart Array Controller Family
OEM Controllers based on the Microsemi Chipset.

Submitted by: deepak.ukey@microsemi.com
Relnotes: yes
Sponsored by: Microsemi
Differential Revision: https://reviews.freebsd.org/D14514

Add VMCI (Virtual Machine Communication Interface) driver

In a virtual machine, VMCI is exposed as a regular PCI device. The primary
communication mechanisms supported are a point-to-point bidirectional
transport based on a pair of memory-mapped queues, and asynchronous
notifications in the form of datagrams and doorbells. These features are
available to kernel level components such as vSockets through the VMCI
kernel API. In addition to this, the VMCI kernel API provides support for
receiving events related to the state of the VMCI communication channels,
and the virtual machine itself.

Submitted by: Vishnu Dasa <vdasa@vmware.com>
Reviewed by: bcr, imp
Obtained from: VMware
Differential Revision: https://reviews.freebsd.org/D14289

Rename assym.s to assym.inc

assym is only to be included by other .s files, and should never
actually be assembled by itself.

Reviewed by: imp, bdrewery (earlier)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D14180

Share a single bsd-linux errno table across MD consumers

Three copies of the linuxulator linux_sysvec.c contained identical
BSD to Linux errno translation tables, and future work to support other
architectures will also use the same table. Move the table to a common
file to be used by all. Make it 'const int' to place it in .rodata.

(Some existing Linux architectures use MD errno values, but x86 and Arm
share the generic set.)

This change should introduce no functional change; a followup will add
missing errno values.

MFC after: 3 weeks
Sponsored by: Turing Robotic Industries Inc.
Differential Revision: https://reviews.freebsd.org/D14665

imcsmb(4): Intel integrated Memory Controller (iMC) SMBus controller driver

imcsmb(4) provides smbus(4) support for the SMBus controller functionality
in the integrated Memory Controllers (iMCs) embedded in Intel Sandybridge-
Xeon, Ivybridge-Xeon, Haswell-Xeon, and Broadwell-Xeon CPUs. Each CPU
implements one or more iMCs, depending on the number of cores; each iMC
implements two SMBus controllers (iMC-SMBs).

*** IMPORTANT NOTE ***
Because motherboard firmware or the BMC might try to use the iMC-SMBs for
monitoring DIMM temperatures and/or managing an NVDIMM, the driver might
need to temporarily disable those functions, or take a hardware interlock,
before using the iMC-SMBs. Details on how to do this may vary from board to
board, and the procedure may be proprietary. It is strongly suggested that
anyone wishing to use this driver contact their motherboard vendor, and
modify the driver as described in the manual page and in the driver itself.
(For what it's worth, the driver as-is has been tested on various SuperMicro
motherboards.)

Reviewed by: avg, jhb
MFC after: 1 week
Relnotes: yes
Sponsored by: Panasas
Differential Revision: https://reviews.freebsd.org/D14447
Discussed with: avg, ian, jhb
Tested by: allanjude (previous version), Panasas

[chvgpio] add GPIO driver for Intel Z8xxx SoC family

Add chvgpio(4) driver for Intel Z8xxx SoC family. This product
was formerly known as Cherry Trail but Linux and OpenBSD drivers
refer to it as Cherry View. This driver is derived from OpenBSD
one so the name is kept for alignment with another BSD system.

Submitted by: Tom Jones <tj@enoti.me>
Reviewed by: gonzo, wblock(man page)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D13086

Remove some duplicated sys/conf/files* entries.

net80211/ieee80211_ageq.c was present twice in sys/conf/files so leave the
correctly sorted one. dev/wpi/if_wpi.c was present in sys/conf/files as well
as sys/conf/files.amd64 and sys/conf/files.i386 so prefer the sys/conf/files
entry.

Reviewed by: allanjude, rstone

To prepare for adding EFI runtime services support on arm64 move the
machine independent parts of the existing code to a new file that can be
shared between amd64 and arm64.

Reviewed by: kib (previous version), imp
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D12434

aesni(4): Add support for x86 SHA intrinsics

Some x86 class CPUs have accelerated intrinsics for SHA1 and SHA256.
Provide this functionality on CPUs that support it.

This implements CRYPTO_SHA1, CRYPTO_SHA1_HMAC, and CRYPTO_SHA2_256_HMAC.

Correctness: The cryptotest.py suite in tests/sys/opencrypto has been
enhanced to verify SHA1 and SHA256 HMAC using standard NIST test vectors.
The test passes on this driver. Additionally, jhb's cryptocheck tool has
been used to compare various random inputs against OpenSSL. This test also
passes.

Rough performance averages on AMD Ryzen 1950X (4kB buffer):
aesni: SHA1: ~8300 Mb/s SHA256: ~8000 Mb/s
cryptosoft: ~1800 Mb/s SHA256: ~1800 Mb/s

So ~4.4-4.6x speedup depending on algorithm choice. This is consistent with
the results the Linux folks saw for 4kB buffers.

The driver borrows SHA update code from sys/crypto sha1 and sha256. The
intrinsic step function comes from Intel under a 3-clause BSDL.[0] The
intel_sha_extensions_sha<foo>_intrinsic.c files were renamed and lightly
modified (added const, resolved a warning or two; included the sha_sse
header to declare the functions).

[0]: https://software.intel.com/en-us/articles/intel-sha-extensions-implementations

Reviewed by: jhb
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12452

Leave the Cavium Liquid IO driver exist in files, not files.amd64

Submitted by: imp

The diff is the initial submission of Cavium Liquidio 2350/2360 10/25G
Intelligent NIC driver.

The submission conconsists of firmware binary file and driver sources.

Submitted by: pkanneganti@cavium.com (Prasad V Kanneganti)
Relnotes: Yes
Sponsored by: Cavium Networks
Differential Revision: https://reviews.freebsd.org/D11927

amdtemp(4): Add support for Family 17h temperature sensor

The sensor value is formatted similarly to previous models (same
bitfield sizes, same units), but must be read off of the internal
System Management Network (SMN) from the System Management Unit (SMU)
co-processor.

PR: 218264
Reported and tested by: Nils Beyer <nbe AT renzel.net>
Reviewed by: avg (no +1), mjoras, truckman
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12217

Add smn(4) driver for AMD System Management Network

AMD Family 17h CPUs have an internal network used to communicate between
the host CPU and the PSP and SMU coprocessors. It exposes a simple
32-bit register space.

Reviewed by: avg (no +1), mjoras, truckman
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12217

Add NTB driver for PLX/Avago/Broadcom PCIe switches.

This driver supports both NTB-to-NTB and NTB-to-Root Port modes (though
the second with predictable complications on hot-plug and reboot events).
I tested it with PEX 8717 and PEX 8733 chips, but expect it should work
with many other compatible ones too. It supports up to two NT bridges
per chip, each of which can have up to 2 64-bit or 4 32-bit memory windows,
6 or 12 scratchpad registers and 16 doorbells. There are also 4 DMA engines
in those chips, but they are not yet supported.

While there, rename Intel NTB driver from generic ntb_hw(4) to more specific
ntb_hw_intel(4), so now it is on par with this new ntb_hw_plx(4) driver and
alike to Linux naming.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

Make nvd vs nda choice boot-time rather than build-time

Introduce hw.nvme.use_nvd tunable. This tunable allows both nvd and
nda to be installed in the kernel, while allowing only one of them to
create devices. This is an all-or-nothing setting, and you can't
change it after boot-time. However, it will allow easier A/B testing.

Differential Revision: https://reviews.freebsd.org/D11825

Fix kldload of if_ixl without PCI_IOV kernel option.

This also avoids compiling in pci_iov support into the kernel if_ixoif
the PCI_IOV option is disabled.

Reviewed by: rstone
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D11573

linux vdso: pass -fPIC to the assembler, not linker

-fPIC has no effect on linking although it seems to be ignored by
GNU ld.bfd. However, it causes ld.lld to terminate with an invalid
argument error.

This is equivalent to r296057 but for the kernel (not modules) case.

MFC after: 2 months
Sponsored by: The FreeBSD Foundation

hyperv/vmbus: Reorganize vmbus device tree

For GEN1 Hyper-V, vmbus is attached to pcib0, which contains the
resources for PCI passthrough and SR-IOV. There is no
acpi_syscontainer0 on GEN1 Hyper-V.

For GEN2 Hyper-V, vmbus is attached to acpi_syscontainer0, which
contains the resources for PCI passthrough and SR-IOV. There is
no pcib0 on GEN2 Hyper-V.

The ACPI VMBUS device now only holds its _CRS, which is empty as
of this commit; its existence is mainly for upward compatibility.

Device tree structure is suggested by jhb@.

Tested-by: dexuan@
Collabrated-wth: dexuan@
MFC after: 1 week
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D10565

Cleanup the bitmap_xxx() functions in the LinuxKPI:

- Move all bitmap related functions from bitops.h to bitmap.h, similar
to what Linux does.

- Apply some minor code cleanup and simplifications to optimize the
generated code when using static inline functions.

- Implement the following list of bitmap functions which are needed by
drm-next and ibcore:
- bitmap_find_next_zero_area_off()
- bitmap_find_next_zero_area()
- bitmap_or()
- bitmap_and()
- bitmap_xor()

- Add missing include directives to the qlnxe driver
(davidcs@ has been notified)

MFC after: 1 week
Sponsored by: Mellanox Technologies

hyperv/kbd: Add support for synthetic keyboard.

Synthetic keyboard is the only supported keyboard on GEN2 Hyper-V.

Submitted by: Hongjiang Zhang <honzhan microsoft com>
MFC after: 1 week
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D10196

Add 25/40/100Gigabit Ethernet Driver version v1.3.0 for Cavium Inc's.
Qlogic 45000 Series Adapters

MFC after:2 weeks

Add nctgpio conf lines so it can be compiled into the kernel.

MFC after: 2 days

Use inline asm instead of unportable intrinsics for the SSE4 crc32
optimization.

This fixes building with gcc-4.2.1 (it doesn't support SSE4).
gas-2.17.50 [FreeBSD] supports SSE4 instructions, so this doesn't
need using .byte directives.

This fixes depending on host user headers in the kernel.

Fix user includes (don't depend on namespace pollution in <nmmintrin.h>
that is not included now).

The instrinsics had no advantages except to sometimes avoid compiler
pessimixations. clang understands them a bit better than inline asm,
and generates better looking code which also runs better for cem, but
for me it just at the same speed or slower by doing excessive
unrollowing in all the wrong places. gcc-4.2.1 also doesn't understand
what it is doing with unrolling, but with -O3 somehow it does more
unrolling that helps.

Reduce 1 of the the compiler pessimizations (copying a variable which
already satisfies an "rm" constraint in a good way by being in memory
and not used again, to different memory and accessing it there. Force
copying it to a register instead).

Try to optimize the inner loops significantly, so as to run at full
speed on smaller inputs. The algorithm is already very MD, and was
tuned for the throughput of 3 crc32 instructions per cycle found on
at least Sandybridge through Haswell. Now it is even more tuned for
this, so depends more on the compiler not rearranging or unrolling
things too much. The main inner loop for should have no difficulty
runing at full speed on these CPUs unless the compiler unrolls it too
much. However, the main inner loop wasn't even used for buffers smaller
than 24K. Now it is used for buffers larger than 384 bytes. Now it
is not so long, and the main outer loop is used more. The new
optimization is to try to arrange that the outer loop runs in parallel
with the next inner loop except for the final iteration; then reduce
the loop sizes significantly to take advantage of this.

Approved by: cem
Not tested in production by: bde

Exclude -flto when building *genassym.o

The build process generates *assym.h using nm from *genassym.o (which is
in turn created from *genassym.c).

When compiling with link-time optimization (LTO) using -flto, .o files
are LLVM bitcode, not ELF objects. This is not usable by genassym.sh,
so remove -flto from those ${CC} invocations.

Submitted by: George Rimar
Reviewed by: dim
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D9659

Merge i386 and amd64 mtrr drivers.

Reviewed by: royger, jhb
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D9648

ixl(4): Update to 1.7.12-k

Refresh upstream driver before impending conversion to iflib.

Major new features:

- Support for Fortville-based 25G adapters
- Support for I2C reads/writes

(To prevent getting or sending corrupt data, you should set
dev.ixl.0.debug.disable_fw_link_management=1 when using I2C
[this will disable link!], then set it to 0 when done. The driver implements
the SIOCGI2C ioctl, so ifconfig -v works for reading I2C data,
but there are read_i2c and write_i2c sysctls under the .debug sysctl tree
[the latter being useful for upper page support in QSFP+]).

- Addition of an iWARP client interface (so the future iWARP driver for
X722 devices can communicate with the base driver).
- Compiling this option in is enabled by default, with "options IXL_IW" in
GENERIC.

Differential Revision: https://reviews.freebsd.org/D9227
Reviewed by: sbruno
MFC after: 2 weeks
Sponsored by: Intel Corporation

Only build the ACPI PCI drivers on x86, they are unlikely to be used on
arm64 without dignificant changes.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation

Merge projects/ipsec into head/.

Small summary
-------------

o Almost all IPsec releated code was moved into sys/netipsec.
o New kernel modules added: ipsec.ko and tcpmd5.ko. New kernel
option IPSEC_SUPPORT added. It enables support for loading
and unloading of ipsec.ko and tcpmd5.ko kernel modules.
o IPSEC_NAT_T option was removed. Now NAT-T support is enabled by
default. The UDP_ENCAP_ESPINUDP_NON_IKE encapsulation type
support was removed. Added TCP/UDP checksum handling for
inbound packets that were decapsulated by transport mode SAs.
setkey(8) modified to show run-time NAT-T configuration of SA.
o New network pseudo interface if_ipsec(4) added. For now it is
build as part of ipsec.ko module (or with IPSEC kernel).
It implements IPsec virtual tunnels to create route-based VPNs.
o The network stack now invokes IPsec functions using special
methods. The only one header file <netipsec/ipsec_support.h>
should be included to declare all the needed things to work
with IPsec.
o All IPsec protocols handlers (ESP/AH/IPCOMP protosw) were removed.
Now these protocols are handled directly via IPsec methods.
o TCP_SIGNATURE support was reworked to be more close to RFC.
o PF_KEY SADB was reworked:
- now all security associations stored in the single SPI namespace,
and all SAs MUST have unique SPI.
- several hash tables added to speed up lookups in SADB.
- SADB now uses rmlock to protect access, and concurrent threads
can do SA lookups in the same time.
- many PF_KEY message handlers were reworked to reflect changes
in SADB.
- SADB_UPDATE message was extended to support new PF_KEY headers:
SADB_X_EXT_NEW_ADDRESS_SRC and SADB_X_EXT_NEW_ADDRESS_DST. They
can be used by IKE daemon to change SA addresses.
o ipsecrequest and secpolicy structures were cardinally changed to
avoid locking protection for ipsecrequest. Now we support
only limited number (4) of bundled SAs, but they are supported
for both INET and INET6.
o INPCB security policy cache was introduced. Each PCB now caches
used security policies to avoid SP lookup for each packet.
o For inbound security policies added the mode, when the kernel does
check for full history of applied IPsec transforms.
o References counting rules for security policies and security
associations were changed. The proper SA locking added into xform
code.
o xform code was also changed. Now it is possible to unregister xforms.
tdb_xxx structures were changed and renamed to reflect changes in
SADB/SPDB, and changed rules for locking and refcounting.

Reviewed by: gnn, wblock
Obtained from: Yandex LLC
Relnotes: yes
Sponsored by: Yandex LLC
Differential Revision: https://reviews.freebsd.org/D9352

calculate_crc32c: Add SSE4.2 implementation on x86

Derived from an implementation by Mark Adler.

The fast loop performs three simultaneous CRCs over subsets of the data
before composing them. This takes advantage of certain properties of
the CRC32 implementation in Intel hardware. (The CRC instruction takes 1
cycle but has 2-3 cycles of latency.)

The CRC32 instruction does not manipulate FPU state.

i386 does not have the crc32q instruction, so avoid it there. Otherwise
the implementation is identical to amd64.

Add basic userland tests to verify correctness on a variety of inputs.

PR: 216467
Reported by: Ben RUBSON <ben.rubson at gmail.com>
Reviewed by: kib@, markj@ (earlier version)
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D9342

Remove pc98 support completely.
I thank all developers and contributors for pc98.

Relnotes: yes

sfxge(4): remove obsolete Wake-On-LAN support

Wake-on-lan is not supported in production on any of our adapters, as
they don't have the required AUX power connector. (It's possible that
AUX power is supplied to some of our ALOM or mezz adapters, but if so
then we've never implemented or tested WoL support.)

Reviewed by: gnn
Sponsored by: Solarflare Communications, Inc.
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D8972

sfxge(4): delete hunt_phy.c

Submitted by: Mark Spender <mspender at solarflare.com>
Sponsored by: Solarflare Communications, Inc.
MFC after: 2 days

[intelspi] Add SPI driver for Intel BayTrail SoC

Add SPI mode (PIO-only) support for Intel Synchronous Serial Port that
can be found in several Intel's products starting from PXA family.
Most of implementations have slight differences in behavior and in
addresses for registers subset. This driver covers only BayTrail SoC
implementation for it's the only hardware I have to test it on.

Driver attaches to ACPI bus only and does not have PCI or FDT support
for now due to lack of hardware to test it on.

"intelspi" is the best name I've managed to come up with. Linux driver
name (spi-pxa2xx) does not make sense because current implementation
does not support actual PXA2xx SoCs. And as far as I know there is no
codename assigned to Intel SSP chip.

Reviewed by: br, manu
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D8896

hyperv/ic: Rename cleaned up files.

MFC after: 1 week
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8850

hyperv/ic: Rname cleaned up file.

MFC after: 1 week
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8848

Only build acpi_timer.c on x86, it fails on arm64 as it attempts to access
an invalid address. It is also unneeded on arm64 as we use the ARM Generic
Timer driver.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation

hyperv/pcib: change the file name: pcib.c -> vmbus_pcib.c

This makes the file name and the variable naming in the file consistent.

Reviewed by: sephe
Approved by: sephe (mentor)
MFC after: 1 week
Sponsored by: Microsoft

hyperv/pcib: Fix the build for some kernel configs

Add the dependency on pci explicitly for the pcib and vmbus drivers.
The related Makefiles are updated accordingly too.

Reviewed by: sephe
Approved by: sephe (mentor)
MFC after: 1 week
Sponsored by: Microsoft

hyperv/pcib: enable PCIe pass-through (a.k.a. Discrete Device Assignment)

The feature enables us to pass through physical PCIe devices to FreeBSD VM
running on Hyper-V (Windows Server 2016) to get near-native performance with
low CPU utilization.

The patch implements a PCI bridge driver to support the feature:

1) The pcib driver talks to the host to discover device(s) and presents
the device(s) to FreeBSD's pci driver via PCI configuration space (note:
to access the configuration space, we don't use the standard I/O port
0xCF8/CFC method; instead, we use an MMIO-based method supplied by Hyper-V,
which is very similar to the 0xCF8/CFC method).

2) The pcib driver allocates resources for the device(s) and initialize
the related BARs, when the device driver's attach method is invoked;

3) The pcib driver talks to the host to create MSI/MSI-X interrupt
remapping between the guest and the host;

4) The pcib driver supports device hot add/remove.

Reviewed by: sephe
Approved by: sephe (mentor)
MFC after: 1 week
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8332

hyperv/vss: Add driver and tools for VSS

VSS stands for "Volume Shadow Copy Service". Unlike virtual machine
snapshot, it only takes snapshot for the virtual disks, so both
filesystem and applications have to aware of it, and cooperate the
whole VSS process.

This driver exposes two device files to the userland:

/dev/hv_fsvss_dev

Normally userland programs should _not_ mess with this device file.
It is currently used by the hv_vss_daemon(8), which freezes and
thaws the filesystem. NOTE: currently only UFS is supported, if
the system mounts _any_ other filesystems, the hv_vss_daemon(8)
will veto the VSS process.

If hv_vss_daemon(8) was disabled, then this device file must be
opened, and proper ioctls must be issued to keep the VSS working.

/dev/hv_appvss_dev

Userland application can opened this device file to receive the
VSS freeze notification, hold the VSS for a while (mainly to flush
application data to filesystem), release the VSS process, and
receive the VSS thaw notification i.e. applications can run again.

The VSS will still work, even if this device file is not opened.
However, only filesystem consistency is promised, if this device
file is not opened or is not operated properly.

hv_vss_daemon(8) is started by devd(8) by default. It can be disabled
by editting /etc/devd/hyperv.conf.

Submitted by: Hongjiang Zhang <honzhan microsoft com>
Reviewed by: kib, mckusick
MFC after: 3 weeks
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8224

[gpio] Add GPIO driver for Intel Bay Trail SoC

Bay Trail has three banks of GPIOs exposed to userland as /dev/gpiocN,
where N is 1, 2, and 3. Pins in each bank are pre-named to match names
on boards schematics: GPIO_S0_SCnn, GPIO_S0_NCnn, and GPIO_S5_nn.

Controller supports edge-triggered and level-triggered interrupts but
current version of the driver does not have interrupts support

hyperv/hn: Rename cleaned up file.

MFC after: 1 week
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8390

hyperv/hn: Rename cleaned up RNDIS source file.

MFC after: 1 week
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8361

hyperv/hn: Rename cleaned up NVS source file.

MFC after: 1 week
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8354

hyperv/storvsc: Fix the blkvsc disk attachment issues.

- The original 'disengage' ATA controller model does not work properly
for all possible disk configurations. Use the newly added ATA disk
veto eventhandler to fit into all possible disk configuration.
- If the 'invalid LUN' happens on blkvsc controllers, return
CAM_DEV_NOT_THERE so that CAM will not destroy attached disks under
the blkvsc controllers.

Submitted by: Hongjiang Zhang <honzhan microsoft com>
Discussed with: mav
MFC after: 1 week
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7693

Add kernel interfaces to call EFI Runtime Services.

Runtime services require special execution environment for the call.
Besides that, OS must inform firmware about runtime virtual memory map
which will be active during the calls, with the SetVirtualAddressMap()
runtime call, done while the 1:1 mapping is still used. There are two
complication: the SetVirtualAddressMap() effectively must be done from
loader, which needs to know kernel address map in advance. More,
despite not explicitely mentioned in the specification, both 1:1 and
the map passed to SetVirtualAddressMap() must be active during the
SetVirtualAddressMap() call. Second, there are buggy BIOSes which
require both mappings active during runtime calls as well, most likely
because they fail to identify all relocations to perform.

On amd64, we can get rid of both problems by providing 1:1 mapping for
the duration of runtime calls, by temprorary remapping user addresses.
As result, we avoid the need for loader to know about future kernel
address map, and avoid bugs in BIOSes. Typically BIOS only maps
something in low 4G. If not runtime bugs, we would take advantage of
the DMAP, as previous versions of this patch did.

Similar but more complicated trick can be used even for i386 and 32bit
runtime, if and when the EFI boot on i386 is supported. We would need
a trampoline page, since potentially whole 4G of VA would be switched
on calls, instead of only userspace portion on amd64.

Context switches are disabled for the duration of the call, FPU access
is granted, and interrupts are not disabled. The later is possible
because kernel is mapped during calls.

To test, the sysctl mib debug.efi_time is provided, setting it to 1
makes one call to EFI get_time() runtime service, on success the efitm
structure is printed to the control terminal. Load efirt.ko, or add
EFIRT option to the kernel config, to enable code.

Discussed with: emaste, imp
Tested by: emaste (mac, qemu)
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks

Make execution of 32-bit CloudABI executables work on amd64.

A nice thing about requiring a vDSO is that it makes it incredibly easy
to provide full support for running 32-bit processes on 64-bit systems.
Instead of letting the kernel be responsible for composing/decomposing
64-bit arguments across multiple registers/stack slots, all of this can
now be done in the vDSO. This means that there is no need to provide
duplicate copies of certain system calls, like the sys_lseek() and
freebsd32_lseek() we have for COMPAT_FREEBSD32.

This change imports a new vDSO from the CloudABI repository that has
automatically generated code in it that copies system call arguments
into a buffer, padding them to eight bytes and zero-extending any
pointers/size_t arguments. After returning from the kernel, it does the
inverse: extracting return values, in the process truncating
pointers/size_t values to 32 bits.

Obtained from: https://github.com/NuxiNL/cloudabi

Move the linker script from cloudabi64/ to cloudabi/.

It turns out that it works perfectly fine for generating 32-bits vDSOs
as well. While there, get rid of the extraneous .s file extension.

Rewrite the vDSOs for CloudABI in assembly.

The reason why the old vDSOs were written in C using inline assembly was
purely because they were embedded in the C library directly as static
inline functions. This was practical during development, because it
meant you could invoke system calls without any library dependencies.
The vDSO was simply a copy of these functions.

Now that we require the use of the vDSO, there is no longer any need for
embedding them in C code directly. Rewriting them in assembly has the
advantage that they are closer to ideal (less useless branching, less
assumptions about registers remaining unclobbered by the kernel, etc).
They are also easier to build, as they no longer depend on the C type
information for CloudABI.

Obtained from: https://github.com/NuxiNL/cloudabi

hyperv/vmbus: Add APIs for various types of transactions.

Reviewed by: Jun Su <junsu microsoft com>
MFC after: 1 week
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7456

Provide the CloudABI vDSO to its executables.

CloudABI executables already provide support for passing in vDSOs. This
functionality is used by the emulator for OS X to inject system call
handlers. On FreeBSD, we could use it to optimize calls to
gettimeofday(), etc.

Though I don't have any plans to optimize any system calls right now,
let's go ahead and already pass in a vDSO. This will allow us to
simplify the executables, as the traditional "syscall" shims can be
removed entirely. It also means that we gain more flexibility with
regards to adding and removing system calls.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D7438

ixl(4): Update to ixl-1.6.6-k.

Submitted by: erj
Reviewed by: jeffrey.e.pieper@intel.com
MFC after: 3 days
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D7391

Merge i386 and amd64 variants of mp_watchdog.c into x86/, there is no
difference between files.
For pc98, put x86/mp_x86.c into the same place as used by i386 file list.
Fix typo in comment.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

hyperv/vmbus: Rename cleaned up bufring code

MFC after: 1 week
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7318

Merge {amd64,i386}/instr_size.c into x86_instr_size.c.

Also reduce the diff between us and upstream: the input data model will
always be DATAMODEL_NATIVE because of a bug (p_model is never set but is
always initialized to 0), so we don't need to override the caller anyway.
This change is also necessary to support the pid provider for 32-bit
processes on amd64.

MFC after: 2 weeks

hyperv/vmbus: Rename laundered vmbus channel code

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7232

hyperv/vmbus: Merge hv_channel_mgmt.c into hv_channel.c

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7126

hyperv/vmbus: Merge hv_connection.c into hv_channel.c

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7004

hyperv/vmbus: Rework vmbus version accessing.

Instead of global variable, vmbus version is accessed through
a vmbus DEVMETHOD now.

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6953

Add linux_mmap.c to the appropriate conf/files.

Reported by: kib@
MFC after: 1 week

NewBus'ify NTB subsystem.

This follows NTB subsystem modularization in Linux, tuning it to FreeBSD
native NewBus interfaces. This change allows to support different types
of hardware with different drivers, support multiple NTB instances in a
system, ntb_transport module use for needs other then if_ntb, etc.

Sponsored by: iXsystems, Inc.

New NVMe front end (nda).

hyperv: Move machine dependent bits into machine dependent files.

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6701

hyperv: Rename some cleaned up/almost cleaned up files

MFC after: 1 week
Sponsored by: Microsoft OSTC

hyperv: Add helpers for busdma(9) operation

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6443

Update file list for sfgxe(4) again and hey, my amd64 kernels compile again.

Remove more files from sfxge(4) which are no longer in the tree
in order to make the universe a more happy place.

hunt_ev.c was deleted with r299596. Remove it from the files list to
(hopefully) unbreak amd64 LINT kernels.

MFC after: 1 week

hyperv: Deprecate HYPERV option by moving Hyper-V IDT vector into vmbus

Submitted by: Jun Su <junsu microsoft com>
Reviewed by: jhb, kib, sephe
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D5910

Move i386/i386/autoconf.c to sys/x86/x86 and use it on both amd64 and i386.

simplify compile-time default keyboard map generation

In r296926 the -P <path> option was added to kbdcontrol, which enables
this change for a simplified compile-time default keymap build process.

PR: 193865
Reviewed by: Oliver Pinter
Tested by: Oliver Pinter
MFC After: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D5708

Fix atkbdmap.h generation for sc/vt consoles

Keymap header files have historically been generated using the build
host's /usr/sbin/kbdcontrol and using the host's keymap files.

However, that introduces an issue when building a kernel to use vt(4)
on a system using sc(4), or vice versa: kbdcontrol searches for keymap
files in the /usr/share subdirectory appropriate for the host, not the
target.

With this change the build searches both the and sc keymap directories
from the source tree.

PR: 193865
Submitted by: Harald Schmalzbauer

hyperv/utils: Code rearrange and cleanup

Split heartbeat, shutdown and timesync out of utils code
and name them properly.

Submitted by: Jun Su <junsu microsoft com>
Reviewed by: adrian, sephe, Hongjiang Zhang <honzhan microsoft com>
MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D5216

Remove fdt_fixup_table from architectures where it's unneeded. We only make
use of fdt_fixup_table on PowerPC and ARM. As such we can remove it from
other architectures as it's unneeded.

Reviewed by: nwhitehorn
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D5013

sfxge: add accessors for license-related MCDI calls to common code

Add support for Huntington MCDI licensing interface to common code.
Ported from Linux net driver IOCTL functions with restructuring for
initial support for V3 licensing API.

Submitted by: Richard Houldsworth <rhouldsworth at solarflare.com>
Reviewed by: gnn
Sponsored by: Solarflare Communications, Inc.
MFC after: 2 days
Differential Revision: https://reviews.freebsd.org/D4918

hyperv: implement an event timer

Submitted by: Howard Su <howard0su@gmail.com>
Reviewed by: delphij, royger, adrian
Approved by: adrian (mentor)
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D4676

Move kbd.c to main sys/conf/files list

It is (optionally) used on all architectures.

Sponsored by: The FreeBSD Foundation

Merge stack(9) implementations for i386 and amd64 under x86/.

Reviewed by: jhb, kib
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D3255

Import ioat(4) driver

I/OAT is also referred to as Crystal Beach DMA and is a Platform Storage
Extension (PSE) on some Intel server platforms.

This driver currently supports DMA descriptors only and is part of a
larger effort to upstream an interconnect between multiple systems using
the Non-Transparent Bridge (NTB) PSE.

For now, this driver is only built on AMD64 platforms. It may be ported
to work on i386 later, if that is desired. The hardware is exclusive to
x86.

Further documentation on ioat(4), including API documentation and usage,
can be found in the new manual page.

Bring in a test tool, ioatcontrol(8), in tools/tools/ioat. The test
tool is not hooked up to the build and is not intended for end users.

Submitted by: jimharris, Carl Delsey <carl.r.delsey@intel.com>
Reviewed by: jimharris (reviewed my changes)
Approved by: markj (mentor)
Relnotes: yes
Sponsored by: Intel
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D3456

sys/conf: pass NMFLAGS to nm(1) via genassym.sh.

Better support memory mapped console devices, such as VGA and EFI
frame buffers and memory mapped UARTs.

1. Delay calling cninit() until after pmap_bootstrap(). This makes
sure we have PMAP initialized enough to add translations. Keep
kdb_init() after cninit() so that we have console when we need
to break into the debugger on boot.
2. Unfortunately, the ATPIC code had be moved as well so as to
avoid a spurious trap #30. The reason for which is not known
at this time.
3. In pmap_mapdev_attr(), when we need to map a device prior to the
VM system being initialized, use virtual_avail as the KVA to map
the device at. In particular, avoid using the direct map on amd64
because we can't demote by virtue of not being able to allocate
yet. Keep track of the translation.
Re-use the translation after the VM has been initialized to not
waste KVA and to satisfy the assumption in uart(4) that the handle
returned for the low-level console is the same as later returned
when the device is probed and attached.
4. In pmap_unmapdev() remove the mapping from the table when called
pre-init. Otherwise keep the mapping. During bus probe and attach
device resources are mapped and unmapped multiple times, which
would have us destroy the mapping used by the low-level console.
5. In pmap_init(), set pmap_initialized to signal that we're not
pre-init anymore. On amd64, bring the direct map in sync with the
translations created at that time.
6. Implement bus_space_map() and bus_space_unmap() for real: when
the tag corresponds to memory space, call the corresponding
pmap_mapdev() and pmap_unmapdev() functions to construct and
actual handle.
7. In efifb.c and vt_vga.c, remove the crutches and hacks and simply
call pmap_mapdev_attr() or bus_space_map() as desired.

Notes:
1. uart(4) already used bus_space_map() during low-level console
setup but since serial ports have traditionally been I/O port
based, the lack of a proper implementation for said function
was not a problem. It has always supported memory mapped UARTs
for low-level consoles by setting hw.uart.console accordingly.
2. The use of the direct map on amd64 without setting caching
attributes has been a bigger problem than previously thought.
This change has the fortunate (and unexpected) side-effect of
fixing various EFI frame buffer problems (though not all).

PR: 191564, 194952

Special thanks to:
1. XipLink, Inc -- generously donated an Intel Bay Trail E3800
based eval board (ADLE3800PC).
2. The FreeBSD Foundation, in particular emaste@ -- for UEFI
support in general and testing.
3. Everyone who tested the proposed for PR 191564.
4. jhb@ and kib@ for being a soundboard and applying a clue bat
if so needed.

files.amd64: Build ntb_hw.o if if_ntb OR ntb_hw

Approved by: markj (mentor)
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D3349

Add a sysentvec for CloudABI on x86-64.

Summary:
For CloudABI we need to put two things on the stack of new processes:
the argument data (a binary blob; not strings) and a startup data
structure. The startup data structure contains interesting things such
as a pointer to the ELF program header, the thread ID of the initial
thread, a stack smashing protection canary, and a pointer to the
argument data.

Fetching system call arguments and setting the return value is similar
to FreeBSD. The only differences are that system call 0 does not exist
and that we call into cloudabi_convert_errno() to convert the error
code. We also need this function in a couple of other places, so we'd
better reuse it here.

Reviewers: dchagin, kib

Reviewed By: kib

Subscribers: imp

Differential Revision: https://reviews.freebsd.org/D3098

Allow DTrace to be compiled-in to the kernel.
This will require for AArch64 as we dont have modules yet.

Sponsored by: HEIF5
Sponsored by: ARM Ltd.
Differential Revision: https://reviews.freebsd.org/D1997

sfxge: add 7xxx NICs family support

Support 7xxx adapters including firmware-assisted TSO and VLAN tagging:

- Solarflare Flareon Ultra 7000 series 10/40G adapters:
- Solarflare SFN7042Q QSFP+ Server Adapter
- Solarflare SFN7142Q QSFP+ Server Adapter

- Solarflare Flareon Ultra 7000 series 10G adapters:
- Solarflare SFN7022F SFP+ Server Adapter
- Solarflare SFN7122F SFP+ Server Adapter
- Solarflare SFN7322F Precision Time Synchronization Server Adapter

- Solarflare Flareon 7000 series 10G adapters:
- Solarflare SFN7002F SFP+ Server Adapter

Support utilities to configure adapters and update firmware.

The work is done by Solarflare developers
(Andy Moreton, Andrew Lee and many others),
Artem V. Andreev <Artem.Andreev at oktetlabs.ru> and me.

Sponsored by: Solarflare Communications, Inc.
MFC after: 2 weeks
Causually read by: gnn
Differential Revision: https://reviews.freebsd.org/D2618

For objcopy, use --input-target and --output-target

When building with gcc 4.9 and binutils 2.25,
using '--input' and '--output' returns an error
message:
objcopy: option `--input' is ambiguous

Reported by: Jenkins

Rework signal code to allow using it by other modules, like linprocfs:

1. Linux sigset always 64 bit on all platforms. In order to move Linux
sigset code to the linux_common module define it as 64 bit int. Move
Linux sigset manipulation routines to the MI path.

2. Move Linux signal number definitions to the MI path. In general, they
are the same on all platforms except for a few signals.

3. Map Linux RT signals to the FreeBSD RT signals and hide signal conversion
tables to avoid conversion errors.

4. Emulate Linux SIGPWR signal via FreeBSD SIGRTMIN signal which is outside
of allowed on Linux signal numbers.

PR: 197216

Implement epoll family system calls. This is a tiny wrapper
around kqueue() to implement epoll subset of functionality.
The kqueue user data are 32bit on i386 which is not enough for
epoll user data, so we keep user data in the proc emuldata.

Initial patch developed by rdivacky@ in 2007, then extended
by Yuri Victorovich @ r255672 and finished by me
in collaboration with mjg@ and jillies@.

Differential Revision: https://reviews.freebsd.org/D1092

Introduce a new module linux_common.ko which is intended for the
following primary purposes:

1. Remove the dependency of linsysfs and linprocfs modules from linux.ko,
which will be architecture specific on amd64.

2. Incorporate into linux_common.ko general code for platforms on which
we'll support two Linuxulator modules (for both instruction set - 32 & 64 bit).

3. Move malloc(9) declaration to linux_common.ko, to enable getting memory
usage statistics properly.

Currently linux_common.ko incorporates a code from linux_mib.c and linux_util.c
and linprocfs, linsysfs and linux kernel modules depend on linux_common.ko.

Temporarily remove dtrace garbage from linux_mib.c and linux_util.c

Differential Revision: https://reviews.freebsd.org/D1072
In collaboration with: Vassilis Laganakos.

Reviewed by: trasz

Implement vdso - virtual dynamic shared object. Through vdso Linux
exposes functions from kernel with proper DWARF CFI information so that
it becomes easier to unwind through them.
Using vdso is a mandatory for a thread cancelation && cleanup
on a modern glibc.

Differential Revision: https://reviews.freebsd.org/D1060

Remove leftover from r282269.

MFC after: 2 weeks
X-MFC with: r282269

Remove support for Xen PV domU kernels. Support for HVM domU kernels
remains. Xen is planning to phase out support for PV upstream since it
is harder to maintain and has more overhead. Modern x86 CPUs include
virtualization extensions that support HVM guests instead of PV guests.
In addition, the PV code was i386 only and not as well maintained recently
as the HVM code.
- Remove the i386-only NATIVE option that was used to disable certain
components for PV kernels. These components are now standard as they
are on amd64.
- Remove !XENHVM bits from PV drivers.
- Remove various shims required for XEN (e.g. PT_UPDATES_FLUSH, LOAD_CR3,
etc.)
- Remove duplicate copy of <xen/features.h>.
- Remove unused, i386-only xenstored.h.

Differential Revision: https://reviews.freebsd.org/D2362
Reviewed by: royger
Tested by: royger (i386/amd64 HVM domU and amd64 PVH dom0)
Relnotes: yes

Only enable the hpet driver on x86 hardware, it doesn't exist on arm64.

Sponsored by: The FreeBSD Foundation

Move common code from sys/i386/i386/mp_machdep.c and
sys/amd64/amd64/mp_machdep.c, to the new common x86 source
sys/x86/x86/mp_x86.c.

Proposed and reviewed by: jhb
Review: https://reviews.freebsd.org/D2347
Sponsored by: The FreeBSD Foundation

Move some common code from sys/amd64/amd64/machdep.c and
sys/i386/i386/machdep.c to new file sys/x86/x86/cpu_machdep.c. Most
of the code is related to the idle handling.

Discussed with: pluknet
Sponsored by: The FreeBSD Foundation

Use VT-d interrupt remapping block (IR) to perform FSB messages
translation. In particular, despite IO-APICs only take 8bit apic id,
IR translation structures accept 32bit APIC Id, which allows x2APIC
mode to function properly. Extend msi_cpu of struct msi_intrsrc and
io_cpu of ioapic_intsrc to full int from one byte.

KPI of IR is isolated into the x86/iommu/iommu_intrmap.h, to avoid
bringing all dmar headers into interrupt code. The non-PCI(e) devices
which generate message interrupts on FSB require special handling. The
HPET FSB interrupts are remapped, while DMAR interrupts are not.

For each msi and ioapic interrupt source, the iommu cookie is added,
which is in fact index of the IRE (interrupt remap entry) in the IR
table. Cookie is made at the source allocation time, and then used at
the map time to fill both IRE and device registers. The MSI
address/data registers and IO-APIC redirection registers are
programmed with the special values which are recognized by IR and used
to restore the IRE index, to find proper delivery mode and target.
Map all MSI interrupts in the block when msi_map() is called.

Since an interrupt source setup and dismantle code are done in the
non-sleepable context, flushing interrupt entries cache in the IR
hardware, which is done async and ideally waits for the interrupt,
requires busy-wait for queue to drain. The dmar_qi_wait_for_seq() is
modified to take a boolean argument requesting busy-wait for the
written sequence number instead of waiting for interrupt.

Some interrupts are configured before IR is initialized, e.g. ACPI
SCI. Add intr_reprogram() function to reprogram all already
configured interrupts, and call it immediately before an IR unit is
enabled. There is still a small window after the IO-APIC redirection
entry is reprogrammed with cookie but before the unit is enabled, but
to fix this properly, IR must be started much earlier.

Add workarounds for 5500 and X58 northbridges, some revisions of which
have severe flaws in handling IR. Use the same identification methods
as employed by Linux.

Review: https://reviews.freebsd.org/D1892
Reviewed by: neel
Discussed with: jhb
Tested by: glebius, pho (previous versions)
Sponsored by: The FreeBSD Foundation
MFC after: 3 weeks

Record the dependency to x86bios in vga_pci

This fixes the build of XEN and XBOX kernels on i386, which was broken
in r279487.

While here, do not build vga_pci_repost() on PC98.

Reported by: bz@

Generalized parts of the XEN timer code into a generic pvclock

KVM clock shares the same data structures between the guest and the host
as Xen so it makes sense to just have a single copy of this code.

Differential Revision: https://reviews.freebsd.org/D1429
Reviewed by: royger (eariler version)
MFC after: 1 month

For clang, disable -Wcast-qual warnings for specific aesni files, since
clang 3.6.0 will emit a number of such warnings for those files, and
they are partially contributed code.

Add some new modes to OpenCrypto. These modes are AES-ICM (can be used
for counter mode), and AES-GCM. Both of these modes have been added to
the aesni module.

Included is a set of tests to validate that the software and aesni
module calculate the correct values. These use the NIST KAT test
vectors. To run the test, you will need to install a soon to be
committed port, nist-kat that will install the vectors. Using a port
is necessary as the test vectors are around 25MB.

All the man pages were updated. I have added a new man page, crypto.7,
which includes a description of how to use each mode. All the new modes
and some other AES modes are present. It would be good for someone
else to go through and document the other modes.

A new ioctl was added to support AEAD modes which AES-GCM is one of them.
Without this ioctl, it is not possible to test AEAD modes from userland.

Add a timing safe bcmp for use to compare MACs. Previously we were using
bcmp which could leak timing info and result in the ability to forge
messages.

Add a minor optimization to the aesni module so that single segment
mbufs don't get copied and instead are updated in place. The aesni
module needs to be updated to support blocked IO so segmented mbufs
don't have to be copied.

We require that the IV be specified for all calls for both GCM and ICM.
This is to ensure proper use of these functions.

Obtained from: p4: //depot/projects/opencrypto
Relnotes: yes
Sponsored by: FreeBSD Foundation
Sponsored by: NetGate

xen: create a new PCI bus override

When running as a Xen PVH Dom0 we need to add custom buses that override
some of the functionality present in the ACPI PCI Bus and the PCI Bus. We
currently override the ACPI PCI Bus, but not the PCI Bus, so add a new
override for the PCI Bus and share the generic functions between them.

Reported by: David P. Discher <dpd@dpdtech.com>
Sponsored by: Citrix Systems R&D

conf/files.amd64:
- Add the new files.

x86/xen/xen_pci_bus.c:
- Generic file that contains the PCI overrides so they can be used by the
several PCI specific buses.

xen/xen_pci.h:
- Prototypes for the generic overried functions.

dev/xen/pci/xen_pci.c:
- Xen specific override for the PCI bus.

dev/xen/pci/xen_acpi_pci.c:
- Xen specific override for the ACPI PCI bus.

o Add Virtio MMIO bus driver to config
o Move Virtio-related to common config file

Add VirtIO console to the x86 NOTES and files

Requested by: jhb

agp_intel.c provides support for AGP on Intel chipsets from 440/BX to
875. This intersects with the agp_i810.c, which supports all Intels
from i810 to Core i5/7. Both agp_intel.c and agp_i810.c are compiled
into kernel when device agp is specified in config, and agp_i810
attach seems to be selected by chance due to linking order.

Strip support for 810 and later from agp_intel.c. Since 440-class
chipsets do not support any long-mode capable CPUs, remove agp_intel.c
from amd64 kernel file list. Note that agp_intel.c is not compiled
into agp.ko on amd64 already.

Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks

msi: add Xen MSI implementation

This patch adds support for MSI interrupts when running on Xen. Apart
from adding the Xen related code needed in order to register MSI
interrupts this patch also makes the msi_init function a hook in
init_ops, so different MSI implementations can have different
initialization functions.

Sponsored by: Citrix Systems R&D

xen/interface/physdev.h:
- Add the MAP_PIRQ_TYPE_MULTI_MSI to map multi-vector MSI to the Xen
public interface.

x86/include/init.h:
- Add a hook for setting custom msi_init methods.

amd64/amd64/machdep.c:
i386/i386/machdep.c:
- Set the default msi_init hook to point to the native MSI
initialization method.

x86/xen/pv.c:
- Set the Xen MSI init hook when running as a Xen guest.

x86/x86/local_apic.c:
- Call the msi_init hook instead of directly calling msi_init.

xen/xen_intr.h:
x86/xen/xen_intr.c:
- Introduce support for registering/releasing MSI interrupts with
Xen.
- The MSI interrupts will use the same PIC as the IO APIC interrupts.

xen/xen_msi.h:
x86/xen/xen_msi.c:
- Introduce a Xen MSI implementation.

x86/xen/xen_nexus.c:
- Overwrite the default MSI hooks in the Xen Nexus to use the Xen MSI
implementation.

x86/xen/xen_pci.c:
- Introduce a Xen specific PCI bus that inherits from the ACPI PCI
bus and overwrites the native MSI methods.
- This is needed because when running under Xen the MSI messages used
to configure MSI interrupts on PCI devices are written by Xen
itself.

dev/acpica/acpi_pci.c:
- Lower the quality of the ACPI PCI bus so the newly introduced Xen
PCI bus can take over when needed.

conf/files.i386:
conf/files.amd64:
- Add the newly created files to the build process.

As per [1] Intel only supports this driver on 64bit platforms.
For now restrict it to amd64. Other architectures might be
re-added later once tested.

Remove the drivers from the global NOTES and files files and move
them to the amd64 specifics.
Remove the drivers from the i386 modules build and only leave the
amd64 version.

Rather than depending on "inet" depend on "pci" and make sure that
ixl(4) and ixlv(4) can be compiled independently [2]. This also
allows the drivers to build properly on IPv4-only or IPv6-only
kernels.

PR: 193824 [2]
Reviewed by: eric.joyner intel.com
MFC after: 3 days

References:
[1] http://lists.freebsd.org/pipermail/svn-src-all/2014-August/090470.html

Implement most of timer_{create,settime,gettime,getoverrun,delete}
for amd64/linux32. Fix the entirely bogus (untested) version from
r161310 for i386/linux using the same shared code in compat/linux.

It is unclear to me if we could support more clock mappings but
the current set allows me to successfully run commercial
32bit linux software under linuxolator on amd64.

Reviewed by: jhb
Differential Revision: D784
MFC after: 3 days
Sponsored by: DARPA, AFRL

Import HyperV Key-Value Pair (KVP) driver and daemon code by Microsoft,
many thanks for their continued support of FreeBSD.

While I'm there, also implement a new build knob, WITHOUT_HYPERV to
disable building and installing of the HyperV utilities when necessary.

The HyperV utilities are only built for i386 and amd64 targets.

This is a stable/10 candidate for inclusion with 10.1-RELEASE.

Submitted by: Wei Hu <weh microsoft com>
MFC after: 1 week

Merge the amd64 and i386 identcpu.c into a single x86 implementation.
This brings the structured extended features mask and VT-x reporting to
i386 and Intel cache and TLB info (under bootverbose) to amd64.

xen: create a Xen nexus to use in PV/PVH

Introduce a Xen specific nexus that is going to be used by Xen PV/PVH
guests.

Sponsored by: Citrix Systems R&D
Approved by: gibbs

x86/xen/xen_nexus.c:
- Introduce a Nexus to use on Xen PV(H) guests, this prevents PV(H)
guests from using the legacy Nexus.

conf/files.amd64:
conf/files.i386:
- Add the xen nexus to the build.

xen: introduce xenpv bus

Create a dummy bus so top level Xen devices can attach to it (instead
of attaching directly to the nexus). This allows to have all the Xen
related devices grouped under a single bus.

Sponsored by: Citrix Systems R&D
Approved by: gibbs

x86/xen/xenpv.c:
- Attach the xenpv bus when running as a Xen guest.
- Attach the ISA bus if needed, in order to attach syscons.

conf/files.amd6:
conf/files.i386:
- Include the xenpv.c file in the build of i386/amd64 kernels using
XENHVM.

dev/xen/console/console.c:
dev/xen/timer/timer.c:
xen/xenstore/xenstore.c:
- Attach to the xenpv bus instead of the Nexus.

dev/xen/xenpci/xenpci.c:
- Xen specific devices on PVHVM guests are no longer attached to the
xenpci device, they are instead attached to the xenpv bus, remove
the now unused methods.

xen: add hooks for Xen PV APIC

Create the necessary hooks in order to provide a Xen PV APIC
implementation that can be used on PVH. Most of the lapic ops
shouldn't be called on Xen, since we trap those operations at a higher
layer.

Sponsored by: Citrix Systems R&D
Approved by: gibbs

x86/xen/hvm.c:
x86/xen/xen_apic.c:
- Move IPI related code to xen_apic.c

x86/xen/xen_apic.c:
- Introduce Xen PV APIC implementation, most of the functions of the
lapic interface should never be called when running as PV(H) guest,
so make sure FreeBSD panics when trying to use one of those.
- Define the Xen APIC implementation in xen_apic_ops.

xen/xen_pv.h:
- Extern declaration of the xen_apic struct.

x86/xen/pv.c:
- Use xen_apic_ops as apic_ops when running as PVH guest.

conf/files.amd64:
conf/files.i386:
- Include the xen_apic.c file in the build of i386/amd64 kernels
using XENHVM.

imagact_binmisc builds for all supported architectures, so enable it for all.

Any bugs in execution will be dealt with as they crop up.

MFC after: 3 weeks
Relnotes: Yes

lindev(4): finish the partial commit in r265212

lindev(4) was only used to provide /dev/full which is now a standard feature of
FreeBSD. /dev/full was never linux-specific and provides a generally useful
feature.

Document this in UPDATING and bump __FreeBSD_version. This will be documented
in the PH shortly.

Reported by: jkim

Add Stacey Son's binary activation patches that allow remapping of
execution to a emumation program via parsing of ELF header information.

With this kernel module and userland tool, poudriere is able to build
ports packages via the QEMU userland tools (or another emulator program)
in a different architecture chroot, e.g. TARGET=mips TARGET_ARCH=mips

I'm not connecting this to GENERIC for obvious reasons, but this should
allow the kernel module to be built by default and enable the building
of the userland tool (which automatically loads the kernel module).

Submitted by: sson@
Reviewed by: jhb@

xen: add an apic_enumerator for PVH

On PVH there's no ACPI, so the CPU enumeration must be implemented
using Xen hypercalls.

Approved by: gibbs
Sponsored by: Citrix Systems R&D

x86/xen/pvcpu_enum.c:
- Enumerate avaiable vCPUs on PVH by using the VCPUOP_is_up
hypercall.
- Set vcpu_id for PVH guests.

conf/files.amd64:
- Include the PV CPU enumerator in the XENHVM build.

xen: implement an early timer for Xen PVH

When running as a PVH guest, there's no emulated i8254, so we need to
use the Xen PV timer as the early source for DELAY. This change allows
for different implementations of the early DELAY function and
implements a Xen variant for it.

Approved by: gibbs
Sponsored by: Citrix Systems R&D

dev/xen/timer/timer.c:
dev/xen/timer/timer.h:
- Implement Xen early delay functions using the PV timer and declare
them.

x86/include/init.h:
- Add hooks for early clock source initialization and early delay
functions.

i386/i386/machdep.c:
pc98/pc98/machdep.c:
amd64/amd64/machdep.c:
- Set early delay hooks to use the i8254 on bare metal.
- Use clock_init (that will in turn make use of init_ops) to
initialize the early clock source.

amd64/include/clock.h:
i386/include/clock.h:
- Declare i8254_delay and clock_init.

i386/xen/clock.c:
- Rename DELAY to i8254_delay.

x86/isa/clock.c:
- Introduce clock_init that will take care of initializing the early
clock by making use of the init_ops hooks.
- Move non ISA related delay functions to the newly introduced delay
file.

x86/x86/delay.c:
- Add moved delay related functions.
- Implement generic DELAY function that will use the init_ops hooks.

x86/xen/pv.c:
- Set PVH hooks for the early delay related functions in init_ops.

conf/files.amd64:
conf/files.i386:
conf/files.pc98:
- Add delay.c to the kernel build.

xen: add PV/PVH kernel entry point

Add the PV/PVH entry point and the low level functions for PVH
early initialization.

Approved by: gibbs
Sponsored by: Citrix Systems R&D

amd64/amd64/genassym.c:
- Add __FreeBSD_version define to assym.s so it can be used for the
Xen notes.

amd64/amd64/locore.S:
- Make bootstack global so it can be used from Xen kernel entry
point.

amd64/amd64/xen-locore.S:
- Add Xen notes to the kernel.
- Add the Xen PV entry point, that is going to call hammer_time_xen.

amd64/include/asmacros.h:
- Add ELFNOTE macros.

i386/xen/xen_machdep.c:
- Define HYPERVISOR_start_info for the XEN i386 PV port, which is
going to be used in some shared code between PV and PVH.

x86/xen/hvm.c:
- Define HYPERVISOR_start_info for the PVH port.

x86/xen/pv.c:
- Introduce hammer_time_xen which is going to perform early setup for
Xen PVH:
- Setup shared Xen variables start_info, shared_info and
xen_store.
- Set guest type.
- Create initial page tables as FreeBSD expects to find them.
- Call into native init function (hammer_time).

xen/xen-os.h:
- Declare HYPERVISOR_start_info.

conf/files.amd64:
- Add amd64/amd64/locore.S and x86/xen/pv.c to the list of files.

Retire the nve(4) driver; nfe(4) has been the default driver for NVIDIA
nForce MCP adapters for a long time.

Yays: jhb, remko, yongari
Nays: none on the current and stable lists

Add very simple virtio_random(4) driver to harvest entropy from host

Reviewed by: markm (random bits only)

Merge VT(9) project (a.k.a. newcons).

Reviewed by: nwhitehorn
MFC_to_10_after: re approval

Sponsored by: The FreeBSD Foundation

Add support for queued invalidation.

Right now, the semaphore write is scheduled after each batch, which is
not optimal and must be tuned.

Discussed with: alc
Tested by: pho
MFC after: 1 month

Import the driver for VT-d DMAR hardware, as specified in the revision
1.3 of Intelб╝ Virtualization Technology for Directed I/O Architecture
Specification. The Extended Context and PASIDs from the rev. 2.2 are
not supported, but I am not aware of any released hardware which
implements them. Code does not use queued invalidation, see comments
for the reason, and does not provide interrupt remapping services.

Code implements the management of the guest address space per domain
and allows to establish and tear down arbitrary mappings, but not
partial unmapping. The superpages are created as needed, but not
promoted. Faults are recorded, fault records could be obtained
programmatically, and printed on the console.

Implement the busdma(9) using DMARs. This busdma backend avoids
bouncing and provides security against misbehaving hardware and driver
bad programming, preventing leaks and corruption of the memory by wild
DMA accesses.

By default, the implementation is compiled into amd64 GENERIC kernel
but disabled; to enable, set hw.dmar.enable=1 loader tunable. Code is
written to work on i386, but testing there was low priority, and
driver is not enabled in GENERIC. Even with the DMAR turned on,
individual devices could be directed to use the bounce busdma with the
hw.busdma.pci<domain>:<bus>:<device>:<function>.bounce=1 tunable. If
DMARs are capable of the pass-through translations, it is used,
otherwise, an identity-mapping page table is constructed.

The driver was tested on Xeon 5400/5500 chipset legacy machine,
Haswell desktop and E5 SandyBridge dual-socket boxes, with ahci(4),
ata(4), bce(4), ehci(4), mfi(4), uhci(4), xhci(4) devices. It also
works with em(4) and igb(4), but there some fixes are needed for
drivers, which are not committed yet. Intel GPUs do not work with
DMAR (yet).

Many thanks to John Baldwin, who explained me the newbus integration;
Peter Holm, who did all testing and helped me to discover and
understand several incredible bugs; and to Jim Harris for the access
to the EDS and BWG and for listening when I have to explain my
findings to somebody.

Sponsored by: The FreeBSD Foundation
MFC after: 1 month

Add a virtual table for the busdma methods on x86, to allow different
busdma implementations to coexist. Copy busdma_machdep.c to
busdma_bounce.c, which is still a single implementation of the busdma
interface on x86 for now. The busdma_machdep.c only contains common
and dispatch code.

Tested by: pho (as part of the larger patch)
Sponsored by: The FreeBSD Foundation
MFC after: 1 month

Snapshot.

Looking pretty good; this mostly works now. New code includes:

* Read cached entropy at startup, both from files and from loader(8) preloaded entropy. Failures are soft, but announced. Untested.

* Use EVENTHANDLER to do above just before we go multiuser. Untested.

Snapshot. This passes the build test, but has not yet been finished or debugged.

Contains:

* Refactor the hardware RNG CPU instruction sources to feed into
the software mixer. This is unfinished. The actual harvesting needs
to be sorted out. Modified by me (see below).

* Remove 'frac' parameter from random_harvest(). This was never
used and adds extra code for no good reason.

* Remove device write entropy harvesting. This provided a weak
attack vector, was not very good at bootstrapping the device. To
follow will be a replacement explicit reseed knob.

* Separate out all the RANDOM_PURE sources into separate harvest
entities. This adds some secuity in the case where more than one
is present.

* Review all the code and fix anything obviously messy or inconsistent.
Address som review concerns while I'm here, like rename the pseudo-rng
to 'dummy'.

Submitted by: Arthur Mesh <arthurmesh@gmail.com> (the first item)

Substantial rewrite of bxe(4) to add support for the BCM57712 and
BCM578XX controllers.

Approved by: re
MFC after: 4 weeks

Revert r255672, it has some serious flaws, leaking file references etc.

Approved by: re (delphij)

Implement epoll support in Linuxulator. This is a tiny wrapper around kqueue
to implement epoll subset of functionality. The kqueue user data are 32bit
on i386 which is not enough for epoll user data so this patch overrides
kqueue fileops to maintain enough space in struct file.

Initial patch developed by me in 2007 and then extended and finished
by Yuri Victorovich.

Approved by: re (delphij)
Sponsored by: Google Summer of Code
Submitted by: Yuri Victorovich <yuri at rawbw dot com>
Tested by: Yuri Victorovich <yuri at rawbw dot com>

Revert the kvp code - there's still some work that
needs to be done for that.

Discussed with: Microsoft hyper-v devs

Latest update from Microsoft.

Obtained from: Microsoft Hyper-v dev team

IFC @ r255209

Use the fact that the AES-NI instructions can be pipelined to improve
performance... Use SSE2 instructions for calculating the XTS tweek
factor... Let the compiler do more work and handle register allocation
by using intrinsics, now only the key schedule is in assembly...

Replace .byte hard coded instructions w/ the proper instructions now
that both clang and gcc support them...

On my machine, pulling the code to userland I saw performance go from
~150MB/sec to 2GB/sec in XTS mode. GELI on GNOP saw a more modest
increase of about 3x due to other system overhead (geom and
opencrypto)...

These changes allow almost full disk io rate w/ geli...

Reviewed by: -current, -security
Thanks to: Mike Hamburg for the XTS tweek algorithm

Implement vector callback for PVHVM and unify event channel implementations

Re-structure Xen HVM support so that:
- Xen is detected and hypercalls can be performed very
early in system startup.
- Xen interrupt services are implemented using FreeBSD's native
interrupt delivery infrastructure.
- the Xen interrupt service implementation is shared between PV
and HVM guests.
- Xen interrupt handlers can optionally use a filter handler
in order to avoid the overhead of dispatch to an interrupt
thread.
- interrupt load can be distributed among all available CPUs.
- the overhead of accessing the emulated local and I/O apics
on HVM is removed for event channel port events.
- a similar optimization can eventually, and fairly easily,
be used to optimize MSI.

Early Xen detection, HVM refactoring, PVHVM interrupt infrastructure,
and misc Xen cleanups:

Sponsored by: Spectra Logic Corporation

Unification of PV & HVM interrupt infrastructure, bug fixes,
and misc Xen cleanups:

Submitted by: Roger Pau Monné
Sponsored by: Citrix Systems R&D

sys/x86/x86/local_apic.c:
sys/amd64/include/apicvar.h:
sys/i386/include/apicvar.h:
sys/amd64/amd64/apic_vector.S:
sys/i386/i386/apic_vector.s:
sys/amd64/amd64/machdep.c:
sys/i386/i386/machdep.c:
sys/i386/xen/exception.s:
sys/x86/include/segments.h:
Reserve IDT vector 0x93 for the Xen event channel upcall
interrupt handler. On Hypervisors that support the direct
vector callback feature, we can request that this vector be
called directly by an injected HVM interrupt event, instead
of a simulated PCI interrupt on the Xen platform PCI device.
This avoids all of the overhead of dealing with the emulated
I/O APIC and local APIC. It also means that the Hypervisor
can inject these events on any CPU, allowing upcalls for
different ports to be handled in parallel.

sys/amd64/amd64/mp_machdep.c:
sys/i386/i386/mp_machdep.c:
Map Xen per-vcpu area during AP startup.

sys/amd64/include/intr_machdep.h:
sys/i386/include/intr_machdep.h:
Increase the FreeBSD IRQ vector table to include space
for event channel interrupt sources.

sys/amd64/include/pcpu.h:
sys/i386/include/pcpu.h:
Remove Xen HVM per-cpu variable data. These fields are now
allocated via the dynamic per-cpu scheme. See xen_intr.c
for details.

sys/amd64/include/xen/hypercall.h:
sys/dev/xen/blkback/blkback.c:
sys/i386/include/xen/xenvar.h:
sys/i386/xen/clock.c:
sys/i386/xen/xen_machdep.c:
sys/xen/gnttab.c:
Prefer FreeBSD primatives to Linux ones in Xen support code.

sys/amd64/include/xen/xen-os.h:
sys/i386/include/xen/xen-os.h:
sys/xen/xen-os.h:
sys/dev/xen/balloon/balloon.c:
sys/dev/xen/blkback/blkback.c:
sys/dev/xen/blkfront/blkfront.c:
sys/dev/xen/console/xencons_ring.c:
sys/dev/xen/control/control.c:
sys/dev/xen/netback/netback.c:
sys/dev/xen/netfront/netfront.c:
sys/dev/xen/xenpci/xenpci.c:
sys/i386/i386/machdep.c:
sys/i386/include/pmap.h:
sys/i386/include/xen/xenfunc.h:
sys/i386/isa/npx.c:
sys/i386/xen/clock.c:
sys/i386/xen/mp_machdep.c:
sys/i386/xen/mptable.c:
sys/i386/xen/xen_clock_util.c:
sys/i386/xen/xen_machdep.c:
sys/i386/xen/xen_rtc.c:
sys/xen/evtchn/evtchn_dev.c:
sys/xen/features.c:
sys/xen/gnttab.c:
sys/xen/gnttab.h:
sys/xen/hvm.h:
sys/xen/xenbus/xenbus.c:
sys/xen/xenbus/xenbus_if.m:
sys/xen/xenbus/xenbusb_front.c:
sys/xen/xenbus/xenbusvar.h:
sys/xen/xenstore/xenstore.c:
sys/xen/xenstore/xenstore_dev.c:
sys/xen/xenstore/xenstorevar.h:
Pull common Xen OS support functions/settings into xen/xen-os.h.

sys/amd64/include/xen/xen-os.h:
sys/i386/include/xen/xen-os.h:
sys/xen/xen-os.h:
Remove constants, macros, and functions unused in FreeBSD's Xen
support.

sys/xen/xen-os.h:
sys/i386/xen/xen_machdep.c:
sys/x86/xen/hvm.c:
Introduce new functions xen_domain(), xen_pv_domain(), and
xen_hvm_domain(). These are used in favor of #ifdefs so that
FreeBSD can dynamically detect and adapt to the presence of
a hypervisor. The goal is to have an HVM optimized GENERIC,
but more is necessary before this is possible.

sys/amd64/amd64/machdep.c:
sys/dev/xen/xenpci/xenpcivar.h:
sys/dev/xen/xenpci/xenpci.c:
sys/x86/xen/hvm.c:
sys/sys/kernel.h:
Refactor magic ioport, Hypercall table and Hypervisor shared
information page setup, and move it to a dedicated HVM support
module.

HVM mode initialization is now triggered during the
SI_SUB_HYPERVISOR phase of system startup. This currently
occurs just after the kernel VM is fully setup which is
just enough infrastructure to allow the hypercall table
and shared info page to be properly mapped.

sys/xen/hvm.h:
sys/x86/xen/hvm.c:
Add definitions and a method for configuring Hypervisor event
delievery via a direct vector callback.

sys/amd64/include/xen/xen-os.h:
sys/x86/xen/hvm.c:

sys/conf/files:
sys/conf/files.amd64:
sys/conf/files.i386:
Adjust kernel build to reflect the refactoring of early
Xen startup code and Xen interrupt services.

sys/dev/xen/blkback/blkback.c:
sys/dev/xen/blkfront/blkfront.c:
sys/dev/xen/blkfront/block.h:
sys/dev/xen/control/control.c:
sys/dev/xen/evtchn/evtchn_dev.c:
sys/dev/xen/netback/netback.c:
sys/dev/xen/netfront/netfront.c:
sys/xen/xenstore/xenstore.c:
sys/xen/evtchn/evtchn_dev.c:
sys/dev/xen/console/console.c:
sys/dev/xen/console/xencons_ring.c
Adjust drivers to use new xen_intr_*() API.

sys/dev/xen/blkback/blkback.c:
Since blkback defers all event handling to a taskqueue,
convert this task queue to a "fast" taskqueue, and schedule
it via an interrupt filter. This avoids an unnecessary
ithread context switch.

sys/xen/xenstore/xenstore.c:
The xenstore driver is MPSAFE. Indicate as much when
registering its interrupt handler.

sys/xen/xenbus/xenbus.c:
sys/xen/xenbus/xenbusvar.h:
Remove unused event channel APIs.

sys/xen/evtchn.h:
Remove all kernel Xen interrupt service API definitions
from this file. It is now only used for structure and
ioctl definitions related to the event channel userland
device driver.

Update the definitions in this file to match those from
NetBSD. Implementing this interface will be necessary for
Dom0 support.

sys/xen/evtchn/evtchnvar.h:
Add a header file for implemenation internal APIs related
to managing event channels event delivery. This is used
to allow, for example, the event channel userland device
driver to access low-level routines that typical kernel
consumers of event channel services should never access.

sys/xen/interface/event_channel.h:
sys/xen/xen_intr.h:
Standardize on the evtchn_port_t type for referring to
an event channel port id. In order to prevent low-level
event channel APIs from leaking to kernel consumers who
should not have access to this data, the type is defined
twice: Once in the Xen provided event_channel.h, and again
in xen/xen_intr.h. The double declaration is protected by
__XEN_EVTCHN_PORT_DEFINED__ to ensure it is never declared
twice within a given compilation unit.

sys/xen/xen_intr.h:
sys/xen/evtchn/evtchn.c:
sys/x86/xen/xen_intr.c:
sys/dev/xen/xenpci/evtchn.c:
sys/dev/xen/xenpci/xenpcivar.h:
New implementation of Xen interrupt services. This is
similar in many respects to the i386 PV implementation with
the exception that events for bound to event channel ports
(i.e. not IPI, virtual IRQ, or physical IRQ) are further
optimized to avoid mask/unmask operations that aren't
necessary for these edge triggered events.

Stubs exist for supporting physical IRQ binding, but will
need additional work before this implementation can be
fully shared between PV and HVM.

sys/amd64/amd64/mp_machdep.c:
sys/i386/i386/mp_machdep.c:
sys/i386/xen/mp_machdep.c
sys/x86/xen/hvm.c:
Add support for placing vcpu_info into an arbritary memory
page instead of using HYPERVISOR_shared_info->vcpu_info.
This allows the creation of domains with more than 32 vcpus.

sys/i386/i386/machdep.c:
sys/i386/xen/clock.c:
sys/i386/xen/xen_machdep.c:
sys/i386/xen/exception.s:
Add support for new event channle implementation.

Add vmx(4), a VMware VMXNET3 ethernet driver ported from OpenBSD

IFC @ r253862

- change the SI_SUB_RUN_SCHEDULER sysinits in hv_utilc and
hv_netvsc_drv_freebsd.c to SI_SUB_KTHREAD_IDLE, since the
former is no longer in FreeBSD.
The use of these SYSINITs can probably be removed.

Add message when nvd disks are attached and detached.

As part of this commit, add an nvme_strvis() function which borrows
heavily from cam_strvis(). This will allow stripping of
leading/trailing whitespace and also handle unprintable characters
in model/serial numbers. This function goes into a new nvme_util.c
file which is used by both the driver and nvmecontrol.

Sponsored by: Intel
Reviewed by: carl
MFC after: 3 days

First cut at adding the hyperv drivers to GENERIC.
The files inventory should probably have the modules split
out into net/storage/common etc as the modules build is,
but this will do for now.

Import HighPoint DC Series Data Center HBA (DC7280 and R750) driver.
This driver works for FreeBSD/i386 and FreeBSD/amd64 platforms.

Many thanks to HighPoint for providing this driver.

MFC after: 1 day

- Trim an unused and bogus Makefile for mount_smbfs.
- Reconnect with some minor modifications, in particular now selsocket()
internals are adapted to use sbintime units after recent'ish calloutng
switch.

Add Qlogic 10Gb Ethernet Driver for Qlogic 8100 Series CNA Adapter
Driver version (v2.0.0)

Submitted by: David C Somayajulu (davidcs@freebsd.org) QLogic Corporation
Approved by: George Neville-Neil (gnn@freebsd.org)

Add basic support for FDT to i386 & amd64. This change includes:
1. Common headers for fdt.h and ofw_machdep.h under x86/include
with indirections under i386/include and amd64/include.
2. New modinfo for loader provided FDT blob.
3. Common x86_init_fdt() called from hammer_time() on amd64 and
init386() on i386.
4. Split-off FDT specific low-level console functions from FDT
bus methods for the uart(4) driver. The low-level console
logic has been moved to uart_cpu_fdt.c and is used for arm,
mips & powerpc only. The FDT bus methods are shared across
all architectures.
5. Add dev/fdt/fdt_x86.c to hold the fdt_fixup_table[] and the
fdt_pic_table[] arrays. Both are empty right now.

FDT addresses are I/O ports on x86. Since the core FDT code does
not handle different address spaces, adding support for both I/O
ports and memory addresses requires some thought and discussion.
It may be better to use a compile-time option that controls this.

Obtained from: Juniper Networks, Inc.

Add Qlogic 10Gigabit Ethernet & CNA Adapter Driver Version 3.10.10 for
QLogic 8300 Series Adapters

Submitted by: David C Somayajulu (davidcs@freebsd.org) QLogic Corporation
Approved by: George Neville-Neil (gnn@freebsd.org)

Add a new driver to support the Intel Non-Transparent Bridge(NTB).

The NTB allows you to connect two systems with this device using a PCI-e
link. The driver is made of two modules:
- ntb_hw which is a basic hardware abstraction layer for the device.
- if_ntb which implements the ntb network device and the communication
protocol.

The driver is limited at the moment to CPU memcpy instead of using DMA, and
only Back-to-Back mode is supported. Also the network device isn't full
featured yet. These changes will be coming soon. The DMA change will also
bring in the ioat driver from the project branch it is on now.

This is an initial port of the GPL/BSD Linux driver contributed by Jon Mason
from Intel. Any bugs are my contributions.

Sponsored by: Intel
Reviewed by: jimharris, joel (man page only)
Approved by: jimharris (mentor)

Unbreak tinderbox build after r249420.

Remove the virtio dependency entry for the VirtIO device drivers. This
will prevent the kernel from linking if the device driver are included
without the virtio module. Remove pci and scbus for the same reason.

Also explain the relationship and necessity of the virtio and virtio_pci
modules. Currently in FreeBSD, we only support VirtIO PCI, but it could
be replaced with a different interface (like MMIO) and the device
(network, block, etc) will still function.

Requested by: luigi
Approved by: grehan (mentor)
MFC after: 3 days

- Don't include date and time the driver is built, this is useful for
generating binary diffs.
- Constify a few strings used in the driver.
- Style changes to make the driver compile with default clang settings.

Approved by: HighPoint Technologies
MFC after: 3 days

IFC @ r245509

Add VirtIO to the i386 and amd64 GENERIC kernels

This also removes the kludge from r239009 that covered only
the network driver.

Reviewed by: grehan
Approved by: grehan (mentor)
MFC after: 1 week

IFC @ r242684

Hoist the MI compat_freebsd32 files up into files from files.*.