usr.sbin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

bhyve: Replace many fprintf(stderr, ...) calls with EPRINTLN

EPRINTLN handles newlines appropriately when stdout/stderr have been
reused as the backend for a serial port.

For bhyverun.c itself, the rule this attempts to follow is to use
regular fprintf/perror/warn/err prior to init_pci() (which is when
serial ports are configured) and to switch to EPRINTLN afterwards.

Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D42182

bhyve ahci: Replace WPRINTF with EPRINTLN

Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D42181

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

bhyve: add bootindex option for several devices

The bootindex option creates an entry in the "bootorder" fwcfg file.
This file can be picked up by the guest firmware to determine the
bootorder. Nevertheless, it's not guaranteed that the guest firmware
uses the bootorder. At the moment, our OVMF ignores the bootorder. This
will change in the future.

If guest firmware supports the "bootorder" fwcfg file and no device uses
the bootindex option, the boot order is determined by the firmware
itself. If one or more devices specify a bootindex, the first bootable
device with the lowest bootindex will be booted. It's not garanteed that
devices without a bootindex will be recognized as bootable from the
firmware in that case.

Reviewed by: jhb
MFC after: 1 week
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D39285

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

bhyve: Remove vmctx member from struct vm_snapshot_meta.

This is a userland-only pointer that isn't relevant to the kernel and
doesn't belong in the ioctl structure shared between userland and the
kernel. For the kernel, the old structure for the ioctl is still
supported under COMPAT_FREEBSD13.

This changes vm_snapshot_req() in libvmmapi to accept an explicit
vmctx argument.

It also changes vm_snapshot_guest2host_addr to take an explicit vmctx
argument. As part of this change, move the declaration for this
function and its wrapper macro from vmm_snapshot.h to snapshot.h as it
is a userland-only API.

Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D38125

bhyve: Remove vmctx argument from PCI device model methods.

Most of these arguments were unused. Device models which do need
access to the vmctx in one of these methods can obtain it from the
pi_vmctx member of the pci_devinst argument instead.

Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D38096

bhyve: Remove unused vcpu argument from PCI read/write methods.

Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37652

bhyve: Avoid unlikely truncation of the blockif ident strings.

The ident string for NVMe and VirtIO block deivces do not contain the
bus, and the various fields can potentially use up to three characters
when printed as unsigned values (full range of uint8_t) even if not
likely in practice.

Reviewed by: corvink, chuck
Differential Revision: https://reviews.freebsd.org/D37488

bhyve: Address signed/unsigned comparison warnings in the AHCI model

No functional change intended.

MFC after: 1 week

bhyve: Avoid arithmetic on void pointers

No functional change intended.

MFC after: 1 week

bhyve: Annotate unused function parameters

MFC after: 1 week

bhyve: Mark variables and functions as static where appropriate

Mark them const as well when it makes sense to do so. No functional
change intended.

MFC after: 1 week
Sponsored by: The FreeBSD Foundation

bhyve: Snapshot impovements for 'blockif' backend

When pausing a block I/O device model as part of suspending a VM, wait
for all active block I/O requests to finish before saving snapshot
data. This avoids having to save information about in-flight requests
both in the block_if layer and in storage device models.

For the AHCI device model, the queues are now guaranteed to be idle
when taking a snapshot, so remove the code to save queue state and
rely on the initial state in a resumed VM having all queues already
idle.

This will also simplify adding NVMe snapshot support in the future.

Reviewed by: jhb
Sponsored by: vStack
Differential Revision: https://reviews.freebsd.org/D26267

bhyve: ahci: Fix regression with no ports

An AHCI controller may be specified with no connected ports. Avoid
dumping core in this case for compatibility with existing VM configs.

Reviewed by: khng, jhb
Fixes: 621b5090487de Refactor configuration management in bhyve.
MFC after: 1 week
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D33969

bhyve: clean up trailing whitespaces

Clean up trailing whitespaces. No functional changes.

Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D33681

Refactor configuration management in bhyve.

Replace the existing ad-hoc configuration via various global variables
with a small database of key-value pairs. The database supports
heirarchical keys using a MIB-like syntax to name the path to a given
key. Values are always stored as strings. The API used to manage
configuation values does include wrappers to handling boolean values.
Other values use non-string types require parsing by consumers.

The configuration values are stored in a tree using nvlists. Leaf
nodes hold string values. Configuration values are permitted to
reference other configuration values using '%(name)'. This permits
constructing template configurations.

All existing command line arguments now set configuration values. For
devices, the "-s" option parses its option argument to generate a list
of key-value pairs for the given device.

A new '-o' command line option permits setting an individual
configuration variable. The key name is always given as a full path
of dot-separated components.

A new '-k' command line option parses a simple configuration file.
This configuration file holds a flat list of 'key=value' lines where
the 'key' is the full path of a configuration variable. Lines
starting with a '#' are comments.

In general, bhyve starts by parsing command line options in sequence
and applying those settings to configuration values. Once this is
complete, bhyve then begins initializing its state based on the
configuration values. This means that subsequent configuration
options or files may override or supplement previously given settings.

A special 'config.dump' configuration value can be set to true to help
debug configuration issues. When this value is set, bhyve will print
out the configuration variables as a flat list of 'key=value' lines.

Most command line argments map to a single configuration variable,
e.g. '-w' sets the 'x86.strictmsr' value to false. A few command
line arguments have less obvious effects:

- Multiple '-p' options append their values (as a comma-seperated
list) to "vcpu.N.cpuset" values (where N is a decimal vcpu number).

- For '-s' options, a pci.<bus>.<slot>.<function> node is created.
The first argument to '-s' (the device type) is used as the value of
a "device" variable. Additional comma-separated arguments are then
parsed into 'key=value' pairs and used to set additional variables
under the device node. A PCI device emulation driver can provide
its own hook to override the parsing of the additonal '-s' arguments
after the device type.

After the configuration phase as completed, the init_pci hook
then walks the "pci.<bus>.<slot>.<func>" nodes. It uses the
"device" value to find the device model to use. The device
model's init routine is passed a reference to its nvlist node
in the configuration tree which it can query for specific
variables.

The result is that a lot of the string parsing is removed from
the device models and centralized. In addition, adding a new
variable just requires teaching the model to look for the new
variable.

- For '-l' options, a similar model is used where the string is
parsed into values that are later read during initialization.
One key note here is that the serial ports use the commonly
used lowercase names from existing documentation and examples
(e.g. "lpc.com1") instead of the uppercase names previously
used internally in bhyve.

Reviewed by: grehan
MFC after: 3 months
Differential Revision: https://reviews.freebsd.org/D26035

Fix regression in AHCI controller settings.

When the AHCI code was reworked to use FreeBSD struct
definitions, the valid element was mis-transcribed resulting
in the UMDA capability being hidden. This prevented Illumos
from using AHCI disk/cdrom drives.

Fix by using definitions that match the code pre-rework.

PR: 250924
Submitted by: Rolf Stalder
Reported by: Rolf Stalder
MFC after: 3 days

bhyve: Fix build with option BHYVE_SNAPSHOT

'ident' was replaced with 'ata_ident' in revision r363596.

Submitted by: Vitaliy Gusev <gusev.vitaliy_gmail.com>
Reviewed by: Darius Mihai
Differential Revision: https://reviews.freebsd.org/D26263

Replace magic numbers in Identify page register 0 with ATA definitions.

No functional change. Verified with objdump output before/after.

Requested by: rpokala
Reviewed by: rpokala
MFC after: 3 weeks

Support the setting of additional AHCI controller parameters.

Allow the serial number, firmware revision, model number and nominal media
rotation rate (nmrr) parameters to be set from the command line.

Note that setting the nmrr value can be used to indicate the AHCI
device is an SSD.

Submitted by: Wanpeng Qian
Reviewed by: jhb, grehan (#bhyve)
Approved by: jhb, grehan
MFC after: 3 weeks
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D24174

Initial support for bhyve save and restore.

Save and restore (also known as suspend and resume) permits a snapshot
to be taken of a guest's state that can later be resumed. In the
current implementation, bhyve(8) creates a UNIX domain socket that is
used by bhyvectl(8) to send a request to save a snapshot (and
optionally exit after the snapshot has been taken). A snapshot
currently consists of two files: the first holds a copy of guest RAM,
and the second file holds other guest state such as vCPU register
values and device model state.

To resume a guest, bhyve(8) must be started with a matching pair of
command line arguments to instantiate the same set of device models as
well as a pointer to the saved snapshot.

While the current implementation is useful for several uses cases, it
has a few limitations. The file format for saving the guest state is
tied to the ABI of internal bhyve structures and is not
self-describing (in that it does not communicate the set of device
models present in the system). In addition, the state saved for some
device models closely matches the internal data structures which might
prove a challenge for compatibility of snapshot files across a range
of bhyve versions. The file format also does not currently support
versioning of individual chunks of state. As a result, the current
file format is not a fixed binary format and future revisions to save
and restore will break binary compatiblity of snapshot files. The
goal is to move to a more flexible format that adds versioning,
etc. and at that point to commit to providing a reasonable level of
compatibility. As a result, the current implementation is not enabled
by default. It can be enabled via the WITH_BHYVE_SNAPSHOT=yes option
for userland builds, and the kernel option BHYVE_SHAPSHOT.

Submitted by: Mihai Tiganus, Flavius Anton, Darius Mihai
Submitted by: Elena Mihailescu, Mihai Carabas, Sergiu Weisz
Relnotes: yes
Sponsored by: University Politehnica of Bucharest
Sponsored by: Matthew Grooms (student scholarships)
Sponsored by: iXsystems
Differential Revision: https://reviews.freebsd.org/D19495

bhyve: add wrapper for debug printf statements

Add printf() wrapper to use CR/CRLF terminators depending on whether
stdio is mapped to a tty open in raw mode.
Try to use the wrapper everywhere.
For now we leave the custom DPRINTF/WPRINTF defined by device
models, but we may remove them in the future.

Reviewed by: grehan, jhb
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D22657

bhyve: uniform printf format string newlines

Some of the printf statements only use LF to get a newline. However, a CR character is also required for the serial console to print debug logs in a nice way.
Fix those code locations that only use LF, by adding a CR character.

Reviewed by: markj, aleksandr.fedorov@itglobal.com
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D22552

Define AHCI_PORT_IDENT and increase by 1 the VTBLK_BLK_ID_BYTES
to avoid buffer accessed out of bounds, also switch to snprintf(3).

PR: 200859
Submitted by: Caglar <caglar@10ur.org>
Obtained from: https://github.com/mist64/xhyve/pull/24
MFC after: 4 weeks
Sponsored by: iXsystems Inc.

various: general adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.

Fix variable for sizeof() in previous commit.

MFC after: 2 weeks

Add Log directory and SATA NCQ Send and Receive Log.

Those are used at least by Linux guests to detect queued TRIM support.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

Fix NCQ TRIM emulation.

When this code was written, there was no guests using it to test.

Add emulation for multiple (up to 16) MSI vectors for AHCI.

It was useless before, but may improve performance now if multiple devices
are configured and guest supports this feature.

Sponsored by: iXsystems, Inc.

Allow AHCI controller to support up to 32 arbitrary devices.

While old syntax is still supported, new syntax looks like this:

-s 3,ahci,hd:/dev/zvol/XXX,hd:/dev/zvol/YYY,cd:/storage/ZZZ.iso

Sponsored by: iXsystems, Inc.

Fix gcc warnings

Put cfl/prdt under AHCI_DEBUG #defines as they are only used in
those cases.

Approved by: re (gjb)
Differential Revision: https://reviews.freebsd.org/D7119
MFC after: 1 week
Reported by: Jenkins
Reviewed by: grehan (maintainer)
Sponsored by: EMC / Isilon Storage Division

Use MIN()/MAX() macros from sys/param.h.

Reviewed by: grehan
Differential Revision: https://reviews.freebsd.org/D6054

bhyve: Remove unneeded variable ncq.

ncq was not being inititialized properly but it was not actually
necessary either, so make the code smaller by removing it.

CID: 1248842
Reviewed by: grehan

Cleanup unused-but-set-variable spotted by gcc-4.9.

Reviewed by: neel
Approved by: rodrigc (mentor)
Differential Revision: https://reviews.freebsd.org/D5042

Clean up some harmless unimplemented-command warning messages.

- Don't advertize trusted-computing capability in the Identify page.
This prevents Windows from issuing a TRUSTED_RECEIVE_DMA command.
- Windows will send down SMART and SECURITY_FREEZE_LOCK
even though smart and security capabilities were not advertized.
Send back a silent abort.

Reviewed by: mav

Add simple (no-op) emulations for the CHECK_POWER_MODE,
READ_VERIFY and READ_VERIFY_EXT commands.

Reviewed by: mav

Allow byte reads of AHCI registers.

This is needed to support Windows guests that use byte reads to access certain
AHCI registers (e.g. PxTFD.Status and PxTFD.Error).

Reviewed by: grehan, mav
Reported by: Leon Dang (ldang@nahannisys.com)
Differential Revision: https://reviews.freebsd.org/D2469
MFC after: 2 weeks

Reimplement queue freeze on error, added in r282429:

It is not required to use CLO to recover from task file error, it should
be enough to do only stop/start, that does not clear the PxTFD.STS.ERR.

MFC after: 13 days

Implement in-order execution of non-NCQ commands.

Using status updates in r282364, block queue on BSY, DRQ or ERR bits set.
This can be a performance penalization for non-NCQ commands, but it is
required for proper error recovery and standard compliance.

MFC after: 2 weeks

Implement basic PxTFD.STS.BSY reporting.

MFC after: 2 weeks

Initialize PxCMD on reset and make its read-only bits such.

MFC after: 2 weeks

Handle ATA_SEND_FPDMA_QUEUED as NCQ in ahci_port_stop().

MFC after: 1 week

Don't set bits that should be zero for SATA devices.

Old value made Linux think that it is PATA device with SATA bridge.

MFC after: 2 weeks

Workaround bhyve virtual disks operation on top of GEOM providers.

GEOM does not support scatter/gather lists in its I/Os. Such requests
are cut in pieces by physio(), that may be problematic, if those pieces
are not multiple of provider's sector size. If such case is detected,
move the data through temporary sequential buffer.

MFC after: 2 weeks

Make virtual AHCI more careful with I/O lengths.

MFC after: 2 weeks

Add missing variable initialization.

Reported by: Coverity
CID: 1288938
MFC after: 3 days

Give AHCI disk serial based on backing file path same as for virtio block.

It is still not good that they may intersect on different hosts, but that
is better then intersecting on the same host.

MFC after: 2 weeks

Block delete capability for read-only devices.

Submitted by: neel
MFC after: 2 weeks

Add checksums to identify data and NCQ command error log.

MFC after: 2 weeks

Slightly polish virtual AHCI CD reporting.

MFC after: 2 weeks

Fix NOP and IDLE commands for virtual AHCI disks.

MFC after: 2 weeks

Add support for NCQ variant of DSM TRIM for virtual AHCI disks.

The code is not really tested yet due to lack of initiator support.

Requested by: imp
MFC after: 2 weeks

Improve NCQ errors reporting for virtual AHCI disks.

While this implementation is still not perfect, previous was just broken.

MFC after: 2 weeks

Remove incorrect SERR register setting.

At this point we have nothing to report through that register.

MFC after: 2 weeks

Change prdbc value reporting.

MFC after: 2 weeks

Polish AHCI disk identify data and fix speed negotiation.

MFC after: 2 weeks

Add support for PIO variants of READ/WRITE commands for AHCI disks.

AHCI API hides all PIO specifics, so this functionality is almost free.

MFC after: 2 weeks

Use ahci_write_fis_d2h() for commands completion.

MFC after: 2 weeks

Add DSM TRIM command support for virtual AHCI disks.

It works only for virtual disks backed by ZVOLs and raw devices supporting
BIO_DELETE. Virtual disks backed by files won't report this capability.

MFC after: 2 weeks
Relnotes: yes

Report logical/physical sector sizes for virtual SATA disk.

MFC after: 2 weeks

Don't close a block context if it couldn't be opened,
for example if the backing file doesn't exist,
avoiding a null deref.

Reviewed by: neel
MFC after: 1 week.

If the start bit, PxCMD.ST, is cleared and nothing is in-flight then
PxCI, PxSACT, PxCMD.CCS and PxCMD.CR should be 0.

Reviewed by: grehan

Support stopping and restarting the AHCI command list via toggling
PxCMD.ST from '1' to '0' and back. This allows the driver a chance to
recover if for instance a timeout occurred due to activity on the
host.

Reviewed by: grehan

Commands which encounter a fatal error shouldn't be marked as completed.
Furthermore, provide an indication of the current command so it can be
determined which one actually failed.

Reviewed by: grehan

Use the blockif CHS routine to create fake CHS values,
and then populate them in the identity page.

This fixes a divide-by-zero error at probe time with NetBSD.

MFC after: 1 week.

Implement legacy interrupts for the AHCI device emulation
according to the method outlined in the AHCI spec.

Tested with FreeBSD 9/10/11 with MSI disabled,
and also NetBSD/amd64 (lightly).

Reviewed by: neel, tychon
MFC after: 3 weeks

Use calloc() in favor of malloc + memset.

Reviewed by: neel

Constrain the amount of data returned to what is actually available
not the size of the buffer.

Approved by: grehan (co-mentor)

Don't reissue in-flight commands.

Approved by: neel (co-mentor)

Though there currently isn't a way to insert new media into an ATAPI
drive, at least pretend to support Asynchronous Notification (AN) to
avoid a guest needlessly polling for it.

Approved by: grehan (co-mentor)

Provide an indication a "PIO Setup Device to Host FIS" occurred while executing
the IDENTIFY DEVICE and IDENTIFY PACKET DEVICE commands.

Also, provide an indication a "D2H Register FIS" occurred during a SET FEATURES
command.

Approved by: grehan (co-mentor)

Fix issue with stale fields from a recycled request pulled off the freelist.

Approved by: grehan (co-mentor)

The Data Byte Count (DBC) field of a Physical Region Descriptor
Table is 22 bits, with the bit 31 being the interrupt-on-completion
bit.

OpenBSD and UEFI set this bit, resulting in large block i/o lengths
being sent to bhyve and coredumping the process. Fix by masking off
the relevant 22 bits when using the DBC field as a length.

Reviewed by: Zhixiang Yu
Discussed with: Tycho Nightingale (tycho.nightingale@pluribusnetworks.com)
MFC after: 10.0

Add the VM name to the process name with setproctitle().
Remove the VM name from some of the thread-naming calls
since it is now in the proc title.
Slightly modify the thread-naming for the net and block
threads.

This improves readability when using top/ps with the -a
and -H options on a system with a large number of bhyve VMs.

Requested by: Michael Dexter
Reviewed by: neel
MFC after: 4 weeks

Fix AHCI ATAPI emulation when backed with /dev/cd0

- remove assumption that the backing file/device had
512-byte sectors
- fix incorrect iovec size variable that would result
in a buffer overrun when an o/s issued an i/o request
with more s/g elements than the blockif api

Reviewed by: Zhixiang Yu (zxyu.core@gmail.com)
MFC after: 3 days

Eliminate unconditional debug printfs.

Linux writes to these nominally read-only registers,
so avoid having bhyve write warning messages to stdout
when the reg writes can be safely ignored. Change the
WPRINTF to DPRINTF which is conditional.

Reviewed by: mav
Discussed with: mav, Zhixiang Yu
MFC after: 3 days

In usr.sbin/bhyve/pci_ahci.c, fix several gcc warnings of the form
"assignment makes pointer from integer without a cast", by changing the
cmd_lst and rbis members of struct ahci_port from integers to pointers.

Also surround a pow-of-2 test expression with parentheses to clarify it,
and avoid another gcc warning.

Approved by: re (glebius)
Reviewed by: grehan, mav

Import Zhixiang Yu's GSoC'13 AHCI emulation:
https://wiki.freebsd.org/SummerOfCode2013/bhyveAHCI

This provides ICH8 SATA disk and ATAPI ports, selectable
via the bhyve slot command-line parameter:

SATA
-s <slot>,ahci-hd,<image-file>

ATAPI
-s <slot>,ahci-cd,<image-file>

Slight modifications by: grehan@
Approved by: re@ (blanket)
Obtained from: FreeBSD GSoC'13