diskinfo(8): introduce new option -l

In modes -p or -s, add an option -l to start each line
with a device name separated with a tab. Update the manual page.
Add an example to list names with corresponding serial numbers:

diskinfo -ls /dev/da?

MFC after: 2 weeks

Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

diskinfo: Remove useless return at the end of void function

Signed-off-by: Elyes Haouas <ehaouas@noos.fr>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/656

Make "diskinfo -i" also test 1MB reads.

Add GEOM attribute to report physical device name, and report it
via 'diskinfo -v'. This avoids the need to track it down via CAM,
and should also work for disks that don't use CAM. And since it's
inherited thru the GEOM hierarchy, in most cases one doesn't need
to walk the GEOM graph either, eg you can use it on a partition
instead of disk itself.

Reviewed by: allanjude, imp
Sponsored by: Klara Inc
Differential Revision: https://reviews.freebsd.org/D22249

Sanity check media size and sector counts to ensure that we don't
produce negative sector numbers in the testing algorithm.

CID: 1198994

diskinfo(8): Delete dead code in slog test

Reported by: Coverity
CID: 1377556
Reviewed by: mav
MFC after: 3 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D13275

Fix integer overflow in SLOG test.

MFC after: 1 week

various: general adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.

Display rotation rate and TRIM/UNMAP support in diskinfo(8)

Bump __FreeBSD_version due to the expansion of struct diocgattr_arg.

Reviewed by: mav, allanjude, imp
MFC after: 3 weeks
Relnotes: yes
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D12578

Align test I/O buffer to page boundary.

This is more alike to typical kernel behavior, that can be useful from
benchmarking point of view.

MFC after: 1 week

Alike to ZFS disable cache flush after first ENOTSUP error.

MFC after: 1 week

Report device descr in addition to ident.

Serial number without device model is somewhat less useful.

MFC after: 2 weeks

Add naive benchmark for SSDs in ZFS SLOG role.

ZFS SLOGs have very specific access pattern with many cache flushes,
which none of benchmarks I know can simulate. Since SSD vendors rarely
specify cache flush time, this measurement can be useful to explain why
some ZFS pools are slower then expected. This test writes data chunks
of different size followed by cache flush, alike to what ZFS SLOG does,
and measures average time.

To illustrate, here is result for 6 years old SATA Intel 710 Series SSD:

Synchronous random writes:
0.5 kbytes: 138.3 usec/IO = 3.5 Mbytes/s
1 kbytes: 137.7 usec/IO = 7.1 Mbytes/s
2 kbytes: 151.1 usec/IO = 12.9 Mbytes/s
4 kbytes: 158.2 usec/IO = 24.7 Mbytes/s
8 kbytes: 175.6 usec/IO = 44.5 Mbytes/s
16 kbytes: 210.1 usec/IO = 74.4 Mbytes/s
32 kbytes: 274.2 usec/IO = 114.0 Mbytes/s
64 kbytes: 416.5 usec/IO = 150.1 Mbytes/s
128 kbytes: 776.6 usec/IO = 161.0 Mbytes/s
256 kbytes: 1503.1 usec/IO = 166.3 Mbytes/s
512 kbytes: 2968.7 usec/IO = 168.4 Mbytes/s
1024 kbytes: 5866.8 usec/IO = 170.5 Mbytes/s
2048 kbytes: 11696.6 usec/IO = 171.0 Mbytes/s
4096 kbytes: 23329.6 usec/IO = 171.5 Mbytes/s
8192 kbytes: 46779.5 usec/IO = 171.0 Mbytes/s

, and much newer and supposedly much faster NVMe Samsung 950 PRO SSD:

Synchronous random writes:
0.5 kbytes: 2092.9 usec/IO = 0.2 Mbytes/s
1 kbytes: 2013.1 usec/IO = 0.5 Mbytes/s
2 kbytes: 2014.8 usec/IO = 1.0 Mbytes/s
4 kbytes: 2090.7 usec/IO = 1.9 Mbytes/s
8 kbytes: 2044.5 usec/IO = 3.8 Mbytes/s
16 kbytes: 2084.8 usec/IO = 7.5 Mbytes/s
32 kbytes: 2137.1 usec/IO = 14.6 Mbytes/s
64 kbytes: 2173.4 usec/IO = 28.8 Mbytes/s
128 kbytes: 2923.9 usec/IO = 42.8 Mbytes/s
256 kbytes: 3085.3 usec/IO = 81.0 Mbytes/s
512 kbytes: 3112.2 usec/IO = 160.7 Mbytes/s
1024 kbytes: 2430.6 usec/IO = 411.4 Mbytes/s
2048 kbytes: 3788.9 usec/IO = 527.9 Mbytes/s
4096 kbytes: 6198.0 usec/IO = 645.4 Mbytes/s
8192 kbytes: 10764.9 usec/IO = 743.2 Mbytes/s

While the first one obviously has maximal throughput limitations, the
second one has so high cache flush latency (about 2 millisecond), that
it makes one almost useless in SLOG role, despite of its good throughput
numbers. Power loss protection is out of scope of this test, but I
suspect it can be related.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

Add -s (serial) and -p (physpath) to diskinfo

Return the bare requested information, intended for scripting.

The serial number of a SAS/SCSI device can be returned with
'camcontrol inquiry disk -S', but there is no similar switch for SATA.

This provides a way to get this information from both SAS and SATA disks

the -s and -p flags are mutually exclusive, and cannot be used with any
other flags.

Reviewed by: rpokala, wblock
MFC after: 1 month
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D7828

Quell Coverity for diskinfo(8)

* CID 1198994: Don't run the speed disk on a disk with no sectors
* CID 1011442: Don't call close(2) if open(2) fails
* CID 1011161: Use snprintf instead of sprintf
* CID 1009825: Check the return value of lseek

Reported by: Coverity
CID: 1198994 1011442 1011161 1009825
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp

Add "diskinfo -i", a simple aio-based IOPS benchmark.

MFC after: 1 month

Reduce code duplication by factoring out the delta_t() function.
No functional changes.

MFC after: 1 month

Cosmetics - add missing newline.

MFC after: 1 month

Make it possible for diskinfo(8) to operate on files. This is useful
with -t and upcoming -i.

MFC after: 1 month

Add support for managing Shingled Magnetic Recording (SMR) drives.

This change includes support for SCSI SMR drives (which conform to the
Zoned Block Commands or ZBC spec) and ATA SMR drives (which conform to
the Zoned ATA Command Set or ZAC spec) behind SAS expanders.

This includes full management support through the GEOM BIO interface, and
through a new userland utility, zonectl(8), and through camcontrol(8).

This is now ready for filesystems to use to detect and manage zoned drives.
(There is no work in progress that I know of to use this for ZFS or UFS, if
anyone is interested, let me know and I may have some suggestions.)

Also, improve ATA command passthrough and dispatch support, both via ATA
and ATA passthrough over SCSI.

Also, add support to camcontrol(8) for the ATA Extended Power Conditions
feature set. You can now manage ATA device power states, and set various
idle time thresholds for a drive to enter lower power states.

Note that this change cannot be MFCed in full, because it depends on
changes to the struct bio API that break compatilibity. In order to
avoid breaking the stable API, only changes that don't touch or depend on
the struct bio changes can be merged. For example, the camcontrol(8)
changes don't depend on the new bio API, but zonectl(8) and the probe
changes to the da(4) and ada(4) drivers do depend on it.

Also note that the SMR changes have not yet been tested with an actual
SCSI ZBC device, or a SCSI to ATA translation layer (SAT) that supports
ZBC to ZAC translation. I have not yet gotten a suitable drive or SAT
layer, so any testing help would be appreciated. These changes have been
tested with Seagate Host Aware SATA drives attached to both SAS and SATA
controllers. Also, I do not have any SATA Host Managed devices, and I
suspect that it may take additional (hopefully minor) changes to support
them.

Thanks to Seagate for supplying the test hardware and answering questions.

sbin/camcontrol/Makefile:
Add epc.c and zone.c.

sbin/camcontrol/camcontrol.8:
Document the zone and epc subcommands.

sbin/camcontrol/camcontrol.c:
Add the zone and epc subcommands.

Add auxiliary register support to build_ata_cmd(). Make sure to
set the CAM_ATAIO_NEEDRESULT, CAM_ATAIO_DMA, and CAM_ATAIO_FPDMA
flags as appropriate for ATA commands.

Add a new get_ata_status() function to parse ATA result from SCSI
sense descriptors (for ATA passthrough over SCSI) and ATA I/O
requests.

sbin/camcontrol/camcontrol.h:
Update the build_ata_cmd() prototype

Add get_ata_status(), zone(), and epc().

sbin/camcontrol/epc.c:
Support for ATA Extended Power Conditions features. This includes
support for all features documented in the ACS-4 Revision 12
specification from t13.org (dated February 18, 2016).

The EPC feature set allows putting a drive into a power power mode
immediately, or setting timeouts so that the drive will
automatically enter progressively lower power states after various
idle times.

sbin/camcontrol/fwdownload.c:
Update the firmware download code for the new build_ata_cmd()
arguments.

sbin/camcontrol/zone.c:
Implement support for Shingled Magnetic Recording (SMR) drives
via SCSI Zoned Block Commands (ZBC) and ATA Zoned Device ATA
Command Set (ZAC).

These specs were developed in concert, and are functionally
identical. The primary differences are due to SCSI and ATA
differences. (SCSI is big endian, ATA is little endian, for
example.)

This includes support for all commands defined in the ZBC and
ZAC specs.

sys/cam/ata/ata_all.c:
Decode a number of additional ATA command names in ata_op_string().

Add a new CCB building function, ata_read_log().

Add ata_zac_mgmt_in() and ata_zac_mgmt_out() CCB building
functions. These support both DMA and NCQ encapsulation.

sys/cam/ata/ata_all.h:
Add prototypes for ata_read_log(), ata_zac_mgmt_out(), and
ata_zac_mgmt_in().

sys/cam/ata/ata_da.c:
Revamp the ada(4) driver to support zoned devices.

Add four new probe states to gather information needed for zone
support.

Add a new adasetflags() function to avoid duplication of large
blocks of flag setting between the async handler and register
functions.

Add new sysctl variables that describe zone support and paramters.

Add support for the new BIO_ZONE bio, and all of its subcommands:
DISK_ZONE_OPEN, DISK_ZONE_CLOSE, DISK_ZONE_FINISH, DISK_ZONE_RWP,
DISK_ZONE_REPORT_ZONES, and DISK_ZONE_GET_PARAMS.

sys/cam/scsi/scsi_all.c:
Add command descriptions for the ZBC IN/OUT commands.

Add descriptions for ZBC Host Managed devices.

Add a new function, scsi_ata_pass() to do ATA passthrough over
SCSI. This will eventually replace scsi_ata_pass_16() -- it
can create the 12, 16, and 32-byte variants of the ATA
PASS-THROUGH command, and supports setting all of the
registers defined as of SAT-4, Revision 5 (March 11, 2016).

Change scsi_ata_identify() to use scsi_ata_pass() instead of
scsi_ata_pass_16().

Add a new scsi_ata_read_log() function to facilitate reading
ATA logs via SCSI.

sys/cam/scsi/scsi_all.h:
Add the new ATA PASS-THROUGH(32) command CDB. Add extended and
variable CDB opcodes.

Add Zoned Block Device Characteristics VPD page.

Add ATA Return SCSI sense descriptor.

Add prototypes for scsi_ata_read_log() and scsi_ata_pass().

sys/cam/scsi/scsi_da.c:
Revamp the da(4) driver to support zoned devices.

Add five new probe states, four of which are needed for ATA
devices.

Add five new sysctl variables that describe zone support and
parameters.

The da(4) driver supports SCSI ZBC devices, as well as ATA ZAC
devices when they are attached via a SCSI to ATA Translation (SAT)
layer. Since ZBC -> ZAC translation is a new feature in the T10
SAT-4 spec, most SATA drives will be supported via ATA commands
sent via the SCSI ATA PASS-THROUGH command. The da(4) driver will
prefer the ZBC interface, if it is available, for performance
reasons, but will use the ATA PASS-THROUGH interface to the ZAC
command set if the SAT layer doesn't support translation yet.
As I mentioned above, ZBC command support is untested.

Add support for the new BIO_ZONE bio, and all of its subcommands:
DISK_ZONE_OPEN, DISK_ZONE_CLOSE, DISK_ZONE_FINISH, DISK_ZONE_RWP,
DISK_ZONE_REPORT_ZONES, and DISK_ZONE_GET_PARAMS.

Add scsi_zbc_in() and scsi_zbc_out() CCB building functions.

Add scsi_ata_zac_mgmt_out() and scsi_ata_zac_mgmt_in() CCB/CDB
building functions. Note that these have return values, unlike
almost all other CCB building functions in CAM. The reason is
that they can fail, depending upon the particular combination
of input parameters. The primary failure case is if the user
wants NCQ, but fails to specify additional CDB storage. NCQ
requires using the 32-byte version of the SCSI ATA PASS-THROUGH
command, and the current CAM CDB size is 16 bytes.

sys/cam/scsi/scsi_da.h:
Add ZBC IN and ZBC OUT CDBs and opcodes.

Add SCSI Report Zones data structures.

Add scsi_zbc_in(), scsi_zbc_out(), scsi_ata_zac_mgmt_out(), and
scsi_ata_zac_mgmt_in() prototypes.

sys/dev/ahci/ahci.c:
Fix SEND / RECEIVE FPDMA QUEUED in the ahci(4) driver.

ahci_setup_fis() previously set the top bits of the sector count
register in the FIS to 0 for FPDMA commands. This is okay for
read and write, because the PRIO field is in the only thing in
those bits, and we don't implement that further up the stack.

But, for SEND and RECEIVE FPDMA QUEUED, the subcommand is in that
byte, so it needs to be transmitted to the drive.

In ahci_setup_fis(), always set the the top 8 bits of the
sector count register. We need it in both the standard
and NCQ / FPDMA cases.

sys/geom/eli/g_eli.c:
Pass BIO_ZONE commands through the GELI class.

sys/geom/geom.h:
Add g_io_zonecmd() prototype.

sys/geom/geom_dev.c:
Add new DIOCZONECMD ioctl, which allows sending zone commands to
disks.

sys/geom/geom_disk.c:
Add support for BIO_ZONE commands.

sys/geom/geom_disk.h:
Add a new flag, DISKFLAG_CANZONE, that indicates that a given
GEOM disk client can handle BIO_ZONE commands.

sys/geom/geom_io.c:
Add a new function, g_io_zonecmd(), that handles execution of
BIO_ZONE commands.

Add permissions check for BIO_ZONE commands.

Add command decoding for BIO_ZONE commands.

sys/geom/geom_subr.c:
Add DDB command decoding for BIO_ZONE commands.

sys/kern/subr_devstat.c:
Record statistics for REPORT ZONES commands. Note that the
number of bytes transferred for REPORT ZONES won't quite match
what is received from the harware. This is because we're
necessarily counting bytes coming from the da(4) / ada(4) drivers,
which are using the disk_zone.h interface to communicate up
the stack. The structure sizes it uses are slightly different
than the SCSI and ATA structure sizes.

sys/sys/ata.h:
Add many bit and structure definitions for ZAC, NCQ, and EPC
command support.

sys/sys/bio.h:
Convert the bio_cmd field to a straight enumeration. This will
yield more space for additional commands in the future. After
change r297955 and other related changes, this is now possible.
Converting to an enumeration will also prevent use as a bitmask
in the future.

sys/sys/disk.h:
Define the DIOCZONECMD ioctl.

sys/sys/disk_zone.h:
Add a new API for managing zoned disks. This is very close to
the SCSI ZBC and ATA ZAC standards, but uses integers in native
byte order instead of big endian (SCSI) or little endian (ATA)
byte arrays.

This is intended to offer to the complete feature set of the ZBC
and ZAC disk management without requiring the application developer
to include SCSI or ATA headers. We also use one set of headers
for ioctl consumers and kernel bio-level consumers.

sys/sys/param.h:
Bump __FreeBSD_version for sys/bio.h command changes, and inclusion
of SMR support.

usr.sbin/Makefile:
Add the zonectl utility.

usr.sbin/diskinfo/diskinfo.c
Add disk zoning capability to the 'diskinfo -v' output.

usr.sbin/zonectl/Makefile:
Add zonectl makefile.

usr.sbin/zonectl/zonectl.8
zonectl(8) man page.

usr.sbin/zonectl/zonectl.c
The zonectl(8) utility. This allows managing SCSI or ATA zoned
disks via the disk_zone.h API. You can report zones, reset write
pointers, get parameters, etc.

Sponsored by: Spectra Logic
Differential Revision: https://reviews.freebsd.org/D6147
Reviewed by: wblock (documentation)

remove duplicate semicolons where possible.

Approved by: cperciva
MFC after: 1 week

Improve error reporting in diskinfo(8) by not displaying errno when
it doesn't make sense.

Fix `diskinfo -t` operation for disks smaller then 8GB or bigger then 2TB.
Tested to work with 1.44MB floppy, 4GB USB stick and 4TB disk array.

Approved by: re (kib)

Plumb device physical path reporting from CAM devices, through GEOM and
DEVFS, and make it accessible via the diskinfo utility.

Extend GEOM's generic attribute query mechanism into generic disk consumers.
sys/geom/geom_disk.c:
sys/geom/geom_disk.h:
sys/cam/scsi/scsi_da.c:
sys/cam/ata/ata_da.c:
- Allow disk providers to implement a new method which can override
the default BIO_GETATTR response, d_getattr(struct bio *). This
function returns -1 if not handled, otherwise it returns 0 or an
errno to be passed to g_io_deliver().

sys/cam/scsi/scsi_da.c:
sys/cam/ata/ata_da.c:
- Don't copy the serial number to dp->d_ident anymore, as the CAM XPT
is now responsible for returning this information via
d_getattr()->(a)dagetattr()->xpt_getatr().

sys/geom/geom_dev.c:
- Implement a new ioctl, DIOCGPHYSPATH, which returns the GEOM
attribute "GEOM::physpath", if possible. If the attribute request
returns a zero-length string, ENOENT is returned.

usr.sbin/diskinfo/diskinfo.c:
- If the DIOCGPHYSPATH ioctl is successful, report physical path
data when diskinfo is executed with the '-v' option.

Submitted by: will
Reviewed by: gibbs
Sponsored by: Spectra Logic Corporation

Add generic attribute change notification support to GEOM.

sys/sys/geom/geom.h:
Add a new attrchanged method field to both g_class
and g_geom.

sys/sys/geom/geom.h:
sys/geom/geom_event.c:
- Provide the g_attr_changed() function that providers
can use to advertise attribute changes.
- Perform delivery of attribute change notifications
from a thread context via the standard GEOM event
mechanism.

sys/geom/geom_subr.c:
Inherit the attrchanged method from class to geom (class instance).

sys/geom/geom_disk.c:
Provide disk_attr_changed() to provide g_attr_changed() access
to consumers of the disk API.

sys/cam/scsi/scsi_pass.c:
sys/cam/scsi/scsi_da.c:
sys/geom/geom_dev.c:
sys/geom/geom_disk.c:
Use attribute changed events to track updates to physical path
information.

sys/cam/scsi/scsi_da.c:
Add AC_ADVINFO_CHANGED to the registered asynchronous CAM
events for this driver. When this event occurs, and
the updated buffer type references our physical path
attribute, emit a GEOM attribute changed event via the
disk_attr_changed() API.

sys/cam/scsi/scsi_pass.c:
Add AC_ADVINFO_CHANGED to the registered asynchronous CAM
events for this driver. When this event occurs, update
the physical patch devfs alias for this pass instance.

Submitted by: gibbs
Sponsored by: Spectra Logic Corporation

Don't consider ioctl returning error as a fatal error and move on to next
disk. When multiple devices are specified, a non-supporting ioctl will
cause diskinfo(8) to stop at the point which does not seem to be necessary.

Ok'ed by: phk
MFC after: 1 month

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

MFC 200968:
Make diskinfo report disk stripe size and offset. It should help users to
make file systems optimally aligned and tuned for better performance.

MFC r196799:
Don't bother obtaining the ident if we are not going to print it.

Make diskinfo report disk stripe size and offset. It should help users to
make file systems optimally aligned and tuned for better performance.

Don't bother obtaining the ident if we are not going to print it.

- Import the HEAD csup code which is the basis for the cvsmode work.

Print provider's ident when in verbose mode.

Add -c to SYNOPSIS and usage().

Add -c option that calculates the overhead of read I/O ops.

Actually negative size is possible for file system, but not for disk.

In verbose mode print disk sizes in human readable form as well.

OK'ed by: phk

Make WARNS=5 clean.

Prodded by: Stefan Farfeleder <stefan@fafoe.narf.at>

Sync manpage's synopsis with usage().

I forgot a rather crucial cast when I made this sectorsize agnostic.

Tune iteration counts a bit.

Diskinfo is a small program to access the basic properties of a disk device:
sectorsize, mediasize etc.

It also contains a small and naïve benchmark which reports on seek
and transfer performance.