nvmecontrol: Add nvme 2.0 fields to read_logpage

Add the lpo, ot, csi and uuid_index fields to read_logpage. The logpage
command has not been updated to allow these to be specified.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D44683

nvmecontrol: Move intel temperature page printing to little endian orderinng

Sponsored by: Netflix
Reviewed by: chuck
Differential Revision: https://reviews.freebsd.org/D44658

nvmecontrol: Move sanitize status page printing to little endian orderinng

Sponsored by: Netflix
Reviewed by: chuck
Differential Revision: https://reviews.freebsd.org/D44657

nvmecontrol: Move reservation notifcation page printing to little endian orderinng

Sponsored by: Netflix
Reviewed by: chuck
Differential Revision: https://reviews.freebsd.org/D44656

nvmecontrol: Move command effeccts page printing to little endian orderinng

Sponsored by: Netflix
Reviewed by: chuck
Differential Revision: https://reviews.freebsd.org/D44655

nvmecontrol: Move self test status page printing to little endian orderinng

Also, add printing vnedor_specific field, which doesn't have a valid
bit, so is always valid.

Sponsored by: Netflix
Reviewed by: chuck
Differential Revision: https://reviews.freebsd.org/D44654

nvmecontrol: Move namespace change page printing to little endian orderinng

Sponsored by: Netflix
Reviewed by: chuck
Differential Revision: https://reviews.freebsd.org/D44653

nvmecontrol: Move smart/health printing to little endian orderinng

Move health printing to little endian ordering...

Sponsored by: Netflix
Reviewed by: chuck
Differential Revision: https://reviews.freebsd.org/D44652

nvmecontrol: Make the error log page work on native format

As the number of page types proliferates, it becomes untennable to
convert them in read_logpage (especailly since new UUID page types will
need to be supported). Convert the error page printing code to operate
on little endian data.

Sponsored by: Netflix
Reviewed by: chuck
Differential Revision: https://reviews.freebsd.org/D44680

nvme: Add my copyright to logpage.c

Reflect the command line refactoring I did, and other changes. git blame
says I'm to blame for ~1/4 of this file.

Sponsored by: Netflix

nvme: Firmware revisions in the firmware slot info logpage are ASCII strings

In particular, don't try to byteswap the values as 64-bit integers and
always print a non-empty version as a string.

Reviewed by: chuck, imp
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D44121

nvmecontrol: Use the NVMEV macro instead of expanded versions

Reviewed by: imp
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D43597

sbin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

nvmecontrol: fix typos

PR: 261911
Reviewed by: kp

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

nvmecontrol: fix wrong temperature unit for INTEL SSDs.

Although intel's specification did not tell which unit for Temperature
Statistics (Log Identifier C5h), I believe it is based on Celsius
instead of Kelvin.

here is my P3700 SSDs result(before):

Intel Temperature Log
=====================
Current: 30 K, -243.15 C, -405.67 F
Overtemp Last Flags 0
Overtemp Lifetime Flags 0
Max Temperature 53 K, -220.15 C, -364.27 F
Min Temperature 17 K, -256.15 C, -429.07 F
Max Operating Temperature 63 K, -210.15 C, -346.27 F
Min Operating Temperature 0 K, -273.15 C, -459.67 F
Estimated Temperature Offset: 0 C/K
after apply the patch, result is

Intel Temperature Log
=====================
Current: 303.15 K, 30 C, 86.00 F
Overtemp Last Flags 0
Overtemp Lifetime Flags 0
Max Temperature 326.15 K, 53 C, 127.40 F
Min Temperature 290.15 K, 17 C, 62.60 F
Max Operating Temperature 336.15 K, 63 C, 145.40 F
Min Operating Temperature 273.15 K, 0 C, 32.00 F
Estimated Temperature Offset: 0 C/K
I also compare to smartctl's report. it match very well.

also tested on Intel P3600, it fixed the problem.

Signed-off-by: Wanpeng Qian <wanpengqian@gmail.com>
Reviewed by: imp (added tweak to samsung.c so it still compiles)
Differential Revision: https://reviews.freebsd.org/D32845

nvme: fix spelling of Namespace

Fix spelling of a macro definition.

Reviewed by: mav, imp
Differential Revision: https://reviews.freebsd.org/D34330

Fix 32-bit build post 6733401935f83754b4b2744bc3d33ef84b1271e0

The general style in sbin/nvmecontrol apppears to print uint64_t types
using %j, so I'm using that instead of the more general (but admittedly
ugly) PRIu64.

nvmecontrol: add device self-test op and log page

Add decoding of the Device Self-test log page and the ability to start
or abort a test.

Reviewed by: imp, mav
Tested by: Muhammad Ahmad <muhammad.ahmad@seagate.com>
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D27517

Improve nvmecontrol error reporting.

MFC after: 1 week
Sponsored by: iXsystems, Inc.

Fix nvmecontrol logpage -i parameter.

MFC after: 3 days

Open device with O_RDONLY when command is non-invasive.

This allows to use some of the subcommands against mounted nvd devices.

MFC after: 1 week
Sponsored by: iXystems, Inc.

Fix various Coverity-detected errors in nvmecontrol

This fixes several Coverity-detected errors in nvmecontrol. While in
here, a couple additional errors with shift/mask confusion that were
not diagnosed by Coverity are also fixed.

CIDs addressed: 1040299, 1040300, 1403972, 1403973, 1403985, 1403988,
1403990, 1404374, 1404427, 1404469, 1404510, 1404534, 1418118

CID 1403657 (resource leak of shared library handle) was marked
"intentional" in the Coverity scan database.

Reviewed by: vangyzen, robert.herndon_dell.com
Reviewed by: daniel.william.ryan_gmail.com (earlier version)
Reviewed by: rramsden_isilon.com (earlier version), imp
MFC after: 5 days
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D24203

Decode few more NVMe log pages.

In particular: Changed Namespace List, Commands Supported and Effects,
Reservation Notification, Sanitize Status.

Add few new arguments to `nvmecontrol log` subcommand.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

Add more new fields and values from NVMe 1.4.

MFC after: 2 weeks

Add IOCTL to translate nvdX into nvmeY and NSID.

While very useful by itself, it also makes `nvmecontrol` not depend on
hardcoded device names parsing, that in its turn makes simple to take
nvdX (and potentially any other) device names as arguments.

Also added IOCTL bypass from nvdX to respective nvmeYnsZ makes them
interchangeable for management purposes.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

Fix the fix to the logic bug. Upon further testing, the bug is that we shadoow
opt.vendor with vendor. We shouldn't. Delete the latter and use the former
everywhere and restore the prior logic which is now correct.

Fix several related coverity issues:

Make sure to always free shortopts and lopts when returning.
Fix minor logic bug to guard against NULLs properly.

CID: 1403654, 1403656, 1403658

Create generic command / arg parsing routines

Create a set of routines and structures to hold the data for the args
for a command. Use them to generate help and to parse args. Convert
all the current commands over to the new format. "comnd" is a hat-tip
to the TOPS-20 %COMND JSYS that (very) loosely inspired much of the
subsequent command line notions in the industry, but this is far
simpler (the %COMND man page is longer than this code) and not in the
kernel... Also, it implements today's de-facto
command [verb]+ [opts]* [args]*
format rather than the old, archaic TOPS-20 command format :)

This is a snapshot of a work in progress to get the nvme passthru
stuff committed. In time it will become a private library and used
by some other programs in the tree that conform to the above pattern.

Differential Revision: https://reviews.freebsd.org/D19296

Rework logpage extensibility.

Move from using a linker set to a constructor function that's
called. This simplifies the code and is slightly more obvious. We now
keep a list of page decoders rather than having an array we managed
before. Commands will move to something similar in the future.

Reviewed by: jhb@
Differential Revision: https://reviews.freebsd.org/D19275

Const poison the command interface

Make the pointers we pass into the commands const, also make the
linker set mirrors const.

Suggested by: cem@
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D18459

Dynamically load .so modules to expand functionality

o Dynamically load all the .so files found in /libexec/nvmecontrol and
/usr/local/libexec/nvmecontrol.
o Link nvmecontrol -rdynamic so that its symbols are visible to the
libraries we load.
o Create concatinated linker sets that we dynamically expand.
o Add the linked-in top and logpage linker sets to the mirrors for them
and add those sets to the mirrors when we load a new .so.
o Add some macros to help hide the names of the linker sets.
o Update the man page.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D18455

fold

Fix typo in comment

Sponsored by: Netflix

Move Intel specific log pages to intel.c

Move the Intel specific log pages (including the one that samsung
implements) to intel.c. Add comment to the samsung vendor that it will
be going away soon.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D18403

Usage cleanup pt 2

Eliminage redundant spaces and nvmecontrol at start of all the usage
strings. Update the usage printing code to add them back when
presenting to the user. Allow multi-line usage messages and print
proper leading spaces for lines starting with a space.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D18403

Usage cleanup pt 1

Provide a usage() function that takes a struct nvme_function pointer
and produces a usage mssage. Eliminate all now-redundant usage
functions. Propigate the new argument through the program as needed.
Use common routine to print usage.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D18403

Move the hgst/wdc log page printing code into wdc.c

These are all hgst/wdc specific, so move them into the wdc.c to live
with the wdc command.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D18403

Move common logpage routines into nvmecontrol.h

For the upcoming move of vendor specific code into vendor specific
files, make the common logpage routines global and move them to
nvmecontrol.h.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D18403

Make logpage functions a linker set.

Move logpage function def to header. Convert all the logpage_function
elements to elements of the linker set. Leave them all in logpage.c
for the moment.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D18403

Move nvmecontrol to using linker sets for commands

More commands will be added to nvmecontrol. Also, there will be a few
more vendor commands (some of which may need to remain private to
companies writing them). The first step on that journey is to move to
using linker sets to dispatch commands. The next step will be using
dlopen to bring in the .so's that have the command that might need
to remain private for seamless integration.

Similar changes to this will be needed for vendor specific log pages.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D18403

Fix typo

The vendor name wds should have been wdc. Add wdc and keep the wds for
script compat.

MFC after: 3 days

Make NVMe compatible with the original API

The original NVMe API used bit-fields to represent fields in data
structures defined by the specification (e.g. the op-code in the command
data structure). The implementation targeted x86_64 processors and
defined the bit fields for little endian dwords (i.e. 32 bits).

This approach does not work as-is for big endian architectures and was
changed to use a combination of bit shifts and masks to support PowerPC.
Unfortunately, this changed the NVMe API and forces #ifdef's based on
the OS revision level in user space code.

This change reverts to something that looks like the original API, but
it uses bytes instead of bit-fields inside the packed command structure.
As a bonus, this works as-is for both big and little endian CPU
architectures.

Bump __FreeBSD_version to 1200081 due to API change

Reviewed by: imp, kbowling, smh, mav
Approved by: imp (mentor)
Differential Revision: https://reviews.freebsd.org/D16404

Add `nvmecontrol format` subcommand.

It allows to change namespace parameters, such as block size, metadata,
protection information, etc. and/or erase the data.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

NVMe: Add big-endian support

Remove bitfields from defined structures as they are not portable.
Instead use shift and mask macros in the driver and nvmecontrol application.

NVMe is now working on powerpc64 host.

Submitted by: Michal Stanek <mst@semihalf.com>
Obtained from: Semihalf
Reviewed by: imp, wma
Sponsored by: IBM, QCM Technologies
Differential revision: https://reviews.freebsd.org/D13916

various: general adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.

Report only the valid slots in the firmware log page.

Printing the entire log page is causing confusion over available
slots. Report only those slots that are valid. In the case where the
firmware download isn't supported, assume that only the first slot is
valid (I have no hardware to test this assumption though)

Sponsored by: Netflix

Add nvme_controller_data argument to all print functions.

It's desirable to access controler data to inform printing log pages
(such as limiting the printing to valid ranges).

Sponsored by: Netflix

Move 128-bit integer routines to util.c so they can be used by more
than just the log page code.

Sponsored by: Netflix, Inc
Submitted by: Matt Williams (via D11330)

Make nvmecontrol logpage -p help list known pages.

Make -p help and -v help list all the pages we know about.
Add -v to usage.
Update the man page.

Sponsored by: Netflix

Fix typos in output.

Sponsored by: Netflix

Add the ability to dump log pages directly in binary to stdout.
Update man page to include this flag, and an example of dumping a
vendor-specific page while I'm here.

Add some descriptions to the man page for the supported log pages as
well as the new wdc commands. Make wdc be an alias for hgst when
specifying the vendor to use to interpret the page.

Implement Intel's log page 0xc1 (Read Command Latency Log) and page
0xc1 (Write Command Latency Log).

Sponsored By: Netflix, Inc

Fix Typo

Flag the vendor specific pages as such. This allows different decoding
for the same page number as different vendors encode vendor specific
pages differently.

i386 turns out to not have __uint128_t. So confusingly use 64-bit math
instead. Since we're little endian, we can get away with it. Also,
since the counters in quesitons would require billions of iops for
tens of billions of seconds to overflow, and since such data rates are
unlikely for people using i386 for a while, that's OK. The fastest
cards today can't do even a million IOPs.

Noticed by: dim@
Sponsored by: Netflix, Inc

Decode the Intel-specific Additional SMART data page (0xca) and print
it in human readable form. Include a pointer to the public spec that
was followed to implement this in the code. Samsung also implements
page 0xca on some of their drives, but the format is slighly
different, so the code skips printing zero keys. Samsung's log page
has additional, unknown data after the end of Intel defined data which
isn't displayed.

Supported by: Netfix, Inc

Implement HGST Log page 0xc1, as documented in the HGST SN100 and
SN150 product manuals. Subpage 0x32 is documented, but not implemented.

Sponsored by: Netflix, Inc

Print Intel's expanded Temperature log page.

Sponsored by: Netflix, Inc

Expand the SMART / Health Information Log Page (Page 02) printout
based on NVM Express 1.2.1 Standard.

Sponsored by: Netflix, Inc

Print numbers instead of hex values for smart data. The full 128-bit
number is printed, even though you'd need like a billion IOPs for a 10
billion seconds to overflow the 64-bit counters (~300 years).

Sponsored by: Netflix, Inc

Use a table for pages we know the size of. We have a special case for
the error log since it isn't a fixed size.

Sponsored by: Netflix, Inc

Remove check for valid log pages. Let the drive tell us which pages
are valid or not. While many pages are reserved in the standard, that
doesn't make them invalid and future versions of the standard may
define then.

Sponsored by: Netflix, Inc

nvmecontrol: read controller identify data before any log page operations

MFC after: 3 days
Sponsored by: Intel

Send per-namespace logpage commands to the controller devnode, so they
are processed as admin commands, not I/O commands.

As part of this change, pull out the code for parsing a namespace node
string into a separate function, since it is used for both identify and
logpage commands.

Sponsored by: Intel
MFC after: 3 days

Incorporate feedback from bde@ based on r252672 changes:

* Use 0/1 instead of sysexits. Man pages are confusing on this topic,
but 0/1 is sufficient for nvmecontrol.
* Use err function family where possible instead of fprintf/exit.
* Fix some typing errors.
* Clean up some error message inconsistencies.

Sponsored by: Intel
Submitted by: bde (parts of firmware.c changes)
MFC after: 3 days

Fix build.

Add log page support to nvmecontrol(8) through a new logpage command.

This includes pretty printers for all of the standard NVMe log pages
(Error, SMART/Health, Firmware), as well as hex output for non-standard
or vendor-specific log pages.

Submitted by: Joe Golio <joseph.golio@emc.com>
Obtained from: EMC / Isilon Storage Division
MFC after: 3 days