cam: Add a XPORT_NVMF for NVMe over Fabrics sims

Reviewed by: ken, imp
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D44713

camcontrol: Enable WITH_NVME unconditionally

MK_NVME is no longer marked broken for any platforms, so just include
support for it always as we do for ATA and SCSI.

Reviewed by: emaste
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D44841

camcontrol: Add a sense subcommand

As the name suggests, this sends a SCSI REQUEST SENSE to a device,
and prints out decoded sense information. It can also print out a
hexdump of the sense data.

sbin/camcontrol/camcontrol.c:
Add the new sense subcommand.

sbin/camcontrol/camcontrol.8:
Document camcontrol sense.

Sponsored by: Spectra Logic
Reviewed by: mav
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D43225

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

camcontrol: Migrate to modern uintXX_t from u_intXX_t

As per https://lists.freebsd.org/archives/freebsd-scsi/2023-July/000257.html
move to the modern uintXX_t.

MFC After: 3 days
Sponsored by: Netflix

camcontrol: Include stdbool.h rather than relying on namespace pollution

I have several environments, and at least one of them fails to build
because bool is undefined. Since we use bool, always include stdbool.h
rather than relying on any indirect definitions to pull it in.

Sponsored by: Netflix

camcontrol: In defects make -S actually imply 12-byte command

camcontrol(8) says that -S to start at a different offset implies that
we're using the 12 byte command. But really, we're using the 10-byte
command. Fix this by setting use_12byte for -S.

Sponsored by: Netflix
Reviewed by: mav
Differential Revision: https://reviews.freebsd.org/D40526

camcontrol: simplify summary logic for defects

We don't need to count the number of lists requested. Instead, use the
more general form of checking to see if any of the non-defect format
bits are set. Also, check summary boolean to control summary reporting
behavior.

Sponsored by: Netflix
Reviewed by: mav
Differential Revision: https://reviews.freebsd.org/D40524

camcontrol: No need to set CAM_ARG_[PG]LIST in defects

We can or in the the list_format bits directly if we or in the
list_format when we look it up the first time. Free up CAM_ARG_[PG]LIST
from the CAM_ARG_xxx enum.

Sponsored by: Netflix
Reviewed by: mav (I made his suggested change)
Differential Revision: https://reviews.freebsd.org/D40523

camcontrol: Simplfiy 12-byte retry logic in defects command

We always start out using the 10-byte version of READ DEFECT DATA, and
then switch to 12-byte when necessary due to errors or data length
requirements. We always need to get the length again when we do this,
and we're always going to be using 12-byte commands from that point
forward. Simplify the logic a bit based on this observation.

Sponsored by: Netflix
Reviewed by: mav
Differential Revision: https://reviews.freebsd.org/D40522

camcontrol: Convert int used as booleans to bools

Sponsored by: Netflix
Reviewed by: mav
Differential Revision: https://reviews.freebsd.org/D40521

camcontrol: Label the unused command args

Remove CAM_ARG_FORMAT_{BLOCK,BLI,PHYS} since they are not used. Label
all the unused CAM_ARG_ bits as unused in comments to make them stand
out.

Sponsored by: Netflix
Reviewed by: mav
Differential Revision: https://reviews.freebsd.org/D40520

camcontrol: Fix typos

Signed-off-by: Elyes Haouas <ehaouas@noos.fr>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/653

camcontrol powermode: fix use-after-free

Free the ccb after processing the response therein.

Reported by: Coverity
Fixes: 3bed0179ee571ba3ceadb2773f86d57af8bc5499
MFC after: 1 week
Sponsored by: Dell EMC Isilon

add and use defintions for ATA power modes

Those can be returned by CHECK POWER MODE command (0xe5).
Note that some of the definitions duplicate definitions for Extended
Power Conditions.

MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D33646

camcontrol: dump received data for MMC command even if it is unknown

For example, EXT_CSD can be read like this:
# camcontrol mmcsdcmd 2:0:0 -c 8 -a 0 -f 0x35 -l 512
CMD 8 arg 0 flags 35
MMCIO: error 0, 00000900 00000000 00000000 00000000
No command-specific decoder for CMD 8
0000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
0010 39 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |9...............|
...
0100 00 00 00 00 00 00 00 00 01 08 00 01 02 02 00 00 |................|
...
01e0 00 00 00 00 00 00 00 00 00 81 c7 00 00 01 03 07 |................|
01f0 05 00 03 01 3f 3f 01 01 01 00 00 00 00 00 00 00 |....??..........|

MFC after: 2 weeks

camcontrol: depop command

Implement and document the new depop command. This command manages drive elements
for drives that support it. Storage elements are typically heads. Element status
can be discovered. Elements may be removed or restored. And the status of any
current depop operation can be assessed.

depop -d elm will remove element elm and truncate available capacity.
depop -l will list the current drive elements and their current status.
depop -r elm will try to restore all retired elements and rebuild capacity.

Changing storage elements may reinitialize the drive. This operation will lose
data and may take hours to complete. Use the drive provided timeout for
operations by default.

Reviewed by: gbe (manpages)
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D29018

camcontrol(8): remove unnecessary CCB zeroing

After 3e404b8c53d, cam_getccb(3) clears the returned CCB, making
a number of calls to CCB_CLEAR_ALL_EXCEPT_HDR(3) unnecessary.

Reviewed By: imp
Sponsored by: NetApp, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D27812

camcontrol: stop pretending cam_cmd is a bitmask

Cleanup only, no functional changes.

Reviewed By: imp
Sponsored By: NetApp, Inc.
Sponsored By: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D27813

camcontrol: change hueristic for I/O-less devtype

Some SATA drives have 'config' set to 0 in the identify block. Rather than rely
on it, use the strings windows uses to display the drive since they are supposed
to be space padded and will always be non-zero.

sbin/camcontrol: use calloc/strlcpy where appropriate.

MFC after: 2 weeks

Add help messages for camcontrol(8) MMCCAM functionality

This adds the help messages for camcontrol(8) in-binary help.
Man page will follow in the separate change.

Reviewed by: bz
Approved by: imp (mentor)
Differential Revision: https://reviews.freebsd.org/D25963

Make it possible to get/set MMC frequency from camcontrol

Enhance camcontrol(8) so that it's possible to manually set frequency for SD/MMC cards.
While here, display more information about the current controller, such as
supported operating modes and VCCQ voltages, as well as current VCCQ voltage.

Reviewed by: manu
Approved by: imp (mentor)
Differential Revision: https://reviews.freebsd.org/D25795

Declare packed struct ata_params as 2-byte-aligned

This avoids gcc9 warning about unaligned access to the structure when
casting to uint16_t pointer type.

Submitted by: imp
Reviewed by: imp
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D22888

Properly detect ATA sanitize errors.

It seems I read specifications not careful enough. There are devices not
setting successful completion bit, causing previous code report false error.

MFC after: 1 week

Fix off-by-one error in HPA/AMA maximum reporting.

Before my refactoring the code reported value as maximum number of sectors,
adding one to the maximum sector number returned by respective command.
While this difference is somewhat confusing, restore previous behavior.

MFC after: 3 days

Remove Tagged Command Queuing feature reporting.

I never saw those devices myself, FreeBSD never supported them, and it is
officially obsolete since ACS-2 specification.

MFC after: 3 days

Report Trusted Computing feature set support.

It practically means the device is SED.

MFC after: 3 days

Remove struct ata_res_pass16, unneeded after r352082.

MFC after: 4 days

Initialize page/subpage in case of `modepage -d`.

Previously without -m parameter it worked mostly by concodence.

While there, make page/subpage values validation more strict.

MFC after: 5 days
Sponsored by: iXsystems, Inc.

Add one more error message to r352082.

MFC after: 5 days
Sponsored by: iXsystems, Inc.

Fix number of problems found while testing on SAT devices.

- Remove incomplete and dangerous ata_res decoding from ata_do_cmd().
Instead switch all functions that need the result to use get_ata_status(),
doing the same, but more careful, also reducing code duplication.
- Made get_ata_status() to also decode fixed format sense. In many cases
it is still not enough to make it useful, since it can only report results
of 28-bit command, but it is slightly better then nothing.
- Organize error reporting in ata_do_cmd(), so that if caller specified
AP_FLAG_CHK_COND, it is responsible for command errors (non-ioctl ones).
- Make HPA/AMA errors not fatal for `identify` subcommand.
- Fix reprobe() not being called on HPA/AMA when in quiet mode.
- Remove not very useful messages from `format` and `sanitize` commands
with -y flag. Once they started, they often can't be stopped any way.

MFC after: 5 days
Sponsored by: iXsystems, Inc.

Unify cam_send_ccb() error reporting.

Error there mean that command was not even executed, and all information
we have about it is errno, and cam_error_print() call is not very useful.
Plus it is most likely a programmatic error, that shoud not happen.

MFC after: 1 week
Sponsored by: iXsystems, Inc.

Supply SAT layer with valid transfer sizes.

This is a rework of r344701, that noticed that number of bytes passes to
8 bit sector count field gets truncated. First decision was to not pass
anything, since ATA specs define the field as N/A. But it appeared to be a
problem for some SAT devices, that require information about data transfer
to operate properly. Some additional investigation shown that it is quite
a common practice to set unused fields of ATA commands (fortunately ATA
specs formally allow it) to supply the information to SAT layer. I have
found SAS-SATA interposer that does not allow pass-through without it.

As side effect, reduce code duplication by removing ata_do_28bit_cmd()
function, replacing it with more universal ata_do_cmd().

MFC after: 1 week
Sponsored by: iXsystems, Inc.

Make `camcontrol modepage` support block descriptors.

It allows to read and write block descriptors alike to mode page parameters.
It allows to change block size or short-stroke HDDs or overprovision SSDs.
Depenting on -P parameter the change can be either persistent or till reset.
In case of block size change device may need reformat after the setting.
In case of SSD overprovisioning format or sanitize may be needed to really
free the flash.

During implementation appeared that csio_encode_visit() can not handle
integers of more then 4 bytes, that makes 8-byte LBA handling awkward.
I had to split it into two 4-byte halves now.

MFC after: 1 week
Relnotes: yes
Sponsored by: iXsystems, Inc.

Make `camcontrol modepage` to use 10 byte commands.

While old devices may not support 10 byte MODE SENSE/MODE SELECT commands,
new ones may not be able to report all mode pages with 6 byte commands.

This patch makes camcontrol by default start with 10 byte commands and
fall back to 6 byte on ILLEGAL REQUEST error, or 6 byte can be forced.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

Decode some more IDENTIFY DEVICE bits.

MFC after: 2 weeks

Make `camcontrol sanitize` support also ATA devices.

ATA sanitize is functionally identical to SCSI, just uses different
initiation commands and status reporting mechanism.

While there, make kernel better handle sanitize commands and statuses.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

Make `camcontrol hpa` and `camcontrol ama` trigger reprobe.

This makes OS automatically see the disk's new disk size.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

Unify BTL parsing for `camcontrol debug` and `reset`.

This makes `camcontrol debug` also allow peripheral device specification.

While there, make BTL parser more strict and switch from strtok() to
strsep().

MFC after: 2 weeks

Properly report ACS revisions alike to kernel.

MFC after: 2 weeks

Add Accessible Max Address Configuration support to camcontrol.

AMA replaced HPA in ACS-3 specification. It allows to limit size of the
disk alike to HPA, but declares inaccessible data as indeterminate. One
of its practical use cases is to under-provision SATA SSDs for better
reliability and performance.

While there, fix HPA Security detection/reporting.

MFC after: 2 weeks
Relnotes: yes
Sponsored by: iXsystems, Inc.

Use a different approach to range check.

gcc hates dt < CC_DT_NONE since it can never be true when dt is an unsigned
type. Since that's a compiler choice and may be affected by weird stuff, instead
use (unsigned)dt > CC_DT_UNKNOWN to test for bounds error since that will work
regardless of the signedness of dt.

Implement a devtype command.

List the device's protocol. The returned value is one of the following:
ata direct attach ATA or SATA device
satl a SATA device attached via SAS
scsi A parallel SCSI or SAS
nvme A direct attached NVMe device
mmcsd A MMC or SD attached device

Reviewed by: scottl@, rpokala@
Differential Revision: https://reviews.freebsd.org/D20950

Use the more proper term of SATL instead of ATA_BEHIND_SCSI.

Most people know SAS attached SATA devices by the name SAT or SATL
(with the latter being a little more common). Change the device type
ATA_BEHIND_SCSI to SATL since it's more specific and meaningful.

Suggested by: scottl@

Add device type NVME and device type MMCSD to get_device_type

For completeness, add nvme and mmc/sd devices to the list of device
types we know.

Retire support for -DMINIMALISTIC

We've not used this in years since we retired sysinstall, and it
hasn't compiled in at least a year. A full camcontrol is only 180k, so
making it smaller is not as important as it once was.

OK'd by: ken@, scottl@

Replay r349341 by imp accidentally reverted by r349352

Use ata_param_fixup instead of a custom copy here

Remove NAND and NANDFS support

NANDFS has been broken for years. Remove it. The NAND drivers that
remain are for ancient parts that are no longer relevant. They are
polled, have terrible performance and just for ancient arm
hardware. NAND parts have evolved significantly from this early work
and little to none of it would be relevant should someone need to
update to support raw nand. This code has been off by default for
years and has violated the vnode protocol leading to panics since it
was committed.

Numerous posts to arch@ and other locations have found no actual users
for this software.

Relnotes: Yes
No Objection From: arch@
Differential Revision: https://reviews.freebsd.org/D20745

Use ata_param_fixup instead of a custom copy here

Increase the timeout for READ NATIVE MAX

READ NATIVE MAX can take longer than a second if the queued NCQ I/Os
take longer than a second to drain.

Add ATA power mode support to camcontrol

Add the ability to report ATA device power mode with the cmmand 'powermode'
to compliment the existing ability to set it using idle, standby and sleep
commands.

MFC after: 2 weeks
Sponsored by: Multiplay

Add new fields to mmc_data in preparation to SDIO CMD53 block mode support

SDIO command CMD53 (IO_RW_EXTENDED) allows data transfers using blocks of 1-2048 bytes,
with a maximum of 511 blocks per request.
Extend mmc_data structure to properly describe such requests,
and initialize the new fields in kernel and userland consumers.

No actual driver changes happen yet, these will follow in the separate changes.

Reviewed by: bz
Approved by: imp (mentor)
Differential Revision: https://reviews.freebsd.org/D19779

Add -l to camcontrol readcap.

The -l flag sends only the READ CAPACITY (16) sevice action. Normally
we send the READ CAPACITY (10) command, and only send RC16 when the
capacity is larger than 2TB (since that's the max RC10 can
report). However, some badly programmed drives report different
numbers for RC10 and RC16. This can be hard to diagnose, but generally
there's a "Logical block address out of range" error when RC16 reports
a larger number than RC10 and the RC10 number is the correct one. By
comparing the output of readcap with and without the -l argmuent, one
can determine if there's a mismatch and if the DA_Q_NO_RC16 quirk is
needed.

Reviewed by: ken@
Differential Revision: https://reviews.freebsd.org/D19536

Fix incorrect / unused sector_count for identify requests

Fix incorrect / unused sector_count for identify requests from camcontrol.

Submitted by: Alexey Dokuchaev
Reported by: Alexey Dokuchaev
MFC after: 1 week
Sponsored by: Multiplay
Differential Revision: https://reviews.freebsd.org/D19408

NVME support is only for x86 and powerpc64.

Implement MK_NVME now that the expression for where NVMe is
complicated. Default it to "yes" for x86 and powerpc64 and
no everywhere else. Use it in camcontrol to define WITH_NVME
for those platforms where we support nvme.

This should fix the newly introduced nvme files to camcontrol
which were building everywhere.

Pointy Hat To: imp
Sponsored by: Netflix

Make camcontrol identify work with nda devices

Both ATA and NVME have an identify command. They are completely
different, but to the user they are the same. Leverage nvmecontrol's
print_controller code to provide that functionality to camcontrol
identify. Query the path to see what kind of protocol it supports, and
send the most appropriate command down. Refactor nvme_print_dev a
little to make it easy to get the nvme cdata.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D15371

nda protocol rate reporting

Report the NVMe spec, number of lanes (and max) as well as the PCIe
generation we're negotiated at (and max) for the camcontrol rate
command.

Reviewed by: scottl (the output, not the code)
Sponsored by: Netflix

Revert ABI breakage to CAM that came in with MMC/SD support in r320844.
Make it possible to retrieve mmc parameters via the XPT_GET_ADVINFO
call instead. Convert camcontrol to the new scheme.

Reviewed by: imp. kibab
Sponsored by: Netflix
Differential Revision: D13868

Refactor code related to 'camcontrol devlist'

Obtained from: Netflix

Implement the ability to query NVME for its controller data so that it will
be shown when issueing the 'camcontrol devlist' command.

Obtained from: Netflix

Print ZAC device type in "camcontrol identify" output

ZAC (Zoned-device ATA Command set) is the standard for addressing SMR
(shingled magnetic recording) devices over SATA. Drives indicate their
support for ZAC in their IDENTIFY block. Print whether and how a drive
supports ZAC in the output of "camcontrol identify".

Reviewed by: ken, imp
MFC after: 3 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D13171

Fix whitespace bugs

- Delete trailing whitespace.
- Replace 8 single column spaces with hard tabs.
- Delete lines with consisting purely of blank space.
- Add space between `return` and `(`, per style(9).

Special care was taken to not blindly replace 8 single column spaces
with tabs; doing so could break tools that do strict string comparisons
with camcontrol output.

Fix -Wuninitialized warning with gcc

Initialize mmc_data_byte to 0, as it could be used uninitialized
later on.

MFC with: r320846
Reported by: Jenkins (powerpc jobs)

New command 'mmcsdcmd' for camcontrol, to allow interacting with SD cards

Submitted by: Ilya Babulin

Fix integer overflow in "camcontrol format"

Reported by: Coverity
CID: 1011426
MFC after: 1 week
Sponsored by: Spectra Logic Corp

Add the ability to rescan or reset devices specified by peripheral
name and unit number in camcontrol(8).

Previously camcontrol(8) only supported rescanning or resetting
devices specified by bus:target:lun. This is because for
rescanning at least, you don't have a peripheral name and unit
number (e.g. da4) for devices that don't exist yet.

That is still the case after this change, but in other cases, when
the device does exist in the CAM EDT (Existing Device Table), we
do a careful lookup of the bus/target/lun if the user supplies a
peripheral name and unit number to find the bus:target:lun and then
issue the requested reset or rescan.

The lookup is done without actually opening the device in question,
since a rescan is often done to make a device go away after it has
been pulled. (This is especially true for busses/controllers, like
parallel SCSI controllers, that don't automatically detect changes
in topology.) Opening a device that is no longer there to
determine the bus/target/lun might result in error recovery actions
when the user really just wanted to make the device go away.

sbin/camcontrol/camcontrol.c:
In dorescan_or_reset(), if the use hasn't specified a
numeric argument, assume he has specified a device. Lookup
the pass(4) instance for that device using the transport
layer CAMGETPASSTHRU ioctl. If that is successful, we can
use the returned bus:target:lun to rescan or reset the
device.

Under the hood, resetting a device using XPT_RESET_DEV is
actually sent via the pass(4) device anyway. But this
provides a way for the user to specify devices in a more
convenient way, and can work on device rescans when the
device is going away, assuming it still exists in the EDT.

sbin/camcontrol/camcontrol.8:
Update the man page for the rescan and reset subcommands
to reflect that you can now use a device name and unit
number with them.

Sponsored by: Spectra Logic
MFC after: 3 days

Fix printing bits above first eight.

Reported by: Coverity
CID: 1372596
MFC after: 1 week

Decode modern PIM flags.

MFC after: 2 weeks

Add task attribute support to camcontrol(8).

Users can use the new generic argument, -Q task_attr, to specify a task
attribute (simple, ordered, head of queue, aca) for the commands issued.
The the default is simple, which works with all SCSI devices that support
tagged queueing.

This will mostly be useful for debugging target behavior in certain
situations.

You can try it out by compiling CTL with CTL_IO_DELAY turned on (in
sys/cam/ctl/ctl_io.h) and then do something like this with one of the CTL
LUNs:

ctladm delay 0:0 -l done -t 10
camcontrol tur da34 -v

And at then before the 10 second timer is up, in another terminal:

camcontrol inquiry da34 -Q ordered -v

The Inquiry should complete just after the TUR completes. Ordinarily
it would complete first because of the delay injection, but because the
task attribute is set to ordered in this case, CTL holds it up until the
previous command has completed.

sbin/camcontrol/camcontrol.c:
Add the new generic argument, -Q, which allows the user to specify
a SCSI task attribute. The user can specify task attributes by
name or numerically.

Add a new task_attr arguments to SCSI sub-functions.

sbin/camcontrol/attrib.c,
sbin/camcontrol/camcontrol.h,
sbin/camcontrol/fwdownload.c,
sbin/camcontrol/modeedit.c,
sbin/camcontrol/persist.c,
sbin/camcontrol/timestamp.c,
sbin/camcontrol/zone.c:
Add the new task_attr argument to SCSI sub-functions.

sbin/camcontrol/camcontrol.8:
Document the new -Q option, and add an example.

Sponsored by: Spectra Logic
MFC after: 1 week

"Buses" is the preferred plural of "bus"

Replace archaic "busses" with modern form "buses."

Intentionally excluded:
* Old/random drivers I didn't recognize
* Old hardware in general
* Use of "busses" in code as identifiers

No functional change.

http://grammarist.com/spelling/buses-busses/

PR: 216099
Reported by: bltsrc at mail.ru
Sponsored by: Dell EMC Isilon

Make `camcontrol cmd ... -i ...` return only valid bytes.

Previously code ignored resid field and returned extra zeroes in case of
data underflow. Now it returns only real bytes received from target.

MFC after: 2 weeks

Make 'camcontrol modepage' support subpages.

MFC after: 2 weeks

Add SCSI REPORT TIMESTAMP and SET TIMESTAMP support.

This adds support to camcontrol(8) and libcam(3) for getting and setting
the time on SCSI protocol drives. This is more commonly found on tape
drives, but is a SPC (SCSI Primary Commands) command, and may be found
on any device that speaks SCSI.

The new camcontrol timestamp subcommand allows getting the current device
time or setting the time to the current system time or any arbitrary time.

sbin/camcontrol/Makefile:
Add timestamp.c.

sbin/camcontrol/camcontrol.8:
Document the new timestamp subcommand.

sbin/camcontrol/camcontrol.c:
Add the timestamp subcommand to camcontrol.

sbin/camcontrol/camcontrol.h:
Add the timestamp() function prototype.

sbin/camcontrol/timestamp.c:
Timestamp setting and reporting functionality.

sys/cam/scsi/scsi_all.c:
Add two new CCB building functions, scsi_set_timestamp() and
scsi_report_timestamp(). Also, add a new helper function,
scsi_create_timestamp().

sys/cam/scsi/scsi_all.h:
Add CDB and parameter data for the the set and report timestamp
commands.

Add function declarations for the new CCB building and helper
functions.

Submitted by: Sam Klopsch
Sponsored by: Spectra Logic
MFC After: 2 weeks

Fix a problem in camcontrol(8) that cropped up with r307684.

In r307684, I changed rescan_or_reset_bus() to bzero stack-allocated CCBs
before sending them to the kernel because there was stack garbage in there
that wound up meaning that bogus CCB flags were set.

While this fixed the 'camcontrol rescan all' case (XPT_DEV_MATCH CCBs were
failing previously), it broke the 'camcontrol rescan 0' (or any other
number) case when INVARIANTS are turned on. Rescanning a single bus
reliably produced an assert in cam_periph_runccb():

panic: cam_periph_runccb: ccb=0xfffff80044ffe000, func_code=0x708, flags=0xffffdde0

The flags values don't make sense from the code. Changing the CCBs in
rescan_or_reset_bus() from stack to heap allocated avoids the problem.

It would be better to understand why userland stack allocated CCBs don't
work properly, since there may be other code that breaks if stack allocated
CCBs don't work.

sbin/camcontrol/camcontrol.c:
In rescan_or_reset_bus(), allocate the CCBs using malloc(3) instead
of on the stack to avoid an assertion in cam_periph_runccb().

MFC after: 3 days
Sponsored by: Spectra Logic

For CCBs allocated on the stack, we need to clear the entire CCB, not just
the header. Otherwise stack garbage can lead to random flags getting set.

This showed up as 'camcontrol rescan all' failing with EINVAL because the
address type wasn't CAM_DATA_VADDR.

sbin/camcontrol/camcontrol.c:
In rescan_or_reset_bus(), bzero the stack-allocated CCBs before
use instead of clearing the body.

MFC after: 3 days
Sponsored by: Spectra Logic

Fix minor copy/paste bug.

Submitted by: Dmitry Luhtionov <dmitryluhtionov@gmail.com>
MFC after: 1 week

Fix a couple of Coverity Unintended sign extension sign extension
defects. When shifting an unsigned byte into the upper 8 bits of
an int and the resulting value is greater than 0x7FFFFFF, the result
will be sign extended when converting to a 64 bit unsigned long.
Fix by casting to (uint64_t) before the shift.

Reported by: Coverity
CID: 1356044, 1356045
Reviewed by: ken

Fix multiple Coverity Out-of-bounds access false postive issues in CAM

The currently used idiom for clearing the part of a ccb after its
header generates one or two Coverity errors for each time it is
used. All instances generate an Out-of-bounds access (ARRAY_VS_SINGLETON)
error because of the treatment of the header as a two element array,
with a pointer to the non-existent second element being passed as
the starting address to bzero(). Some instances also alsp generate
Out-of-bounds access (OVERRUN) errors, probably because the space
being cleared is larger than the sizeofstruct ccb_hdr).

In addition, this idiom is difficult for humans to understand and
it is error prone. The user has to chose the proper struct ccb_*
type (which does not appear in the surrounding code) for the sizeof()
in the length calculation. I found several instances where the
length was incorrect, which could cause either an actual out of
bounds write, or incompletely clear the ccb.

A better way is to write the code to clear the ccb itself starting
at sizeof(ccb_hdr) bytes from the start of the ccb, and calculate
the length based on the specific type of struct ccb_* being cleared
as specified by the union ccb member being used. The latter can
normally be seen in the nearby code. This is friendlier for Coverity
and other static analysis tools because they will see that the
intent is to clear the trailing part of the ccb.

Wrap all of the boilerplate code in a convenient macro that only
requires a pointer to the desired union ccb member (or a pointer
to the union ccb itself) as an argument.

Reported by: Coverity
CID: 1007578, 1008684, 1009724, 1009773, 1011304, 1011306
CID: 1011307, 1011308, 1011309, 1011310, 1011311, 1011312
CID: 1011313, 1011314, 1011315, 1011316, 1011317, 1011318
CID: 1011319, 1011320, 1011321, 1011322, 1011324, 1011325
CID: 1011326, 1011327, 1011328, 1011329, 1011330, 1011374
CID: 1011390, 1011391, 1011392, 1011393, 1011394, 1011395
CID: 1011396, 1011397, 1011398, 1011399, 1011400, 1011401
CID: 1011402, 1011403, 1011404, 1011405, 1011406, 1011408
CID: 1011409, 1011410, 1011411, 1011412, 1011413, 1011414
CID: 1017461, 1018387, 1086860, 1086874, 1194257, 1229897
CID: 1229968, 1306229, 1306234, 1331282, 1331283, 1331294
CID: 1331295, 1331535, 1331536, 1331539, 1331540, 1341623
CID: 1341624, 1341637, 1341638, 1355264, 1355324
Reviewed by: scottl, ken, delphij, imp
MFH: 1 month
Differential Revision: https://reviews.freebsd.org/D6496

Add support for managing Shingled Magnetic Recording (SMR) drives.

This change includes support for SCSI SMR drives (which conform to the
Zoned Block Commands or ZBC spec) and ATA SMR drives (which conform to
the Zoned ATA Command Set or ZAC spec) behind SAS expanders.

This includes full management support through the GEOM BIO interface, and
through a new userland utility, zonectl(8), and through camcontrol(8).

This is now ready for filesystems to use to detect and manage zoned drives.
(There is no work in progress that I know of to use this for ZFS or UFS, if
anyone is interested, let me know and I may have some suggestions.)

Also, improve ATA command passthrough and dispatch support, both via ATA
and ATA passthrough over SCSI.

Also, add support to camcontrol(8) for the ATA Extended Power Conditions
feature set. You can now manage ATA device power states, and set various
idle time thresholds for a drive to enter lower power states.

Note that this change cannot be MFCed in full, because it depends on
changes to the struct bio API that break compatilibity. In order to
avoid breaking the stable API, only changes that don't touch or depend on
the struct bio changes can be merged. For example, the camcontrol(8)
changes don't depend on the new bio API, but zonectl(8) and the probe
changes to the da(4) and ada(4) drivers do depend on it.

Also note that the SMR changes have not yet been tested with an actual
SCSI ZBC device, or a SCSI to ATA translation layer (SAT) that supports
ZBC to ZAC translation. I have not yet gotten a suitable drive or SAT
layer, so any testing help would be appreciated. These changes have been
tested with Seagate Host Aware SATA drives attached to both SAS and SATA
controllers. Also, I do not have any SATA Host Managed devices, and I
suspect that it may take additional (hopefully minor) changes to support
them.

Thanks to Seagate for supplying the test hardware and answering questions.

sbin/camcontrol/Makefile:
Add epc.c and zone.c.

sbin/camcontrol/camcontrol.8:
Document the zone and epc subcommands.

sbin/camcontrol/camcontrol.c:
Add the zone and epc subcommands.

Add auxiliary register support to build_ata_cmd(). Make sure to
set the CAM_ATAIO_NEEDRESULT, CAM_ATAIO_DMA, and CAM_ATAIO_FPDMA
flags as appropriate for ATA commands.

Add a new get_ata_status() function to parse ATA result from SCSI
sense descriptors (for ATA passthrough over SCSI) and ATA I/O
requests.

sbin/camcontrol/camcontrol.h:
Update the build_ata_cmd() prototype

Add get_ata_status(), zone(), and epc().

sbin/camcontrol/epc.c:
Support for ATA Extended Power Conditions features. This includes
support for all features documented in the ACS-4 Revision 12
specification from t13.org (dated February 18, 2016).

The EPC feature set allows putting a drive into a power power mode
immediately, or setting timeouts so that the drive will
automatically enter progressively lower power states after various
idle times.

sbin/camcontrol/fwdownload.c:
Update the firmware download code for the new build_ata_cmd()
arguments.

sbin/camcontrol/zone.c:
Implement support for Shingled Magnetic Recording (SMR) drives
via SCSI Zoned Block Commands (ZBC) and ATA Zoned Device ATA
Command Set (ZAC).

These specs were developed in concert, and are functionally
identical. The primary differences are due to SCSI and ATA
differences. (SCSI is big endian, ATA is little endian, for
example.)

This includes support for all commands defined in the ZBC and
ZAC specs.

sys/cam/ata/ata_all.c:
Decode a number of additional ATA command names in ata_op_string().

Add a new CCB building function, ata_read_log().

Add ata_zac_mgmt_in() and ata_zac_mgmt_out() CCB building
functions. These support both DMA and NCQ encapsulation.

sys/cam/ata/ata_all.h:
Add prototypes for ata_read_log(), ata_zac_mgmt_out(), and
ata_zac_mgmt_in().

sys/cam/ata/ata_da.c:
Revamp the ada(4) driver to support zoned devices.

Add four new probe states to gather information needed for zone
support.

Add a new adasetflags() function to avoid duplication of large
blocks of flag setting between the async handler and register
functions.

Add new sysctl variables that describe zone support and paramters.

Add support for the new BIO_ZONE bio, and all of its subcommands:
DISK_ZONE_OPEN, DISK_ZONE_CLOSE, DISK_ZONE_FINISH, DISK_ZONE_RWP,
DISK_ZONE_REPORT_ZONES, and DISK_ZONE_GET_PARAMS.

sys/cam/scsi/scsi_all.c:
Add command descriptions for the ZBC IN/OUT commands.

Add descriptions for ZBC Host Managed devices.

Add a new function, scsi_ata_pass() to do ATA passthrough over
SCSI. This will eventually replace scsi_ata_pass_16() -- it
can create the 12, 16, and 32-byte variants of the ATA
PASS-THROUGH command, and supports setting all of the
registers defined as of SAT-4, Revision 5 (March 11, 2016).

Change scsi_ata_identify() to use scsi_ata_pass() instead of
scsi_ata_pass_16().

Add a new scsi_ata_read_log() function to facilitate reading
ATA logs via SCSI.

sys/cam/scsi/scsi_all.h:
Add the new ATA PASS-THROUGH(32) command CDB. Add extended and
variable CDB opcodes.

Add Zoned Block Device Characteristics VPD page.

Add ATA Return SCSI sense descriptor.

Add prototypes for scsi_ata_read_log() and scsi_ata_pass().

sys/cam/scsi/scsi_da.c:
Revamp the da(4) driver to support zoned devices.

Add five new probe states, four of which are needed for ATA
devices.

Add five new sysctl variables that describe zone support and
parameters.

The da(4) driver supports SCSI ZBC devices, as well as ATA ZAC
devices when they are attached via a SCSI to ATA Translation (SAT)
layer. Since ZBC -> ZAC translation is a new feature in the T10
SAT-4 spec, most SATA drives will be supported via ATA commands
sent via the SCSI ATA PASS-THROUGH command. The da(4) driver will
prefer the ZBC interface, if it is available, for performance
reasons, but will use the ATA PASS-THROUGH interface to the ZAC
command set if the SAT layer doesn't support translation yet.
As I mentioned above, ZBC command support is untested.

Add support for the new BIO_ZONE bio, and all of its subcommands:
DISK_ZONE_OPEN, DISK_ZONE_CLOSE, DISK_ZONE_FINISH, DISK_ZONE_RWP,
DISK_ZONE_REPORT_ZONES, and DISK_ZONE_GET_PARAMS.

Add scsi_zbc_in() and scsi_zbc_out() CCB building functions.

Add scsi_ata_zac_mgmt_out() and scsi_ata_zac_mgmt_in() CCB/CDB
building functions. Note that these have return values, unlike
almost all other CCB building functions in CAM. The reason is
that they can fail, depending upon the particular combination
of input parameters. The primary failure case is if the user
wants NCQ, but fails to specify additional CDB storage. NCQ
requires using the 32-byte version of the SCSI ATA PASS-THROUGH
command, and the current CAM CDB size is 16 bytes.

sys/cam/scsi/scsi_da.h:
Add ZBC IN and ZBC OUT CDBs and opcodes.

Add SCSI Report Zones data structures.

Add scsi_zbc_in(), scsi_zbc_out(), scsi_ata_zac_mgmt_out(), and
scsi_ata_zac_mgmt_in() prototypes.

sys/dev/ahci/ahci.c:
Fix SEND / RECEIVE FPDMA QUEUED in the ahci(4) driver.

ahci_setup_fis() previously set the top bits of the sector count
register in the FIS to 0 for FPDMA commands. This is okay for
read and write, because the PRIO field is in the only thing in
those bits, and we don't implement that further up the stack.

But, for SEND and RECEIVE FPDMA QUEUED, the subcommand is in that
byte, so it needs to be transmitted to the drive.

In ahci_setup_fis(), always set the the top 8 bits of the
sector count register. We need it in both the standard
and NCQ / FPDMA cases.

sys/geom/eli/g_eli.c:
Pass BIO_ZONE commands through the GELI class.

sys/geom/geom.h:
Add g_io_zonecmd() prototype.

sys/geom/geom_dev.c:
Add new DIOCZONECMD ioctl, which allows sending zone commands to
disks.

sys/geom/geom_disk.c:
Add support for BIO_ZONE commands.

sys/geom/geom_disk.h:
Add a new flag, DISKFLAG_CANZONE, that indicates that a given
GEOM disk client can handle BIO_ZONE commands.

sys/geom/geom_io.c:
Add a new function, g_io_zonecmd(), that handles execution of
BIO_ZONE commands.

Add permissions check for BIO_ZONE commands.

Add command decoding for BIO_ZONE commands.

sys/geom/geom_subr.c:
Add DDB command decoding for BIO_ZONE commands.

sys/kern/subr_devstat.c:
Record statistics for REPORT ZONES commands. Note that the
number of bytes transferred for REPORT ZONES won't quite match
what is received from the harware. This is because we're
necessarily counting bytes coming from the da(4) / ada(4) drivers,
which are using the disk_zone.h interface to communicate up
the stack. The structure sizes it uses are slightly different
than the SCSI and ATA structure sizes.

sys/sys/ata.h:
Add many bit and structure definitions for ZAC, NCQ, and EPC
command support.

sys/sys/bio.h:
Convert the bio_cmd field to a straight enumeration. This will
yield more space for additional commands in the future. After
change r297955 and other related changes, this is now possible.
Converting to an enumeration will also prevent use as a bitmask
in the future.

sys/sys/disk.h:
Define the DIOCZONECMD ioctl.

sys/sys/disk_zone.h:
Add a new API for managing zoned disks. This is very close to
the SCSI ZBC and ATA ZAC standards, but uses integers in native
byte order instead of big endian (SCSI) or little endian (ATA)
byte arrays.

This is intended to offer to the complete feature set of the ZBC
and ZAC disk management without requiring the application developer
to include SCSI or ATA headers. We also use one set of headers
for ioctl consumers and kernel bio-level consumers.

sys/sys/param.h:
Bump __FreeBSD_version for sys/bio.h command changes, and inclusion
of SMR support.

usr.sbin/Makefile:
Add the zonectl utility.

usr.sbin/diskinfo/diskinfo.c
Add disk zoning capability to the 'diskinfo -v' output.

usr.sbin/zonectl/Makefile:
Add zonectl makefile.

usr.sbin/zonectl/zonectl.8
zonectl(8) man page.

usr.sbin/zonectl/zonectl.c
The zonectl(8) utility. This allows managing SCSI or ATA zoned
disks via the disk_zone.h API. You can report zones, reset write
pointers, get parameters, etc.

Sponsored by: Spectra Logic
Differential Revision: https://reviews.freebsd.org/D6147
Reviewed by: wblock (documentation)

camcontrol(8): Fix trival double-free

Reported by: Coverity
CID: 1331223
Sponsored by: EMC / Isilon Storage Division

Add "camcontrol reprobe" subcommand, and implement it for da(4).
This makes it possible to manually force updating capacity data
after the disk got resized. Without it it might be neccessary to
reboot before FreeBSD notices updated disk size under eg VMWare.

Discussed with: imp@
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D6108

Remove logically impossible test in scsidoinquiry(..)

It was already done 4 lines prior and the value of error didn't change

MFC after: 3 days
Reported by: Coverity
CID: 1011236
Sponsored by: EMC / Isilon Storage Division

Revamp camcontrol(8) fwdownload support and add the opcodes subcommand.

The significant changes and bugs fixed here are:

1. Fixed a bug in the progress display code:

When the user's filename is too big, or his terminal width is too
small, the progress code could wind up using a negative number for
the length of the "stars" that it uses to indicate progress.

This negative value was assigned to an unsigned variable, resulting
in a very large positive value.

The result is that we wound up writing garbage from memory to the
user's terminal.

With an 80 column terminal, a file name length of more than 35
characters would generate this problem.

To address this, we now set a minimum progress bar length, and
truncate the user's file name as needed.

This has been tested with large filenames and small terminals, and
at least produces reasonable results. If the terminal is too
narrow, the progress display takes up an additional line with each
update, but this is more user friendly than writing garbage to the
tty.

2. SATA drives connected via a SATA controller didn't have SCSI Inquiry
data populated in struct cam_device. This meant that the code in
fw_get_vendor() in fwdownload.c would try to match a zero-length
vendor ID, and so return the first entry in the vendor table. (Which
used to be HITACHI.) Fixed by grabbing identify data, passing the
identify buffer into fw_get_vendor(), and matching against the model
name.

3. SATA drives connected via a SAS controller do have Inquiry data
populated. The table included a couple of entries -- "ATA ST" and
"ATA HDS", intended to handle Seagate and Hitachi SATA drives attached
via a SAS controller. SCSI to ATA translation layers use a vendor
ID of "ATA" (which is standard), and then the model name from the ATA
identify data as the SCSI product name when they are returning data on
SATA disks. The cam_strmatch code will match the first part of the
string (because the length it is given is the length of the vendor,
"ATA"), and return 0 (i.e. a match). So all SATA drives attached to
a SAS controller would be programmed using the Seagate method
(WRITE BUFFER mode 7) of SCSI firmware downloading.

4. Issue #2 above covered up a bug in fw_download_img() -- if the
maximum packet size in the vendor table was 0, it tried to default
to a packet size of 32K. But then it didn't actually succeed in
doing that, because it set the packet size to the value that was
in the vendor table (0). Now that we actually have ATA attached
drives fall use the VENDOR_ATA case, we need a reasonable default
packet size. So this is fixed to properly set the default packet size.

5. Add support for downloading firmware to IBM LTO drives, and add a
firmware file validation method to make sure that the firmware
file matches the drive type. IBM tape drives include a Load ID and
RU name in their vendor-specific VPD page 0x3. Those should match
the IDs in the header of the firmware file to insure that the
proper firmware file is loaded.

6. This also adds a new -q option to the camcontrol fwdownload
subcommand to suppress informational output. When -q is used in
combination with -y, the firmware upgrade will happen without
prompting and without output except if an error condition occurs.

7. Re-add support for printing out SCSI inquiry information when
asking the user to confirm that they want to download firmware, and
add printing of ATA Identify data if it is a SATA disk. This was
removed in r237281 when support for flashing ATA disks was added.

8. Add a new camcontrol(8) "opcodes" subcommand, and use the
underlying code to get recommended timeout values for drive
firmware downloads.

Many SCSI devices support the REPORT SUPPORTED OPERATION CODES
command, and some support the optional timeout descriptor that
specifies nominal and recommended timeouts for the commands
supported by the device.

The new camcontrol opcodes subcommand allows displaying all
opcodes supported by a drive, information about which fields
in a SCSI CDB are actually used by a given SCSI device, and the
nominal and recommended timeout values for each command.

Since firmware downloads can take a long time in some devices, and
the time varies greatly between different types of devices, take
advantage of the infrastructure used by the camcontrol opcodes
subcommand to determine the best timeout to use for the WRITE
BUFFER command in SCSI device firmware downloads.

If the device recommends a timeout, it is likely to be more
accurate than the default 50 second timeout used by the firmware
download code. If the user specifies a timeout, it will override
the default or device recommended timeout. If the device doesn't
support timeout descriptors, we fall back to the default.

9. Instead of downloading firmware to SATA drives behind a SAS controller
using WRITE BUFFER, use the SCSI ATA PASS-THROUGH command to compose
an ATA DOWNLOAD MICROCODE command and it to the drive. The previous
version of this code attempted to send a SCSI WRITE BUFFER command to
SATA drives behind a SAS controller. Although that is part of the
SAT-3 spec, it doesn't work with the parameters used with LSI
controllers at least.

10.Add a new mechanism for making common ATA passthrough and
ATA-behind-SCSI passthrough commands.

The existing camcontrol(8) ATA command mechanism checks the device
type on every command executed. That works fine for individual
commands, but is cumbersome for things like a firmware download
that send a number of commands.

The fwdownload code detects the device type up front, and then
sends the appropriate commands.

11.In simulation mode (-s), if the user specifies the -v flag, print out
the SCSI CDB or ATA registers that would be sent to the drive. This will
aid in debugging any firmware download issues.

sbin/camcontrol/fwdownload.c:
Add a device type to the fw_vendor structure, so that we can
specify different download methods for different devices from the
same vendor. In this case, IBM hard drives (from when they
still made hard drives) and tape drives.

Add a tur_status field to the fw_vendor structure so that we can
specify whether the drive to be upgraded should be ready, not
ready, or whether it doesn't matter. Add the corresponding
capability in fw_download_img().

Add comments describing each of the vendor table fields.

Add HGST and SmrtStor to the supported SCSI vendors list.

In fw_get_vendor(), look at ATA identify data if we have a SATA
device to try to identify what the drive vendor is.

Add IBM firmware file validation. This gets VPD page 0x3, and
compares the Load ID and RU name in the page to the values
included in the header. The validation code will refuse to load
a firmware file if the values don't match. This does allow the
user to attempt a downgrade; whether or not it succeeds will
likely depend on the drive settings.

Add a -q option, and disable all informative output
(progress bars, etc.) when this is enabled.

Re-add the inquiry in the confirmation dialog so the user has
a better idea of which device he is talking to. Add support for
displaying ATA identify data.

Don't automatically disable confirmation in simulation (-s) mode.
This allows the user to see the inquiry or identify data in the
dialog, and see exactly what they would see when the command
actually runs. Also, in simulation mode, if the user specifies
the -v flag, print out the SCSI CDB or ATA registers that would
be sent to the drive. This will aid in debugging any firmware
download issues.

Add a timeout field and timeout type to the firmware download
vendor table. This allows specifying a default timeout and allows
specifying whether we should attempt to probe for a recommended
timeout from the drive.

Add a new fuction, fw_get_timeout(), that will determine
which timeout to use for the WRITE BUFFER command. If the
user specifies a timeout, we always use that. Otherwise,
we will use the drive recommended timeout, if available,
and fall back to the default when a drive recommended
timeout isn't available.

When we prompt the user, tell him what timeout we're going
to use, and the source of the timeout.

Revamp the way SATA devices are handled.

In fwdownload(), use the new get_device_type() function to
determine what kind of device we're talking to.

Allow firmware downloads to any SATA device, but restrict
SCSI downloads to known devices. (The latter is not a
change in behavior.)

Break out the "ready" check from fw_download_img() into a
new subfunction, fw_check_device_ready(). This sends the
appropriate command to the device in question -- a TEST
UNIT READY or an IDENTIFY. The IDENTIFY for SATA devices
a SAT layer is done using the SCSI ATA PASS-THROUGH
command.

Use the new build_ata_cmd() function to build either a SCSI or
ATA I/O CCB to issue the DOWNLOAD MICROCODE command to SATA
devices. build_ata_cmd() figures looks at the devtype argument
and fills in the correct CCB type and CDB or ATA registers.

Revamp the vendor table to remove the previous
vendor-specific ATA entries and use a generic ATA vendor
placeholder. We currently use the same method for all ATA
drives, although we may have to add vendor-specific
behavior once we test this with more drives.

sbin/camcontrol/progress.c:
In progress_draw(), make barlength a signed value so that
we can easily detect a negative value.

If barlength (the length of the progress bar) would wind up
negative due to a small TTY width or a large filename,
set the bar length to the new minimum (10 stars) and
truncate the user's filename. We will truncate it down to
0 characters if necessary.

Calculate a new prefix_len variable (user's filename length)
and use it as the precision when printing the filename.

sbin/camcontrol/camcontrol.c:
Implement a new camcontrol(8) subcommand, "opcodes". The
opcodes subcommand allows displaying the entire list of
SCSI commands supported by a device, or details on an
individual command. In either case, it can display
nominal and recommended timeout values.

Add the scsiopcodes() function, which calls the new
scsigetopcodes() function to fetch opcode data from a
drive.

Add two new functions, scsiprintoneopcode() and
scsiprintopcodes(), which print information about one
opcode or all opcodes, respectively.

Remove the get_disk_type() function. It is no longer used.

Add a new function, dev_has_vpd_page(), that fetches the
supported INQUIRY VPD list from a device and tells the
caller whether the requested VPD page is available.

Add a new function, get_device_type(), that returns a more
precise device type than the old get_disk_type() function.
The get_disk_type() function only distinguished between
SCSI and ATA devices, and SATA devices behind a SCSI to ATA
translation layer were considered to be "SCSI".

get_device_type() offers a third type, CC_DT_ATA_BEHIND_SCSI.
We need to know this to know whether to attempt to send ATA
passthrough commands. If the device has the ATA
Information VPD page (0x89), then it is an ATA device
behind a SCSI to ATA translation layer.

Remove the type argument from the fwdownload() subcommand.

Add a new function, build_ata_cmd(), that will take one set
of common arguments and build either a SCSI or ATA I/O CCB,
depending on the device type passed in.

sbin/camcontrol/camcontrol.h:
Add a prototype for scsigetopcodes().

Add a new enumeration, camcontrol_devtype.

Add prototypes for dev_has_vpd_page(), get_device_type()
and build_ata_cmd().

Remove the type argument from the fwdownload() subcommand.

sbin/camcontrol/camcontrol.8
Explain that the fwdownload subcommand will use the drive
recommended timeout if available, and that the user can
override the timeout.

Document the new opcodes subcommand.

Explain that we will attempt to download firmware to any
SATA device.

Document supported SCSI vendors, and models tested if known.

Explain the commands used to download firmware for the
three different drive and controller combinations.

Document that the -v flag in simulation mode for the fwdownload
subcommand will print out the SCSI CDBs or ATA registers that would
be used.

sys/cam/scsi/scsi_all.h:
Add new bit definitions for the one opcode descriptor for
the REPORT SUPPORTED OPCODES command.

Add a function prototype for scsi_report_supported_opcodes().

sys/cam/scsi/scsi_all.c:
Add a new CDB building function, scsi_report_supported_opcodes().

Sponsored by: Spectra Logic
MFC after: 1 week

Add -b to the devlist usage info, forgotten in r260059.

PR: 195094
Submitted by: robin.hahling@gw-computing.net
MFC after: 1 week

Add support for reading MAM attributes to camcontrol(8) and libcam(3).

MAM is Medium Auxiliary Memory and is most commonly found as flash
chips on tapes.

This includes support for reading attributes and decoding most
known attributes, but does not yet include support for writing
attributes or reporting attributes in XML format.

libsbuf/Makefile:
Add subr_prf.c for the new sbuf_hexdump() function. This
function is essentially the same function.

libsbuf/Symbol.map:
Add a new shared library minor version, and include the
sbuf_hexdump() function.

libsbuf/Version.def:
Add version 1.4 of the libsbuf library.

libutil/hexdump.3:
Document sbuf_hexdump() alongside hexdump(3), since it is
essentially the same function.

camcontrol/Makefile:
Add attrib.c.

camcontrol/attrib.c:
Implementation of READ ATTRIBUTE support for camcontrol(8).

camcontrol/camcontrol.8:
Document the new 'camcontrol attrib' subcommand.

camcontrol/camcontrol.c:
Add the new 'camcontrol attrib' subcommand.

camcontrol/camcontrol.h:
Add a function prototype for scsiattrib().

share/man/man9/sbuf.9:
Document the existence of sbuf_hexdump() and point users to
the hexdump(3) man page for more details.

sys/cam/scsi/scsi_all.c:
Add a table of known attributes, text descriptions and
handler functions.

Add a new scsi_attrib_sbuf() function along with a number
of other related functions that help decode attributes.

scsi_attrib_ascii_sbuf() decodes ASCII format attributes.

scsi_attrib_int_sbuf() decodes binary format attributes, and
will pass them off to scsi_attrib_hexdump_sbuf() if they're
bigger than 8 bytes.

scsi_attrib_vendser_sbuf() decodes the vendor and drive
serial number attribute.

scsi_attrib_volcoh_sbuf() decodes the Volume Coherency
Information attribute that LTFS writes out.

sys/cam/scsi/scsi_all.h:
Add a number of attribute-related structure definitions and
other defines.

Add function prototypes for all of the functions added in
scsi_all.c.

sys/kern/subr_prf.c:
Add a new function, sbuf_hexdump(). This is the same as
the existing hexdump(9) function, except that it puts the
result in an sbuf.

This also changes subr_prf.c so that it can be compiled in
userland for includsion in libsbuf.

We should work to change this so that the kernel hexdump
implementation is a wrapper around sbuf_hexdump() with a
statically allocated sbuf with a drain. That will require
a drain function that goes to the kernel printf() buffer
that can take a non-NUL terminated string as input.
That is because an sbuf isn't NUL-terminated until it is
finished, and we don't want to finish it while we're still
using it.

We should also work to consolidate the userland hexdump and
kernel hexdump implemenatations, which are currently
separate. This would also mean making applications that
currently link in libutil link in libsbuf.

sys/sys/sbuf.h:
Add the prototype for sbuf_hexdump(), and add another copy
of the hexdump flag values if they aren't already defined.

Ideally the flags should be defined in one place but the
implemenation makes it difficult to do properly. (See
above.)

Sponsored by: Spectra Logic Corporation
MFC after: 1 week

Remove unused variables reported by clang.

Differential Revision: D2688
Reviewed by: rodrigc, mav

Add camcontrol subcommands to control APM and AAM levels.

MFC after: 1 week
Sponsored by: iXsystems, Inc.

Make ATA power management commands to work on SCSI HBAs via PASS THROUGH.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

Make sure that the flags for the XPT_DEV_ADVINFO CCB are initialized
properly.

If there is garbage in the flags field, it can sometimes include a
set CDAI_FLAG_STORE flag, which may cause either an error or
perhaps result in overwriting the field that was intended to be
read.

sys/cam/cam_ccb.h:
Add a new flag to the XPT_DEV_ADVINFO CCB, CDAI_FLAG_NONE,
that callers can use to set the flags field when no store
is desired.

sys/cam/scsi/scsi_enc_ses.c:
In ses_setphyspath_callback(), explicitly set the
XPT_DEV_ADVINFO flags to CDAI_FLAG_NONE when fetching the
physical path information. Instead of ORing in the
CDAI_FLAG_STORE flag when storing the physical path, set
the flags field to CDAI_FLAG_STORE.

sys/cam/scsi/scsi_sa.c:
Set the XPT_DEV_ADVINFO flags field to CDAI_FLAG_NONE when
fetching extended inquiry information.

sys/cam/scsi/scsi_da.c:
When storing extended READ CAPACITY information, set the
XPT_DEV_ADVINFO flags field to CDAI_FLAG_STORE instead of
ORing it into a field that isn't initialized.

sys/dev/mpr/mpr_sas.c,
sys/dev/mps/mps_sas.c:
When fetching extended READ CAPACITY information, set the
XPT_DEV_ADVINFO flags field to CDAI_FLAG_NONE instead of
setting it to 0.

sbin/camcontrol/camcontrol.c:
When fetching a device ID, set the XPT_DEV_ADVINFO flags
field to CDAI_FLAG_NONE instead of 0.

sys/sys/param.h:
Bump __FreeBSD_version to 1100061 for the new XPT_DEV_ADVINFO
CCB flag, CDAI_FLAG_NONE.

Sponsored by: Spectra Logic
MFC after: 1 week

Improve camcontrol(8) handling of drive defect data.

This includes a new summary mode (-s) for camcontrol defects that
quickly tells the user the most important thing: how many defects
are in the requested list. The actual location of the defects is
less important.

Modern drives frequently have more than the 8191 defects that can
be reported by the READ DEFECT DATA (10) command. If they don't
have that many grown defects, they certainly have more than 8191
defects in the primary (i.e. factory) defect list.

The READ DEFECT DATA (12) command allows for longer parameter
lists, as well as indexing into the list of defects, and so allows
reporting many more defects.

This has been tested with HGST drives and Seagate drives, but
does not fully work with Seagate drives. Once I have a Seagate
spec I may be able to determine whether it is possible to make it
work with Seagate drives.

scsi_da.h: Add a definition for the new long block defect
format.

Add bit and mask definitions for the new extended
physical sector and bytes from index defect
formats.

Add a prototype for the new scsi_read_defects() CDB
building function.

scsi_da.c: Add a new scsi_read_defects() CDB building function.
camcontrol(8) was previously composing CDBs manually.
This is long overdue.

camcontrol.c: Revamp the camcontrol defects subcommand. We now
go through multiple stages in trying to get defect
data off the drive while avoiding various drive
firmware quirks.

We start off by requesting the defect header with
the 10 byte command. If we're in summary mode (-s)
and the drive reports fewer defects than can be
represented in the 10 byte header, we're done.
Otherwise, we know that we need to issue the
12 byte command if the drive reports the maximum
number of defects.

If we're in summary mode, we're done if we get a
good response back when asking for the 12 byte header.

If the user has asked for the full list, then we
use the address descriptor index field in the 12
byte CDB to step through the list in 64K chunks.
64K is small enough to work with most any ancient
or modern SCSI controller.

Add support for printing the new long block defect
format, as well as the extended physical sector and
bytes from index formats. I don't have any drives
that support the new formats.

Add a hexadecimal output format that can be turned
on with -X.

Add a quiet mode (-q) that can be turned on with
the summary mode (-s) to just print out a number.

Revamp the error detection and recovery code for
the defects command to work with HGST drives.

Call the new scsi_read_defects() CDB building
function instead of rolling the CDB ourselves.

Pay attention to the residual from the defect list
request when printing it out, so we don't run off
the end of the list.

Use the new scsi_nv library routines to convert
from strings to numbers and back.

camcontrol.8: Document the new defect formats (longblock, extbfi,
extphys) and command line options (-q, -s, -S and
-X) for the defects subcommand.

Explain a little more about what drives generally
do and don't support.

Sponsored by: Spectra Logic
MFC after: 1 week

Attempt to report a better error if sanitize is not supported

MFC after: 1 month

Update CAM CCB accounting for the new status quo.

devq_openings counter lost its meaning after allocation queues has gone.
held counter is still meaningful, but problematic to update due to separate
locking of CCB allocation and queuing.

To fix that replace devq_openings counter with allocated counter. held is
now calculated on request as difference between number of allocated, queued
and active CCBs.

MFC after: 1 month

Add persistent reservation support to camcontrol(8).

camcontrol(8) now supports a new 'persist' subcommand that allows users to
issue SCSI PERSISTENT RESERVE IN / OUT commands.

sbin/camcontrol/Makefile:
Add persist.c.

sbin/camcontrol/persist.c:
New persistent reservation support for camcontrol(8).

We have support for all known operation modes for PERSISTENT RESERVE
IN and PERSISTENT RESERVE OUT.
exceptions noted above.

sbin/camcontrol/camcontrol.8:
Document the new 'persist' subcommand.

In the section on the Transport ID (-I) option, explain what
Transport IDs for each protocol should look like. At some point
some of this information could probably get moved off in a
separate man page, either on Transport IDs alone or a man page
documenting the Transport ID parsing code.

Add a number of examples of persistent reservation commands.
Persistent Reservations are complex enough that the average user
probably won't be able to get the commands exactly right by just
reading the man page. These examples show a few basic and
advanced examples of how to use persistent reservations.

sbin/camcontrol/camcontrol.h:
Move the definition for camcontrol_optret here, so we can use it
for the persistent reservation code.

Add a definition for the new scsipersist() function.

sbin/camcontrol/camcontrol.c:
Add 'persist' to the list of subcommands.

Document 'persist' in the help text.

sys/cam/scsi/scsi_all.c:
Add the scsi_persistent_reserve_in() and
scsi_persistent_reserve_out() CCB building functions.

Add a new function, scsi_transportid_sbuf(). This takes a
SCSI Transport ID (documented in SPC-4), and prints it to
an sbuf(9). There are some transports (like ATA, USB, and
SSA) for which there is no transport defined. We need to
come up with a reasonable thing to do if we're presented
with a Transport ID that claims to be for one of those
protocols.

Add new routines scsi_get_nv() and scsi_nv_to_str().

These functions do a table lookup to go between a string and an
integer. There are lots of table lookups needed in the
persistent reservation code in camcontrol(8).

Add a new function, scsi_parse_transportid(), along with leaf node
functions to parse:
FC, 1394 and SAS (scsi_parse_transportid_64bit())
iSCSI (scsi_parse_transportid_iscsi())
SPI (scsi_parse_transportid_spi())
RDMA (scsi_parse_transportid_rdma())
PCIe (scsi_parse_transportid_sop())

Transport IDs. Given a string with the general form proto,id these
functions create a SCSI Transport ID structure.

sys/cam/scsi/scsi_all.h:
Update the various persistent reservation data structures to
SPC4r36l, but also rename some fields that were previously
obsolete with the proper names from older SCSI specs. This
allows using older, obsolete persistent reservation types when
desired.

Add function prototypes for the new persistent reservation CCB
building functions.

Add a data strucure for the READ FULL STATUS service action
of the PERSISTENT RESERVE IN command.

Add Transport ID structures for all protocols described in SPC-4.

Add a new series of SCSI_PROTO_XXX definitions, and
redefine other defines in terms of these new definitions.

Add a prototype for scsi_transportid_sbuf().

Change a couple of "obsolete" persistent reservation data
structure fields into something more meaningful, based on
what the field was called when it was defined in the spec.
(e.g. SPC, SPC-2, etc.)

Create a new define, SPRI_MAX_LEN, for the maximum allocation
length allowed for the PERSISTENT RESERVE IN command.

Add data structures and enumerations for the new name/value
translation functions.

Add data structures for SCSI over PCIe Routing IDs.

Bring the PERSISTENT RESERVE OUT Register and Move parameter list
structure (struct scsi_per_res_out_parms) up to date with SPC-4.

Add a data structure for the transport IDs that can optionally be
appended to the basic PERSISTENT RESERVE OUT parameter list.

Move SCSI protocol macro definitions out of the VPD page 0x83
definition and combine them with the more up to date protocol
definitions higher in the file.

Add function prototypes for scsi_nv_to_str(), scsi_get_nv(),
scsi_parse_transportid_64bit(), scsi_parse_transportid_spi(),
scsi_parse_transportid_rdma(), scsi_parse_transportid_iscsi(),
scsi_parse_transportid_sop(), and scsi_parse_transportid().

Sponsored by: Spectra Logic Corporation
MFC after: 1 week

Add information about supported NCQ functionality to camcontrol identify.

MFC after: 2 weeks

Replace several instances of -1 with appropriate CAM_*_WILDCARD and types.

It was equal before r259397, but for good or bad, not any more for LUNs.

This change fixes at least CAM debugging.

getopt returns an int, not a char, so use the correct data type for
the return value. Fixes powerpc tinderbox.

MFC after: 2 days

Add the '-b' flag to 'camcontrol devlist'. This prints only the existing
buses and their parent sims, useful for creating a sim->bus->device map.

Obtained from: Netflix
MFC after: 3 days

printf() specifier updates to CAM to handle either 32-bit or 64-bit lun_id_t.

MFC after: 2 weeks

Fix an inverted check for the master user in "camcontrol security -U".

PR: bin/182703
Submitted by: Scott Burns <scott@bqinternet.com>
Approved by: re (gjb)
MFC after: 3 days

Add camcontrol support for the SCSI sanitize command

Reviewed by: ken, mjacob (eariler version)
Sponsored by: Netapp

Add missing newlines to Fibre Channel attributes output.

Fix build after r251654.

Use CAM_DIR_NONE for requests without data.
Wrong values there confuse some drivers.

MFC after: 1 week

sbin/camcontrol/camcontrol.c
If an expander returns 0x00 (no device attached) in the ATTACHED DEVICE
field of the SMP DISCOVER response, ignore the value of ATTACHED SAS
ADDRESS, because it is invalid. Some expanders zero out the address
when the attached device is removed, but others do not. Section
9.4.3.10 of the SAS Protocol Layer 2 revision 04b does not require them
to do so.

Approved by: ken (mentor)
MFC after: 3 weeks

Adds Host Protected Area (HPA) support for ATA disks to camcontrol

Reviewed by: mav
Approved by: pjd (mentor)
MFC after: 2 weeks

Prevent the creation of an unused variable.

We're only interested in the enumeration fields; we don't want to create
a variable to store them.

MFC after: 1 week

Adds security options to camcontrol this includes the ability to secure erase
disks such as SSD's

Adds the ability to run ATA commands via the SCSI ATA Pass-Through(16) comand

Reviewed by: mav
Approved by: pjd (mentor)
MFC after: 2 weeks

More -Wmissing-variable-declarations fixes.

In addition to adding `static' where possible:

- bin/date: Move `retval' into extern.h to make it visible to date.c.
- bin/ed: Move globally used variables into ed.h.
- sbin/camcontrol: Move `verbose' into camcontrol.h and fix shadow warnings.
- usr.bin/calendar: Remove unneeded variables.
- usr.bin/chat: Make `line' local instead of global.
- usr.bin/elfdump: Comment out unneeded function.
- usr.bin/rlogin: Use _Noreturn instead of __dead2.
- usr.bin/tset: Pull `Ospeed' into extern.h.
- usr.sbin/mfiutil: Put global variables in mfiutil.h.
- usr.sbin/pkg: Remove unused `os_corres'.
- usr.sbin/quotaon, usr.sbin/repquota: Remove unused `qfname'.

Check the return value of sbuf_finish().

Avoid segfault in the 'smpphylist' subcommand.
Initialize devlist.dev_queue tail queue early enough before its any
potential traversal in freebusdevlist() when in smpphylist error path.

Reported by: Pavel Polyakov <bsd kobyla org> (on irc)
Reviewed by: ken
MFC after: 5 days

In sbin/camcontrol/camcontrol.c, function smpcmd(), initialize the
'error' variable to zero, to avoid returning garbage in several cases.

This fixes the following clang 3.2 warnings:

sbin/camcontrol/camcontrol.c:4634:8: warning: variable 'error' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized]
if (amt_written == -1) {
^~~~~~~~~~~~~~~~~
sbin/camcontrol/camcontrol.c:4656:10: note: uninitialized use occurs here
return (error);
^~~~~

sbin/camcontrol/camcontrol.c:4619:7: warning: variable 'error' is used uninitialized whenever 'if' condition is true [-Wsometimes-uninitialized]
if (fd_response == 0) {
^~~~~~~~~~~~~~~~
sbin/camcontrol/camcontrol.c:4656:10: note: uninitialized use occurs here
return (error);
^~~~~

sbin/camcontrol/camcontrol.c:4617:6: warning: variable 'error' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized]
if (((ccb->ccb_h.status & CAM_STATUS_MASK) == CAM_REQ_CMP)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
sbin/camcontrol/camcontrol.c:4656:10: note: uninitialized use occurs here
return (error);
^~~~~

MFC after: 1 week

Change 'camcontrol defects' to first probe a drive to find out how much
defect information it has before grabbing the full defect list.

This works around a bug with some Hitachi drives that generate data overrun
errors when they are asked for more defect data than they have.

The change is done in a spec-compliant way, so it should have no negative
impact on drives that don't have this issue.

This is based on work originally done at Sandvine.

scsi_da.h: Add a define for the maximum amount of data that can be
contained in a defect list.

camcontrol.c: Update the readdefects() function to issue an initial
command to determine the length of the defect list, and
then use that length in the request for the full defect
list.

camcontrol.8: Add a note that some drives will report 0 defects available
if you don't request either the PLIST or GLIST.

Submitted by: Mark Johnston <markjdb@gmail.com> (original version)
MFC after: 3 days

Update the 'fwdownload' command to also flash disks connected over an
ATA/SATA transport. The detection logic is automatic, so it should Just
Work. While here, also improve the progress meter that is displayed
during firmware download.

Submitted by: Alistair Crooks
Obtained from: Netflix, Inc.
MFC after: 3 days

Add -p argument for `camcontrol debug` to allow enabling CAM_DEBUG_PROBE
added at r208911.

Rewrite enabling NCQ for SATA devices in a way more alike to SCSI TCQ.
This allows to control it with `camcontrol negotiate adaX -T (en|dis)able`
on the fly, same as for SCSI devices.

Sponsored by: iXsystems, Inc.

Add missing flag enable when certain arguments are parsed

PR: bin/163053
Submitted by: Peter <pmc@citylink.dinoex.sub.org>
Approved by: cperciva
MFC after: 1 week

MFprojects/zfsd:
- Add low-level support for SATA Enclosure Management Bridge (SEMB)
devices -- SATA equivalents of the SCSI SES/SAF-TE devices.
- Add some utility functions for SCSI SAF-TE devices access.

Sponsored by: iXsystems, Inc.

Add CAM infrastructure to allow reporting when a drive's long read capacity
data changes.

cam_ccb.h: Add a new advanced information type, CDAI_TYPE_RCAPLONG,
for long read capacity data.

cam_xpt_internal.h:
Add a read capacity data pointer and length to struct cam_ed.

cam_xpt.c: Free the read capacity buffer when a device goes away.
While we're here, make sure we don't leak memory for other
malloced fields in struct cam_ed.

scsi_all.c: Update the scsi_read_capacity_16() to take a uint8_t * and
a length instead of just a pointer to the parameter data
structure. This will hopefully make this function somewhat
immune to future changes in the parameter data.

scsi_all.h: Add some extra bit definitions to struct
scsi_read_capacity_data_long, and bump up the structure
size to the full size specified by SBC-3.

Change the prototype for scsi_read_capacity_16().

scsi_da.c: Register changes in read capacity data with the transport
layer. This allows the transport layer to send out an
async notification to interested parties. Update the
dasetgeom() API.

Use scsi_extract_sense_len() instead of
scsi_extract_sense().

scsi_xpt.c: Add support for the new CDAI_TYPE_RCAPLONG advanced
information type.

Make sure we set the physpath pointer to NULL after freeing
it. This allows blindly freeing it in the struct cam_ed
destructor.

sys/param.h: Bump __FreeBSD_version from 1000005 to 1000006 to make it
easier for third party drivers to determine that the read
capacity data async notification is available.

camcontrol.c,
mptutil/mpt_cam.c:
Update these for the new scsi_read_capacity_16() argument
structure.

Sponsored by: Spectra Logic

Fix warning when compiling with gcc46:
error: variable 'freq' set but not used
error: variable 'mode_pars' set but not used

Reviewed by: mav
Approved by: dim
MFC after: 3 days

Spelling fixes for sbin/

In sbin/camcontrol/camcontrol.c, fix a few warnings about format strings
not being literals.

MFC after: 1 week

Add firmware update support for SCSI devices.

Firmware can be reprogrammed on devices from Hitachi, HP, IBM, Plextor,
Quantum, and Seagate. At least one device from each manufacturer has
been tested with some version of this code, and it has been used to
update thousands of drives so far.

The man page suggests having a backup of the drive's data, and the
operation must be confirmed, either interactively or on the command
line. (This is the same as the confirmation on the format command.)

This work is largely derived from fwprog.c by Andre Albsmeier.

Submitted by: Nima Misaghian
Sponsored by: Sandvine Incorporated
MFC after: 3 months

Add missing static keywords for global variables to tools in sbin/.

These tools declare global variables without using the static keyword,
even though their use is limited to a single C-file, or without placing
an extern declaration of them in the proper header file.

Add descriptor sense support to CAM, and honor sense residuals properly in
CAM.

Desriptor sense is a new sense data format that originated in SPC-3. Among
other things, it allows for an 8-byte info field, which is necessary to
pass back block numbers larger than 4 bytes.

This change adds a number of new functions to scsi_all.c (and therefore
libcam) that abstract out most access to sense data.

This includes a bump of CAM_VERSION, because the CCB ABI has changed.
Userland programs that use the CAM pass(4) driver will need to be
recompiled.

camcontrol.c: Change uses of scsi_extract_sense() to use
scsi_extract_sense_len().

Use scsi_get_sks() instead of accessing sense key specific
data directly.

scsi_modes: Update the control mode page to the latest version (SPC-4).

scsi_cmds.c,
scsi_target.c: Change references to struct scsi_sense_data to struct
scsi_sense_data_fixed. This should be changed to allow the
user to specify fixed or descriptor sense, and then use
scsi_set_sense_data() to build the sense data.

ps3cdrom.c: Use scsi_set_sense_data() instead of setting sense data
manually.

cam_periph.c: Use scsi_extract_sense_len() instead of using
scsi_extract_sense() or accessing sense data directly.

cam_ccb.h: Bump the CAM_VERSION from 0x15 to 0x16. The change of
struct scsi_sense_data from 32 to 252 bytes changes the
size of struct ccb_scsiio, but not the size of union ccb.
So the version must be bumped to prevent structure
mis-matches.

scsi_all.h: Lots of updated SCSI sense data and other structures.

Add function prototypes for the new sense data functions.

Take out the inline implementation of scsi_extract_sense().
It is now too large to put in a header file.

Add macros to calculate whether fields are present and
filled in fixed and descriptor sense data

scsi_all.c: In scsi_op_desc(), allow the user to pass in NULL inquiry
data, and we'll assume a direct access device in that case.

Changed the SCSI RESERVED sense key name and description
to COMPLETED, as it is now defined in the spec.

Change the error recovery action for a number of read errors
to prevent lots of retries when the drive has said that the
block isn't accessible. This speeds up reconstruction of
the block by any RAID software running on top of the drive
(e.g. ZFS).

In scsi_sense_desc(), allow for invalid sense key numbers.
This allows calling this routine without checking the input
values first.

Change scsi_error_action() to use scsi_extract_sense_len(),
and handle things when invalid asc/ascq values are
encountered.

Add a new routine, scsi_desc_iterate(), that will call the
supplied function for every descriptor in descriptor format
sense data.

Add scsi_set_sense_data(), and scsi_set_sense_data_va(),
which build descriptor and fixed format sense data. They
currently default to fixed format sense data.

Add a number of scsi_get_*() functions, which get different
types of sense data fields from either fixed or descriptor
format sense data, if the data is present.

Add a number of scsi_*_sbuf() functions, which print
formatted versions of various sense data fields. These
functions work for either fixed or descriptor sense.

Add a number of scsi_sense_*_sbuf() functions, which have a
standard calling interface and print the indicated field.
These functions take descriptors only.

Add scsi_sense_desc_sbuf(), which will print a formatted
version of the given sense descriptor.

Pull out a majority of the scsi_sense_sbuf() function and
put it into scsi_sense_only_sbuf(). This allows callers
that don't use struct ccb_scsiio to easily utilize the
printing routines. Revamp that function to handle
descriptor sense and use the new sense fetching and
printing routines.

Move scsi_extract_sense() into scsi_all.c, and implement it
in terms of the new function, scsi_extract_sense_len().
The _len() version takes a length (which should be the
sense length - residual) and can indicate which fields are
present and valid in the sense data.

Add a couple of new scsi_get_*() routines to get the sense
key, asc, and ascq only.

mly.c: Rename struct scsi_sense_data to struct
scsi_sense_data_fixed.

sbp_targ.c: Use the new sense fetching routines to get sense data
instead of accessing it directly.

sbp.c: Change the firewire/SCSI sense data transformation code to
use struct scsi_sense_data_fixed instead of struct
scsi_sense_data. This should be changed later to use
scsi_set_sense_data().

ciss.c: Calculate the sense residual properly. Use
scsi_get_sense_key() to fetch the sense key.

mps_sas.c,
mpt_cam.c: Set the sense residual properly.

iir.c: Use scsi_set_sense_data() instead of building sense data by
hand.

iscsi_subr.c: Use scsi_extract_sense_len() instead of grabbing sense data
directly.

umass.c: Use scsi_set_sense_data() to build sense data.

Grab the sense key using scsi_get_sense_key().

Calculate the sense residual properly.

isp_freebsd.h: Use scsi_get_*() routines to grab asc, ascq, and sense key
values.

Calculate and set the sense residual.

MFC after: 3 days
Sponsored by: Spectra Logic Corporation

Add support for SATA Enclosure Management Bridge (SEMB).

Fix WWN printing in `camcontrol identify` output.

Approved by: re (kib)
MFC after: 1 week

Lay groundwork in CAM for recording and reporting physical path and
other device attributes stored in the CAM Existing Device Table (EDT).
This includes some infrastructure requried by the enclosure services
driver to export physical path information.

Make the CAM device advanced info interface accept store requests.

sys/cam/scsi/scsi_all.c:
sys/cam/scsi/scsi_all.h:
- Replace scsi_get_sas_addr() with a scsi_get_devid() which takes
a callback that decides whether to accept a particular descriptor.
Provide callbacks for NAA IEEE Registered addresses and for SAS
addresses, replacing the old function. This is needed because
the old function doesn't work for an enclosure address for a SAS
device, which is not flagged as a SAS address, but is NAA IEEE
Registered. It may be worthwhile merging this interface with the
devid match interface.
- Add a few more defines for some device ID fields.

sbin/camcontrol/camcontrol.c:
- Update for the CCB_DEV_ADVINFO interface change.

cam/cam_xpt_internal.h:
- Add the new fields for the physical path string to the CAM EDT.
cam/cam_ccb.h:
- Rename CCB_GDEV_ADVINFO to simply CCB_DEV_ADVINFO, and the ccb
structure to ccb_dev_advinfo.
- Add a flag that changes this CCB's action to store, rather than
the default, retrieve.
- Add a new buffer type, CDAI_TYPE_PHYS_PATH, for the new CAM EDT
physpath field.
- Remove the never-implemented transport & proto flags.
cam/cam_xpt.c:
cam/cam_xpt.h:
- Add xpt_getattr(), which provides a wrapper for fetching a device's
attribute using the GEOM strings as key. This method currently
supports "GEOM::ident" and "GEOM::physpath".

Submitted by: will
Reviewed by : gibbs

Extend the XPT_DEV_MATCH api to allow a device search by device ID.
As far as the API is concerned, device ID is a binary blob to be
interpreted by the transport layer. The SCSI implementation assumes
it is an array of VPD device ID descriptors.

sys/cam/cam_ccb.h:
Create a new structure, device_id_match_pattern, and
update the XPT_DEV_MATCH datastructures and flags so
that this pattern type can be used.

sys/cam/cam_xpt.c:
- A single pattern matching on both inquiry data and device
ID is invalid. Report any violators.
- Pass device ID match requests through to the new routine
scsi_devid_match(). The direct call of a SCSI routine is
a layering violation, but no worse than the one a few
lines up that checks inquiry data. Defer cleaning this
up until our future, larger, rototilling of CAM.
- Zero out cam_ed and cam_et nodes on allocation. Prior to
this change, device_id_len and device_id were not inialized,
preventing proper detection of the presence of this
information.

sys/cam/scsi/scsi_all.c:
sys/cam/scsi/scsi_all.h:
Add the scsi_match_devid() routine.

Add a helper function for extracting peripherial driver names

sys/cam/cam_periph.c:
sys/cam/cam_periph.h:
Add the cam_periph_list() method which fills an sbuf
with a comma delimited list of the peripheral instances
associated with a given CAM path.

Add a helper functions for SCSI commands used by the SES driver.

sys/cam/scsi/scsi_all.c:
sys/cam/scsi/scsi_all.h:
Add structure definitions and csio filling functions for
the receive diagnostic results and send diagnostic commands.

Misc CAM XPT cleanups.

sys/cam/cam_xpt.c:
Broadcast AC_FOUND_DEVICE and AC_PATH_REGISTERED
events at the time async event handlers are attached
even when registering just for events on a partitular
SIM. Previously, you had to register for these
events on all SIMs in the system in order to get
the initial broadcast even though subsequent device
and path arrivals would be delivered.

sys/cam/cam_xpt.c:
Remove SIM mutex held asserts from path accessors.
CAM paths are reference counted and it is this
reference count, not the sim mutex, that garantees
they are stable.

Sponsored by: Spectra Logic Corporation

When calling XPT_REL_SIMQ to ajust number of openings, do not try to really
release device. We haven't frozen the device before and attempt to release
it will at least cause warning message from kernel.

Fix a few issues related to the XPT_GDEV_ADVINFO CCB.

camcontrol.c: In buildbusdevlist(), don't attempt to get call
getdevid() for an unconfigured device, even when the
verbose flag is set. The cam_open_btl() call will almost
certainly fail.

Probe for the buffer size when issuing the XPT_GDEV_ADVINFO
CCB. Probing for the buffer size first helps us avoid
allocating the maximum buffer size when it really may not
be necessary. This also helps avoid errors from
cam_periph_mapmem() if we attempt to map more than MAXPHYS.

cam_periph.c: In cam_periph_mapmem(), if the XPT_GDEV_ADVINFO CCB
shows a bufsiz of 0, we don't have anything to map,
so just return.

Also, set the maximum mapping size to MAXPHYS
instead of DFLTPHYS for XPT_GDEV_ADVINFO CCBs,
since they don't actually go down to the hardware.

scsi_pass.c: Don't bother mapping the buffer in XPT_GDEV_ADVINFO
CCBs if bufsiz is 0.

Add Serial Management Protocol (SMP) passthrough support to CAM.

This includes support in the kernel, camcontrol(8), libcam and the mps(4)
driver for SMP passthrough.

The CAM SCSI probe code has been modified to fetch Inquiry VPD page 0x00
to determine supported pages, and will now fetch page 0x83 in addition to
page 0x80 if supported.

Add two new CAM CCBs, XPT_SMP_IO, and XPT_GDEV_ADVINFO. The SMP CCB is
intended for SMP requests and responses. The ADVINFO is currently used to
fetch cached VPD page 0x83 data from the transport layer, but is intended
to be extensible to fetch other types of device-specific data.

SMP-only devices are not currently represented in the CAM topology, and so
the current semantics are that the SIM will route SMP CCBs to either the
addressed device, if it contains an SMP target, or its parent, if it
contains an SMP target. (This is noted in cam_ccb.h, since it will change
later once we have the ability to have SMP-only devices in CAM's topology.)

smp_all.c,
smp_all.h: New helper routines for SMP. This includes
SMP request building routines, response parsing
routines, error decoding routines, and structure
definitions for a number of SMP commands.

libcam/Makefile: Add smp_all.c to libcam, so that SMP functionality
is available to userland applications.

camcontrol.8,
camcontrol.c: Add smp passthrough support to camcontrol. Several
new subcommands are now available:

'smpcmd' functions much like 'cmd', except that it
allows the user to send generic SMP commands.

'smprg' sends the SMP report general command, and
displays the decoded output. It will automatically
fetch extended output if it is available.

'smppc' sends the SMP phy control command, with any
number of potential options. Among other things,
this allows the user to reset a phy on a SAS
expander, or disable a phy on an expander.

'smpmaninfo' sends the SMP report manufacturer
information and displays the decoded output.

'smpphylist' displays a list of phys on an
expander, and the CAM devices attached to those
phys, if any.

cam.h,
cam.c: Add a status value for SMP errors
(CAM_SMP_STATUS_ERROR).

Add a missing description for CAM_SCSI_IT_NEXUS_LOST.

Add support for SMP commands to cam_error_string().

cam_ccb.h: Rename the CAM_DIR_RESV flag to CAM_DIR_BOTH. SMP
commands are by nature bi-directional, and we may
need to support bi-directional SCSI commands later.

Add the XPT_SMP_IO CCB. Since SMP commands are
bi-directional, there are pointers for both the
request and response.

Add a fill routine for SMP CCBs.

Add the XPT_GDEV_ADVINFO CCB. This is currently
used to fetch cached page 0x83 data from the
transport later, but is extensible to fetch many
other types of data.

cam_periph.c: Add support in cam_periph_mapmem() for XPT_SMP_IO
and XPT_GDEV_ADVINFO CCBs.

cam_xpt.c: Add support for executing XPT_SMP_IO CCBs.

cam_xpt_internal.h: Add fields for VPD pages 0x00 and 0x83 in struct
cam_ed.

scsi_all.c: Add scsi_get_sas_addr(), a function that parses
VPD page 0x83 data and pulls out a SAS address.

scsi_all.h: Add VPD page 0x00 and 0x83 structures, and a
prototype for scsi_get_sas_addr().

scsi_pass.c: Add support for mapping buffers in XPT_SMP_IO and
XPT_GDEV_ADVINFO CCBs.

scsi_xpt.c: In the SCSI probe code, first ask the device for
VPD page 0x00. If any VPD pages are supported,
that page is required to be implemented. Based on
the response, we may probe for the serial number
(page 0x80) or device id (page 0x83).

Add support for the XPT_GDEV_ADVINFO CCB.

sys/conf/files: Add smp_all.c.

mps.c: Add support for passing in a uio in mps_map_command(),
so we can map a S/G list at once.

Add support for SMP passthrough commands in
mps_data_cb(). SMP is a special case, because the
first buffer in the S/G list is outbound and the
second buffer is inbound.

Add support for warning the user if the busdma code
comes back with more buffers than will work for the
command. This will, for example, help the user
determine why an SMP command failed if busdma comes
back with three buffers.

mps_pci.c: Add sys/uio.h.

mps_sas.c: Add the SAS address and the parent handle to the
list of fields we pull from device page 0 and cache
in struct mpssas_target. These are needed for SMP
passthrough.

Add support for the XPT_SMP_IO CCB. For now, this
CCB is routed to the addressed device if it supports
SMP, or to its parent if it does not and the parent
does. This is necessary because CAM does not
currently support SMP-only nodes in the topology.

Make SMP passthrough support conditional on
__FreeBSD_version >= 900026. This will make it
easier to MFC this change to the driver without
MFCing the CAM changes as well.

mps_user.c: Un-staticize mpi_init_sge() so we can use it for
the SMP passthrough code.

mpsvar.h: Add a uio and iovecs into struct mps_command for
SMP passthrough commands.

Add a cm_max_segs field to struct mps_command so
that we can warn the user if busdma comes back with
too many segments.

Clear the cm_reply when a command gets freed. If
it is not cleared, reply frames will eventually get
freed into the pool multiple times and corrupt the
pool. (This fix is from scottl.)

Add a prototype for mpi_init_sge().

sys/param.h: Bump __FreeBSD_version to 900026 for the for the
inclusion of the XPT_GDEV_ADVINFO and XPT_SMP_IO
CAM CCBs.

When doing a camcontrol rescan all or a camcontrol reset all, use the wildcard
path id for enumerating the available busses. Previously camcontrol was
implicitly passing 0 as the first path id, which meant that if bus 0 was not
present camcontrol would fail with EINVAL instead of rescanning/resetting any
busses that were present.

Approved by: emaste (mentor)
MFC after: 1 week

r214781 caused the timer value to be rounded down, so that if the user asked
for 59 minutes 30 was sent to the drive. The timer value is now always
rounded up.

Reported by: mav

Fix standby timer calculation: the timer was being set 30 minutes later
than the user requested.
Also, 21 minutes is encoded as 252 and 22-29 minutes cannot be encoded
so must be rounded up to 30.

PR: bin/151871

Do some whitespace and `identify` output cleanup.

Submitted by: arundel
MFC after: 2 weeks

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Stop disallowing device nodes to be passed to camcontrol(8) since libcam
already allows both device names and nodes to be specified.

Reviewed by: avg

Export PCI IDs of ATA/SATA controllers through CAM and ata(4) layers to
GEOM. This information needed for proper soft-RAID's on-disk metadata
reading and writing.

MFC r207499:
Make SATA XPT negotiate and enable some additional SATA features, such as:
- device initiated power management (some devices support only this way);
- Automatic Partial to Slumber Transition (more power saving);
- DMA auto-activation (expected to slightly improve performance).
More features could be added later, when hardware supports.

MFC r207498:
Add -d and -f arguments to `camcontrol cmd`, to execute DMA ATA commands.

Make SATA XPT negotiate and enable some additional SATA features, such as:
- device initiated power management (some devices support only this way);
- Automatic Partial to Slumber Transition (more power saving);
- DMA auto-activation (expected to slightly improve performance).
More features could be added later, when hardware supports.

Add -d and -f arguments to `camcontrol cmd`, to execute DMA ATA commands.

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

MFC r204195:
Improve output for controllers that doesn't report SATA speed.

Improve output for controllers that doesn't report SATA speed.

MFC r203376, r203384:
- Give ATA/SATA SIMs info about ATAPI packet size, supported by device.
- Make ATA XPT to reject longer SCSI CDBs then supported by device, or
any SCSI CDBs, if device doesn't support ATAPI.

MFC r202694:
- Add -v argument to `camcontrol identify` command. It makes camcontrol
print full identify data block.
- Improve identify result view a bit and add TRIM support.

- Give ATA/SATA SIMs info about ATAPI packet size, supported by device.
- Make ATA XPT to reject longer SCSI CDBs then supported by device, or
any SCSI CDBs, if device doesn't support ATAPI.

- Add -v argument to `camcontrol identify` command. It makes camcontrol
print full identify data block.
- Improve identify result view and add TRIM support.

MFC r199747, r199799, r199821:
- Extend XPT-SIM transfer settings control API. Now it allows to report to
SATA SIM number of tags supported by each device, implement ATA mode and
SATA revision negotiation for both SATA and PATA SIMs.
- Make ahci(4) and siis(4) to use submitted maximum tag number, when
scheduling requests. It allows to support NCQ on devices with lower tags
count then controller supports.
- Make PMP driver to report attached devices connection speeds.
- Implement ATA mode negotiation between user settings, device and
controller capabilities.
- Improve ATA mode/SATA revision control.

MFp4:
Improve ATA mode/SATA revision control.

MFp4:
- Extend XPT-SIM transfer settings control API. Now it allows to report to
SATA SIM number of tags supported by each device, implement ATA mode and
SATA revision negotiation for both SATA and PATA SIMs.
- Make ahci(4) and siis(4) to use submitted maximum tag number, when
scheduling requests. It allows to support NCQ on devices with lower tags
count then controller supports.
- Make PMP driver to report attached devices connection speeds.
- Implement ATA mode negotiation between user settings, device and
controller capabilities.

MFC r199079, r199101:
Add support for ATA Power Management.

MFC r198897:
- Add support for sector size > 512 bytes and physical sector of several
logical sectors, introduced by ATA-7 specification.
- Remove some obsoleted code.

MFC r198709:
Sync connection speed reporting with kernel.
Report speed in identify command, same as done by inquiry.

MFC r197545:
Fethch more information from IDENTIFY result.

MFC r197419:
Reduce code duplication.

Fix variable type.

Add support for ATA Power Management.

MFp4:
- Add support for sector size > 512 bytes and physical sector of several
logical sectors, introduced by ATA-7 specification.
- Remove some obsoleted code.

MFp4:
Sync connection speed reporting with kernel.
Report speed in identify command, same as done by inquiry.

Fethch more information from IDENTIFY result.

MFp4:
Reduce code duplication.

MFC r196831:
Add to `camcontrol cmd` support for sending arbitrary ATA commands.

It could be used for broad range of tasks, such as configuring drive
power management, caching, security and any other features and tasks,
not supported by existing drivers.

Approved by: re (ATA-CAM blanket)

Add to `camcontrol cmd` support for sending arbitrary ATA commands.

It could be used for broad range of tasks, such as configuring drive
power management modes, caching, security and any other features and tasks,
not supported by existing drivers.

MFC r196657:
ATA_FLUSHCACHE is a 28bit format command, not 48.

MFC r196658:
Improve camcontrol ATA support:
- Tune protocol version reporting,
- Add supported DMA/PIO modes reporting.
- Fix IDENTIFY request for ATAPI devices.
- Remove confusing "-" for NCQ status.

MFC r196659:
Short ATA command format has 28bit address, not 36bit.
Rename ata_36bit_cmd() into ata_28bit_cmd(), while it didn't become legacy.

Approved by: re (ATA-CAM blanket)

Short ATA command format has 28bit address, not 36bit.
Rename ata_36bit_cmd() into ata_28bit_cmd(), while it didn't become legacy.

MFC after: 2 days

MFp4:
- Tune protocol version reporting,
- Add supported DMA/PIO modes reporting.
- Fix IDENTIFY for ATAPI devices.
- Remove confusing "-" for NCQ status.

Fix alignment issue with ATA IDENTIFY structure.

Approved by: re

Separate the parallel scsi knowledge out of the core of the XPT, and
modularize it so that new transports can be created.

Add a transport for SATA

Add a periph+protocol layer for ATA

Add a driver for AHCI-compliant hardware.

Add a maxio field to CAM so that drivers can advertise their max
I/O capability. Modify various drivers so that they are insulated
from the value of MAXPHYS.

The new ATA/SATA code supports AHCI-compliant hardware, and will override
the classic ATA driver if it is loaded as a module at boot time or compiled
into the kernel. The stack now support NCQ (tagged queueing) for increased
performance on modern SATA drives. It also supports port multipliers.

ATA drives are accessed via 'ada' device nodes. ATAPI drives are
accessed via 'cd' device nodes. They can all be enumerated and manipulated
via camcontrol, just like SCSI drives. SCSI commands are not translated to
their ATA equivalents; ATA native commands are used throughout the entire
stack, including camcontrol. See the camcontrol manpage for further
details. Testing this code may require that you update your fstab, and
possibly modify your BIOS to enable AHCI functionality, if available.

This code is very experimental at the moment. The userland ABI/API has
changed, so applications will need to be recompiled. It may change
further in the near future. The 'ada' device name may also change as
more infrastructure is completed in this project. The goal is to
eventually put all CAM busses and devices until newbus, allowing for
interesting topology and management options.

Few functional changes will be seen with existing SCSI/SAS/FC drivers,
though the userland ABI has still changed. In the future, transports
specific modules for SAS and FC may appear in order to better support
the topologies and capabilities of these technologies.

The modularization of CAM and the addition of the ATA/SATA modules is
meant to break CAM out of the mold of being specific to SCSI, letting it
grow to be a framework for arbitrary transports and protocols. It also
allows drivers to be written to support discrete hardware without
jeopardizing the stability of non-related hardware. While only an AHCI
driver is provided now, a Silicon Image driver is also in the works.
Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware
is possible and encouraged. Help with new transports is also encouraged.

Submitted by: scottl, mav
Approved by: re

- Import the HEAD csup code which is the basis for the cvsmode work.

Add SCSI READ CAPACITY support to camcontrol. The new 'readcap' subcommand
will automatically issue the 16 byte verison of read capacity if the device
in question is larger than 2TB.

There are also a number of output options here (last block, number of
blocks, human readable) that should meet most needs, and also aid in
scripting.

Approved by: re (bmah)
MFC after: 1 week

typo: s/tranport layer/transport layer/

MFC after: 1 week

It's confusing to say that "Command Queueing Supported" just based
upon the scsi flag validity field. Instead, just say "Command Queueing Enabled"
when it is- otherwise remain mute.

2nd and final commit that moves us to CAM_NEW_TRAN_CODE
as the default.

Reviewed by multitudes.

Implement 'camcontrol reportluns'. This allows users to send the SCSI
REPORT LUNS command to a device.

camcontrol.[c8]: Implement reportluns. This tries to print the LUNs
out in a reasonable format. Only the periph
addressing method has been tested, since very little
hardware that I know of supports the other methods.

scsi_all.[ch]: Revamp the report luns CDB structure and helper
functions. This constitutes a little bit of an API
change, but since the old CDB length was 10 bytes,
and the REPORT LUNS CDB length is actually 12 bytes,
it's clear that no one was using this API in the
first place.

MFC After: 1 week

Add "report only" functionality to 'camcontrol format', so users can get a
report on the status of a format already running on a drive.

Fix status reporting for 'camcontrol format'. This was broken in rev 1.34
of camcontrol.c, almost 4 years ago!

Submitted by: joerg (most of the reportonly changes)
MFC after: 3 days

Fix a couple of camcontrol issues that popped up on sparc64:

- bzero the CCB header in getdevtree() and set the path properly, to
avoid having random garbage in the CCB header.
- if the lun isn't specified in a device specifier, it should default to
0, not whatever random data happens to be in the lun variable.
- move the prototype for getdevtree() out from under #ifndef MINIMALISTIC,
since it is used in both cases.

Submitted by: Marius Strobl <marius@alchemy.franken.de> (mostly)
MFC After: 2 weeks

Make this WARNS=6 clean by:
1: add 'const' to char * where needed;
2: mark unused variables with __unused;
3: remove double prototypes for mode_edit and mode_list.
4: moves the global variables 'bus', 'target', and 'lun' into
the main function and protect them with #ifndef MINIMALISTIC,
5: renames 3 variable in order not to shadow other things
index -> indx -- in modepage_dump since index is a function
from <strings.h.>
arglist -> arglst -- in the function parse_btl since arglist
is also a global variable
convertend -> convertend2 -- in the function editentry_set
since that name is used two times within the function.
6: cast 0xffffffff in the macro RESOLUTION_MAX(size) to (int)
since it is unsigned otherwise.

Tested by: make universe
Approved by: ken

Use __FBSDID.

Make camcontrol WARNS=2 clean.

Approved by: ken

Add PERIPH and XPT debug options to camcontrol. This makes all CAM debug
options tunable from userland.

Approved by: ken
MFC after: 3 days

Split cam_argmask and move "commands" into cam_cmdmask. This addresses
the issue of not having any free bits left for additional commands.

Approved by: ken
MFC after: 4 weeks

Add 'camcontrol load' as a complement to 'camcontrol eject'.

Approved by: ken
MFC after: 4 weeks

Take care of WARNS=3 warnings by using unsigned/signed variable
as needed.

Approved by: ken, sheldonh (mentor)

Remove unused variable. camcontrol is now WARNS=2 clean on i386.

Approved by: ken, sheldonh (mentor)

Fix a bzero length calculation. sizeof(struct ccb_getdev) should have been
sizeof(struct ccb_getdevstats).

MFC after: 3 days

Remove 'register' keyword.
It does not help modern compilers, and some may take some hit from it.
(I also found several functions that listed *every* of its 10 local vars with
"register" -- just how many free registers do people think machines have?)

Correct an out of date device node name. We do not have /dev/rsd0.ctl
nowadays.

Spotted by: Sergey Osokin <osa@freebsd.org.ru>
Reviewed by: ken, ru
Approved by: ken, ru
MFC after: 1 week

By popular demand, also include the "devlist" subcommand into the set
of commands available in the boot floppy environment.

MFC after: 1 week

Add 'camcontrol rescan all' and 'camcontrol reset all' functionality to
camcontrol.

This enables rescanning all busses or resetting all busses in a system.
The current implementation is not the ideal way to do it -- the ideal way
to do it would be for the transport layer to handle wildcarded busses on
bus rescan and reset operations. The current implementation enumerates all
the busses and sends a rescan or reset CCB individually. Handling this
behavior in the transport layer will happen later.

Reviewed by: imp
Tested by: joerg
MFC after: 1 week

Provide an option to make camcontrol `minimalistic': if the (env/make)
variable RELEASE_BUILD_FIXIT is defined, a camcontrol binary will be
built that only knows the "rescan" and "reset" subcommands. The
resulting code is small enough to still fit onto the boot floppy.

Reviewed by: ken
MFC after: 1 week

Fix breakage in 'camcontrol defects' introduced in rev 1.34 (the new error
recovery code) back in March, 2001.

In effect, this brain-o would cause 'camcontrol defects' to always return
an error.

Pointed out by: joerg
Tested by: mdodd

Use STD{ERR,IN,OUT}_FILENO instead of their numeric values. The
definitions are more readable, and it's possible that they're
more portable to pathalogical platforms.

Submitted by: David Hill <david@phobia.ms>

Rewrite of the CAM error recovery code.

Some of the major changes include:

- The SCSI error handling portion of cam_periph_error() has
been broken out into a number of subfunctions to better
modularize the code that handles the hierarchy of SCSI errors.
As a result, the code is now much easier to read.

- String handling and error printing has been significantly
revamped. We now use sbufs to do string formatting instead
of using printfs (for the kernel) and snprintf/strncat (for
userland) as before.

There is a new catchall error printing routine,
cam_error_print() and its string-based counterpart,
cam_error_string() that allow the kernel and userland
applications to pass in a CCB and have errors printed out
properly, whether or not they're SCSI errors. Among other
things, this helped eliminate a fair amount of duplicate code
in camcontrol.

We now print out more information than before, including
the CAM status and SCSI status and the error recovery action
taken to remedy the problem.

- sbufs are now available in userland, via libsbuf. This
change was necessary since most of the error printing code
is shared between libcam and the kernel.

- A new transfer settings interface is included in this checkin.
This code is #ifdef'ed out, and is primarily intended to aid
discussion with HBA driver authors on the final form the
interface should take. There is example code in the ahc(4)
driver that implements the HBA driver side of the new
interface. The new transfer settings code won't be enabled
until we're ready to switch all HBA drivers over to the new
interface.

src/Makefile.inc1,
lib/Makefile: Add libsbuf. It must be built before libcam,
since libcam uses sbuf routines.

libcam/Makefile: libcam now depends on libsbuf.

libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the
sbuf sources from sys/kern.

bsd.libnames.mk: Add LIBSBUF.

camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically
linked, we can't depend on the dynamic linker
to pull in libsbuf.

camcontrol.c: Use cam_error_print() instead of checking for
CAM_SCSI_STATUS_ERROR on every failed CCB.

sbuf.9: Change the prototypes for sbuf_cat() and
sbuf_cpy() so that the source string is now a
const char *. This is more in line wth the
standard system string functions, and helps
eliminate warnings when dealing with a const
source buffer.

Fix a typo.

cam.c: Add description strings for the various CAM
error status values, as well as routines to
look up those strings.

Add new cam_error_string() and
cam_error_print() routines for userland and
the kernel.

cam.h: Add a new CAM flag, CAM_RETRY_SELTO.

Add enumerated types for the various options
available with cam_error_print() and
cam_error_string().

cam_ccb.h: Add new transfer negotiation structures/types.

Change inq_len in the ccb_getdev structure to
be "reserved". This field has never been
filled in, and will be removed when we next
bump the CAM version.

cam_debug.h: Fix typo.

cam_periph.c: Modularize cam_periph_error(). The SCSI error
handling part of cam_periph_error() is now
in camperiphscsistatuserror() and
camperiphscsisenseerror().

In cam_periph_lock(), increase the reference
count on the periph while we wait for our lock
attempt to succeed so that the periph won't go
away while we're sleeping.

cam_xpt.c: Add new transfer negotiation code. (ifdefed
out)

Add a new function, xpt_path_string(). This
is a string/sbuf analog to xpt_print_path().

scsi_all.c: Revamp string handing and error printing code.
We now use sbufs for much of the string
formatting code. More of that code is shared
between userland the kernel.

scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly
useful in the first place.

Add a new error action, SS_REQSENSE. (Send a
request sense and then retry the command.)
This is useful when the controller hasn't
performed autosense for some reason.

Change the default actions around a bit.

scsi_cd.c,
scsi_da.c,
scsi_pt.c,
scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection
timeouts shouldn't be covered by a sense flag.

scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO.

Get rid of the last vestiges of a read/write
interface.

libkern/bsearch.c,
sys/libkern.h,
conf/files: Add bsearch.c, which is needed for some of the
new table lookup routines.

aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if
CAM_NEW_TRAN_CODE is defined.

sbuf.h,
subr_sbuf.c: Add the appropriate #ifdefs so sbufs can
compile and run in userland.

Change sbuf_printf() to use vsnprintf()
instead of kvprintf(), which is only available
in the kernel.

Change the source string for sbuf_cpy() and
sbuf_cat() to be a const char *.

Add __BEGIN_DECLS and __END_DECLS around
function prototypes since they're now exported
to userland.

kdump/mkioctls: Include stdio.h before cam.h since cam.h now
includes a function with a FILE * argument.

Submitted by: gibbs (mostly)
Reviewed by: jdp, marcel (libsbuf makefile changes)
Reviewed by: des (sbuf changes)
Reviewed by: ken

Properly check malloc(3) return values

Approved by: ken

Fix format string warnings

Adjust 'camcontrol negotiate -v' so it prints out the initiator ID from the
Path Inquiry CCB.

optreset is declared in unistd.h, so we don't need to declare it here again.

Reviews by: ken

Include new modepage list (-l) option in the verbose usage.

This is an overhaul of the mode page handling in camcontrol as well as
related patches. These include:
* Mode page editting can be scripted. This involves two
things: first, if stdin is not a tty, changes are read from
stdin rather than invoking $EDITOR. Second, and more
importantly, not all modepage entries must be included in the
change set. This means that camcontrol can now gracefully handle
more intrusive editting from the $EDITOR, including removal or
rearrangement of lines. It also means that you can do stuff
like:
# echo "WCE: 1" | camcontrol modepage da3 -m 8 -e
# newfs /dev/da3
# echo "WCE: 0" | camcontrol modepage da3 -m 8 -e
* Range-checking on user-supplied input values. modeedit.c now
uses the field width specifiers to determine the maximum
allowable value for a field. If the user enters a value larger
than the maximum, it clips the value to the max and warns the
user. This also involved patching cam_cmdparse.c to be more
consistent with regards to the "count" parameter to arg_put
(previously is was the length of strings and 1 for all integral
types). The cam_cdbparse(3) man page was also updated to reflect
the revised semantics.
* In the process, I removed the 64 entry limit on mode pages (not
that we were even close to hitting that limit). This was a nice
side-effect of the other changes.
* Technically, the new mode editting functionality allows editting
of character array entries in mode pages (type 'c' or 'z'),
however since buff_encode doesn't grok them it is currently
useless.
* Camcontrol gained two new options related to mode pages: -l and
-b. The former lists all available mode pages for a given
device. The latter forces mode page display in binary format
(the default when no mode page definition was found in
scsi_modes).
* Added support for mode page names to scsi_modes. Allows names to
be displayed alongside mode numbers in the mode page
listing. Updated scsi_modes to use the new functionality. This
also adds the semicolon into the scsi_modes syntax as an
optional mode page definition terminator. This is needed to name
pages without providing a page format definition.
* Updated scsi_all.h to include a structure describing mode page
headers.
* Added $FreeBSD$ line to scsi_modes.

Inspired by: dwhite
Reviewed by: ken

Print the correct speed when speed is less than 1MB/s.

Fixed in cam_xpt.c, r1.81

Submitted by: Gerd Knops <gerti@bitart.com>

When printing out the transfer rate display for 'camcontrol inquiry',
use the current setting for tagged queueing when deciding whether or not to
print "Tagged Queueing Enabled" instead of using the device's actual
capabilities.

This is more consistent with the rest of the transfer rate display, which
relies on current settings, and is more consistent with the way we display
things on boot.

Reported by: Gustavo Vieira Goncalves Coelho Rios <kernel@tdnet.com.br>
Reviewed by: mjacob

Eliminate the default device name (da) and unit number (0) in camcontrol.

This may break some scripts, but with the number of ways users can damage
a system with this tool, it's important to make sure they specify which
device they want to talk to.

Suggested by: joerg

Send 'camcontrol help' usage output to stdout instead of stderr, so it
can be viewed more easily with a pager.

Regular (i.e. short) usage output is still sent to stderr.

PR: bin/12358
Submitted by: Christian Weisgerber <naddy@mips.rhein-neckar.de>

Implement a new camcontrol function, 'camcontrol format'.

libcam/Makefile: Add scsi_da.c to libcam for the new
scsi_format_unit() function.

camcontrol.8: Update the man page for the new format
functionality, and take out the examples section
describing how to do it with 'camcontrol cmd'.

camcontrol.c: New format functionality. Note that unlike the
rest of the camcontrol subcommands, this one is
interactive by default. Because of the potential
destructiveness of the format command, I thought
it necessary to get confirmation from the user
before spamming a disk. You can disable the
interactive behavior, and the status meter with
command line arguments.

scsi_da.c: Add the new scsi_format_unit() cdb building
function and use #ifdef _KERNEL to make this file
compile in both the kernel and userland. The
format unit function is currently only defined in
the non-kernel case, because nothing in the kernel
is using it. If that changes, it should be
un-ifdefed and compiled in both cases.

scsi_da.h: New function declaration, CDB structure and format
data structures.

Thanks to Nick Hibma for providing some valuable input on these changes.

Typo ("frequencey").

Fix 'camcontrol inquiry'. The inquiry data structure changes (increased to
256 bytes) caused it to break on many devices.

The SCSI spec says that for commands with 8-bit length fields, a value of 0
means 256 bytes. As it turns out, many devices don't deal with that
properly. Some interpret the 0 as 0, and return no data. Others return
more than 256 bytes of data, and cause an overrun.

The fix is to tell the device we've only allocated SHORT_INQUIRY_LENGTH (36
bytes) of inquiry data, instead of sizeof(struct scsi_inquiry_data).

camcontrol.c: Change inq_len in the call to scsi_inquiry() to
SHORT_INQUIRY_LENGTH, and add a long comment
explaining the reason for the change.

scsi_all.h: Add a comment above the definitinon of
SHORT_INQUIRY_LENGTH alerting people that it is
both the initial probe inquiry length, and the
minimum amount of data needed for scsi_print_inquiry()
to function.

scsi_all.c: Add a comment about SHORT_INQUIRY_LENGTH being the
minimum amount of data needed for
scsi_print_inquiry() to function.

Reviewed by: gibbs
Approved by: jkh
Reported by: "John W. DeBoskey" <jwd@unx.sas.com>

Fix a couple of typos in the 'camcontrol negotiate' output.

Submitted by: T-Om <tom@iki.fi>
Approved by: jkh

use XPT_GDEV_STATS- XPT_GDEV_TYPE was deprecated a while back

Fix Bus Device Resets from userland, and specifically from camcontrol.

camcontrol was setup to use the old scheme of going through the xpt(4)
device, which never worked properly (and has been disabled for a while).

camcontrol now sends BDRs through the pass(4) device, and XPT_RESET_DEV
CCBs are put on the device queue in the transport layer, as they should be.

Submitted by: luoqi
Reviewed by: ken

Restore alphabetical order of command line options parsing.

Requested by: ken

Debug flags -T and -S were reversed.

Fix two rather embarrassing bugs in camcontrol. The first caused the
CAM_PASS_ERR_RECOVER flag to be set unconditionally on READ DEFECTS
commands, and also caused the CAM_DIR_IN flag to not be set. This was
the cause of all of the "camcontrol defects doesn't work with my NCR
controller" bugs.

The second prevented camcontrol negotiate from negotiating any bus width
other than 8 bits.

Submitted by: groudier@club-internet.fr (Gerard Roudier)

-Wall cleanups, handle unknown cases.

Submitted by: billf, ken
Reviewed by: ken

$Id$ -> $FreeBSD$

For vendor-specific commands sent using the 'camcontrol cmd' facility,
use the passed in CDB length, not 1.

Submitted by: Jean-Marc Zucconi <jmz@FreeBSD.ORG>

Add a new device specification syntax to camcontrol. It is now possible to
do things like:

camcontrol tur da5
or
camcontrol tur 1:2:0
or
camcontrol tur 1:2

These changes are fully backwards compatible with the original device
specification syntax (-n dev -u unit), so it is possible to use either
method to specify a device now.

The device specification changes do not affect the rescan, reset or debug
commands, since by design, those commands work on a bus or bus:target:lun
basis only.

Also, shorten the default usage statement so that it fits in a 24 column
terminal. The full usage statement is still available by using the "help"
"-h" or "-?" arguments to camcontrol.

Submitted by: Joerg Wunsch <joerg_wunsch@interface-business.de>

Add a number of interrelated CAM feature enhancements and bug fixes.

NOTE: These changes will require recompilation of any userland
applications, like cdrecord, xmcd, etc., that use the CAM passthrough
interface. A make world is recommended.

camcontrol.[c8]:
- We now support two new commands, "tags" and "negotiate".

- The tags commands allows users to view the number of tagged
openings for a device as well as a number of other related
parameters, and it allows users to set tagged openings for
a device.

- The negotiate command allows users to enable and disable
disconnection and tagged queueing, set sync rates, offsets
and bus width. Note that not all of those features are
available for all controllers. Only the adv, ahc, and ncr
drivers fully support all of the features at this point.
Some cards do not allow the setting of sync rates, offsets and
the like, and some of the drivers don't have any facilities to
do so. Some drivers, like the adw driver, only support enabling
or disabling sync negotiation, but do not support setting sync
rates.

- new description in the camcontrol man page of how to format a disk
- cleanup of the camcontrol inquiry command
- add support in the 'devlist' command for skipping unconfigured devices if
-v was not specified on the command line.
- make use of the new base_transfer_speed in the path inquiry CCB.
- fix CCB bzero cases

cam_xpt.c, cam_sim.[ch], cam_ccb.h:

- new flags on many CCB function codes to designate whether they're
non-immediate, use a user-supplied CCB, and can only be passed from
userland programs via the xpt device. Use these flags in the transport
layer and pass driver to categorize CCBs.

- new flag in the transport layer device matching code for device nodes
that indicates whether a device is unconfigured

- bump the CAM version from 0x10 to 0x11

- Change the CAM ioctls to use the version as their group code, so we can
force users to recompile code even when the CCB size doesn't change.

- add + fill in a new value in the path inquiry CCB, base_transfer_speed.
Remove a corresponding field from the cam_sim structure, and add code to
every SIM to set this field to the proper value.

- Fix the set transfer settings code in the transport layer.

scsi_cd.c:

- make some variables volatile instead of just casting them in various
places
- fix a race condition in the changer code
- attach unless we get a "logical unit not supported" error. This should
fix all of the cases where people have devices that return weird errors
when they don't have media in the drive.

scsi_da.c:

- attach unless we get a "logical unit not supported" error

scsi_pass.c:

- for immediate CCBs, just malloc a CCB to send the user request in. This
gets rid of the 'held' count problem in camcontrol tags.

scsi_pass.h:

- change the CAM ioctls to use the CAM version as their group code.

adv driver:

- Allow changing the sync rate and offset separately.

adw driver

- Allow changing the sync rate and offset separately.

aha driver:

- Don't return CAM_REQ_CMP for SET_TRAN_SETTINGS CCBs.

ahc driver:

- Allow setting offset and sync rate separately

bt driver:

- Don't return CAM_REQ_CMP for SET_TRAN_SETTINGS CCBs.

NCR driver:

- Fix the ultra/ultra 2 negotiation bug
- allow setting both the sync rate and offset separately

Other HBA drivers:
- Put code in to set the base_transfer_speed field for
XPT_GET_TRAN_SETTINGS CCBs.

Reviewed by: gibbs, mjacob (isp), imp (aha)

Properly print devices that do not have attached peripherals.

Submitted by: Kenneth Merry <ken@FreeBSD.org>

Bad Dog! No Biscuit! *Never* commit without testing- even if it was
"just a printf formatting change"....

add Bus and Device Reset commands

Fix a typo in a comment.

When we send a stop unit command to a device, send it as an ordered tag so
that any transactions in front of the stop command get flushed to disk
first. This will have no effect on devices that have tagged queueing
turned off, or don't support tagged queueing.

Reviewed by: gibbs

Fix a bug in camcontrol that caused 'camcontrol start' to not work.

Noticed by: Philippe Regnauld <regnauld@deepo.prosa.dk>

Add the -c flag in to the usage summaries in the man page and the usage()
function in camcontrol.

Also, fix the modepage example in the examples section of the man page. It
didn't quite come out right with the '.Dl' macro.

Add a new CAM debugging mode, CAM_DEBUG_CDB. This causes the kernel to
print out a one line description/dump of every SCSI CDB sent to a
particular debugging target or targets.

This is a good bit more useful than the other debugging modes, I think.

Change some things in LINT to note the availability of this new option.

Fix an erroneous argument to scsi_cdb_string() in scsi_all.c

Reviewed by: gibbs

Camcontrol - A utility for configuring/manipulating the CAM subsystem

Submitted by: "Kenneth D. Merry" <ken@plutotech.com>