- Copy stable/10 (r259064) to releng/10.0 as part of the
10.0-RELEASE cycle.
- Update __FreeBSD_version [1]
- Set branch name to -RC1

[1] 10.0-CURRENT __FreeBSD_version value ended at '55', so
start releng/10.0 at '100' so the branch is started with
a value ending in zero.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r256552:
Unify periph invalidation and destruction reporting.
Print message containing device model and serial number on invalidation.

Approved by: re (hrs)

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Added 4K quirks for:-
* OCZ Agility 2 SSDs
* Marvell SSDs
* Intel X25-M Series SSDs

Fix returning incorrect bio_resid value with failed BIO_DELETE requests.
Neither residual length reported for ATA/SCSI command nor one from another
BIO_DELETE request are in any way related to the value to be returned.

Synchronize device cache on close only if there were some write operations.
While these operations are not really needed otherwise, at least for SCSI
they may cause extra errors if some other initiator holds write exclusive
reservation on the LUN (SYNCHRONIZE CACHE handled as "write" operation).

Added 4K QUIRK for OCZ Vertex 4 SSDs

Submitted by: Borja Marcos <borjam@sarenet.es>
MFC after: 2 days

Restore use of polling mode for disk cache flush in case of kernel panic.
While I am not sure that any extra hardware access is a good idea after
panic, that is an existing behaviour that should better work correctly.

Revert r251649:
ken@ noticed that with recently added d_gone() disk method GEOM already
holds reference on the periph, so we don't need another one.

Acquire periph reference when handling d_getattr() method call.

While GEOM in general has provider opened while sending BIO_GETATTR,
GEOM DISK does not really need to open disk to read medium-unrelated
attributes for own use.

Proposed by: ken

Added missing SCSI quirks from r241784

Re-ordered SSD quirks alphabetically so they are easier to maintain.

Removed my email and PR reference from comments on each quirk.

Added quirks for more SSDs:
* Crucial M4
* Corsair Force GT
* Intel 520 Series
* Kingston E100 Series
* Samsung 830 Series

Reviewed by: pjd (mentor)
Approved by: pjd (mentor)
MFC after: 1 week

Added output of device QUIRKS for CAM and AHCI devices during boot.

Reviewed by: mav
Approved by: pjd (mentor)
MFC after: 2 weeks

Intel's 320-series and 510-series SSDs advertise 512-byte sectors
sizes for both logical and physical. Add ADA_Q_4K quirks
for both.

PR: kern/178040
Submitted by: Jeremy Chadwick <jdc@koitsu.org>

Disable sending Early R_OK on SiI3726/SiI3826 port multipliers.
With "cached read" HDD testing and multiple ports busy on a SATA
host controller, 3726/3826 PMP will very rarely drop a deferred
R_OK that was intended for the host. Symptom will be all 5 drives
under test will timeout, get reset, and recover.

Submitted by: Rich Futyma <rich.futyma@sanmina.com>
MFC after: 2 weeks

Rework r250298 in more correct way.

Fix byte order of ATA WWN when converting it to SCSI LUN ID.

Correct comment typo's
Add missing comment

Reviewed by: pjd (mentor)
Approved by: pjd (mentor)
MFC after: 2 weeks

MFprojects/camlock r249542:
Remove ADA_FLAG_PACK_INVALID flag. Since ATA disks have no concept of media
change it only duplicates CAM_PERIPH_INVALID flag, so we can use last one.

Slightly cleanup DA_FLAG_PACK_INVALID use.

Added automatic detection of non-rotating media which disables the
use of BIO queue sorting, hence optimising performance for devices
such as SSD's

Reviewed by: scottl
Approved by: pjd (mentor)
MFC after: 2 weeks

Teach GEOM and CAM about the difference between the max "size" of r/w and delete
requests.

sys/geom/geom_disk.h:
- Added d_delmaxsize which represents the maximum size of individual
device delete requests in bytes. This can be used by devices to
inform geom of their size limitations regarding delete operations
which are generally different from the read / write limits as data
is not usually transferred from the host to physical device.

sys/geom/geom_disk.c:
- Use new d_delmaxsize to calculate the size of chunks passed through to
the underlying strategy during deletes instead of using read / write
optimised values. This defaults to d_maxsize if unset (0).

- Moved d_maxsize default up so it can be used to default d_delmaxsize

sys/cam/ata/ata_da.c:
- Added d_delmaxsize calculations for TRIM and CFA

sys/cam/scsi/scsi_da.c:
- Added re-calculation of d_delmaxsize whenever delete_method is set.

- Added kern.cam.da.X.delete_max sysctl which allows the max size for
delete requests to be limited. This is useful in preventing timeouts
on devices who's delete methods are slow. It should be noted that
this limit is reset then the device delete method is changed and
that it can only be lowered not increased from the device max.

Reviewed by: mav
Approved by: pjd (mentor)

Updated TRIM calculations in cam/ata to be based off ATA_DSM_* defines

Reviewed by: mav
Approved by: pjd (mentor)
MFC after: 2 weeks

MFprojects/camlock r248982:
Stop abusing xpt_periph in random plases that really have no periph related
to CCB, for example, bus scanning. NULL value is fine in such cases and it
is correctly logged in debug messages as "noperiph". If at some point we
need some real XPT periphs (alike to pmpX now), quite likely they will be
per-bus, and not a single global instance as xpt_periph now.

MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905,
r248917, r248918, r248978, r249001, r249014, r249030:

Remove multilevel freezing mechanism, implemented to handle specifics of
the ATA/SATA error recovery, when post-reset recovery commands should be
allocated when queues are already full of payload requests. Instead of
removing frozen CCBs with specified range of priorities from the queue
to provide free openings, use simple hack, allowing explicit CCBs over-
allocation for requests with priority higher (numerically lower) then
CAM_PRIORITY_OOB threshold.

Simplify CCB allocation logic by removing SIM-level allocation queue.
After that SIM-level queue manages only CCBs execution, while allocation
logic is localized within each single device.

Suggested by: gibbs

MFprojects/camlock r248894:
Use full freeze while PMP does hard reset. This is only cosmetical change.

Add a callback to the ada(4) driver so that it knows when GEOM has released
references to it.

This is the functional equivalent to change r237518, which added this
functionality to the cd(4) and da(4) drivers.

This fix prevents a panic caused by GEOM calling adaopen() while the device
is going away. We now keep the device around until GEOM has finished
cleaning up its state.

ata_da.c: In adaregister(), add a d_gone callback to the GEOM disk
structure registered for the ada driver. Increment the
peripheral reference count for GEOM.

Add a new callback, adadiskgonecb(), that GEOM calls when
it is done with its resources. This callback releases the
reference acquired in adaregister().

Submitted by: Po-Li Soong
Sponsored by: Spectra Logic
MFC After: 5 days

- With the demise of !ATA_CAM, ATA_STATIC_ID is the only ata(4) related
option left but actually consumed by ada(4), so move it to opt_ada.h
and get rid of opt_ata.h.
- Fix stand-alone build of atacore(4) by adding opt_cam.h.
- Use __FBSDID.
- Use DEVMETHOD_END.
- Use NULL instead of 0 for pointers.

Replicate r245306 from SCSI to ATA. The problem didn't appear so far,
covered by multilevel freeze mechanism, but it is better to be safe.

Unbreak ATA_NO_48BIT_DMA with ATA_CAM by treating 48-bit DMA as an
optional property with PATA transport.

Reviewed by: mav
MFC after: 3 days

MFprojects/camlock:
r249017:
Some cosmetic things:
- Unify device to target insertion inside xpt_alloc_device() instead of
duplicating it three times.
- Remove extra checks for empty lists of devices and targets on release
since zero refcount check also implies it.
- Reformat code to reduce indentation.

r249103:
- Add lock assertions to every point where reference counters are modified.
- When reference counters are reaching zero, add assertions that there are
no children items left.
- Add a bit more locking to the xptpdperiphtraverse().

MFprojects/camlock r248931:
Replace some direct mutex operations with wrappers.

MFC after: 2 weeks

MFprojects/camlock r248930:
Remove extra NULL checks. d_drv1 can never be NULL during periph life cycle.

MFC after: 2 weeks

Remove all legacy ATA code parts, not used since options ATA_CAM enabled in
most kernels before FreeBSD 9.0. Remove such modules and respective kernel
options: atadisk, ataraid, atapicd, atapifd, atapist, atapicam. Remove the
atacontrol utility and some man pages. Remove useless now options ATA_CAM.

No objections: current@, stable@
MFC after: never

Add xpt_release_ccb()'s missed at r248872. That made `shutdown -p` stuck
on controller with small number of queue slots and several disks connected.

Adds the ability to enable / disable sorting of BIO requests queued within
CAM. This can significantly improve performance particularly for SSDs
which don't suffer from seek latencies.

The sysctl / tunable kern.cam.sort_io_queues provides the systems default
setting where:-
0 = queued BIOs are NOT sorted
1 = queued BIOs are sorted (default)

Each device gets its own sysctl kern.cam.<type>.<id>.sort_io_queue
Valid values are:-
-1 = use system default (default)
0 = queued BIOs are NOT sorted
1 = queued BIOs are sorted

Note: Additional patch will look to add automatic use of none sorted queues
for none rotating media e.g. SSD's

Reviewed by: scottl
Approved by: pjd (mentor)
MFC after: 2 weeks

Make pre-shutdown flush and spindown routines to not use xpt_polled_action(),
but execute the commands in regular way. There is no any reason to cook CPU
while the system is still fully operational. After this change polling in
CAM is used only for kernel dumping.

Implement CAM_PERIPH_FOREACH() macro, safely iterating over the list of
driver's periphs, acquiring and releaseing periph references while doing it.

Use it to iterate over the lists of ada and da periphs when flushing caches
and putting devices to sleep on shutdown and suspend. Previous code could
panic in theory if some device disappear in the middle of the process.

Remove two bzero()s that are erasing only few more bytes then set later.

Support unmapped i/o for the md(4).

The vnode-backed md(4) has to map the unmapped bio because VOP_READ()
and VOP_WRITE() interfaces do not allow to pass unmapped requests to
the filesystem. Vnode-backed md(4) uses pbufs instead of relying on
the bio_transient_map, to avoid usual md deadlock.

Sponsored by: The FreeBSD Foundation
Tested by: pho, scottl

Hide SEMB port of the SiI3826 Port Multiplier by default to avoid extra
errors while it tries to talk via I2C to usually missing external SEP.
There is tunable to enable it back when needed.

Fix problem with the Samsung 840 PRO series SSD detection.
The device reports support for SATA Asynchronous Notification in its
IDENTIFY data, but returns error on attempt to enable that feature.
Make SATA XPT of CAM only report these errors, but not fail the device.

MFC after: 1 week

Adds 4K quirks for the some SSD's which all perform better when 4K
aligned and only except 4K deletes (TRIM).

PR: kern/169974
Submitted by: Steven Hartland <steven.hartland@multiplay.co.uk>
Tested by: ak
Reviewed by: mav
Approved by: cperciva (implicit)
MFC after: 1 week

Protect xpt_getattr() calls with the SIM lock and assert that.

Submitted by: ken@ (earlier version)

Don't duplicate path/ccb allocation code, use existing functions.

Remove 'periph == NULL' check from bunch of periph drivers.
This condition can never be true as functions are called from single place
and the checks just pollute the code and confuse Clang Static Analyzer.

ata_da: set disk::d_ident from serial number

MFC after: 10 days

Implement media change notification for DA and CD removable media devices.
It includes three parts:
1) Modifications to CAM to detect media media changes and report them to
disk(9) layer. For modern SATA (and potentially UAS) devices it utilizes
Asynchronous Notification mechanism to receive events from hardware.
Active polling with TEST UNIT READY commands with 3 seconds period is used
for incapable hardware. After that both CD and DA drivers work the same way,
detecting two conditions: "NOT READY: Medium not present" after medium was
detected previously, and "UNIT ATTENTION: Not ready to ready change, medium
may have changed". First one reported to disk(9) as media removal, second
as media insert/change. To reliably receive second event new
AC_UNIT_ATTENTION async added to make UAs broadcasted to all periphs by
generic error handling code in cam_periph_error().
2) Modifications to GEOM core to handle media remove and change events.
Media removal handled by spoiling all consumers attached to the provider.
Media change event also schedules provider retaste after spoiling to probe
new media. New flag G_CF_ORPHAN was added to consumers to reflect that
consumer is in process of destruction. It allows retaste to create new
geom instance of the same class, while previous one is still dying.
3) Modifications to some GEOM classes: DEV -- to report media change
events to devd; VFS -- to handle spoiling same as orphan to prevent
accessing replaced media. PART class already handles spoiling alike to
orphan.

Reviewed by: silence on geom@ and scsi@
Tested by: avg
Sponsored by: iXsystems, Inc. / PC-BSD
MFC after: 2 months

Add and utilize defines for the ATA device register.

PR: 169764
Submitted by: Steven Hartland <steven.hartland@multiplay.co.uk>
Reviewed by: mav
MFC after: 2 weeks

Apply similar same change as in r238379:

Renamed the kern.cam.ada.ada_send_ordered sysctl and tunable to
kern.cam.ada.send_ordered, more in line with the other da sysctls/tunables.

Suggested by: kib

- fix description of SECURITY_FREEZE_LOCK (1)
- add missing underscores to SECURITY_DISABLE_PASSWORD

PR: 169763 (1)
Submitted by: Steven Hartland <steven.hartland@multiplay.co.uk>
Reviewed by: mav
MFC after: 1 week

One more major cam_periph_error() rewrite to improve error handling and
reporting. It includes:
- removing of error messages controlled by bootverbose, replacing them
with more universal and informative debugging on CAM_DEBUG_INFO level,
that is now built into the kernel by default;
- more close following to the arguments submitted by caller, such as
SF_PRINT_ALWAYS, SF_QUIET_IR and SF_NO_PRINT; consumer knows better which
errors are usual/expected at this point and which are really informative;
- adding two new flags SF_NO_RECOVERY and SF_NO_RETRY to allow caller
specify how much assistance it needs at this point; previously consumers
controlled that by not calling cam_periph_error() at all, but that made
behavior inconsistent and debugging complicated;
- tuning debug messages and taken actions order to make debugging output
more readable and cause-effect relationships visible;
- making camperiphdone() (common device recovery completion handler) to
also use cam_periph_error() in most cases, instead of own dumb code;
- removing manual sense fetching code from cam_periph_error(); I was told
by number of people that it is SIM obligation to fetch sense data, so this
code is useless and only significantly complicates recovery logic;
- making ada, da and pass driver to use cam_periph_error() with new limited
recovery options to handle error recovery and debugging in common way;
as one of results, CAM_REQUEUE_REQ and other retrying statuses are now
working fine with pass driver, that caused many problems before.
- reverting r186891 by raj@ to avoid burning few seconds in tight DELAY()
loops on device probe, while device simply loads media; I think that problem
may already be fixed in other way, and even if it is not, solution must be
different.

Sponsored by: iXsystems, Inc.
MFC after: 2 weeks

ATA/SATA controllers have no idea about protocol of the connected device
until transport will do some probe actions (at least soft reset).
Make ATA/SATA SIMs to not report bogus and confusing PROTO_ATA protocol.
Make ATA/SATA transport to fill that gap by reporting protocol to SIM with
XPT_SET_TRAN_SETTINGS and patching XPT_GET_TRAN_SETTINGS results if needed.

Use default error handler when flushing disk caches.

Tune and add some more CAM_DEBUG() points for the probe sequences.

Tune and add some missing CAM_DEBUG() points for better consistency.

Remove some dead code that I doubt will ever be implemented.

Rewrite enabling NCQ for SATA devices in a way more alike to SCSI TCQ.
This allows to control it with `camcontrol negotiate adaX -T (en|dis)able`
on the fly, same as for SCSI devices.

Sponsored by: iXsystems, Inc.

Use AC_GETDEV_CHANGED async to notify ada driver about DMA and NCQ status
change. Now that allows switching between PIO and DMA modes on the fly.

Allow to change number of openings (used tags) for ATA/SATA devices
via `camcontrol tags ... -N ...`. There is no need to tune it in
usual cases, but some users want to have it for debugging purposes.

MFC after: 2 weeks

Plug request and references leak caused by race between invalidated
ond probe periph destruction and new incoming probe request.

This at least caused problems with SATA Port Multipliers hot-plug.

MFC after: 2 weeks

Add tunable/sysctl kern.cam.pmp.hide_special, controlling whether special
PMP ports such as PMP configuration or SEMB should be exposed or hidden.
These ports were always hidden before as useless and sometimes promatic.
But with updated ses driver supporting SEMB it is no longer so straight.
Keep ports hidden by default to avoid probe request ttimeouts if SEP is
not connected to PMP's SEMB via I2C, that is very often situation.

MFprojects/zfsd:
- Add low-level support for SATA Enclosure Management Bridge (SEMB)
devices -- SATA equivalents of the SCSI SES/SAF-TE devices.
- Add some utility functions for SCSI SAF-TE devices access.

Sponsored by: iXsystems, Inc.

Alike to SCSI make adaclose() to not return error if device gone.
This fixes KASSERT panic inside GEOM if kernel built with INVARIANTS.

MFC after: 1 week

Insert ordered command every 1/4 of the current command timeout, not 1/4
of the default one.

Without this change setting kern.cam.ada.default_timeout to 1 instead of 30
allowed me to trigger several false positive command timeouts under heavy
ZFS load on a SiI3132 siis(4) controller with 5 HDDs on a port multiplier.

MFC after: 1 week

Make CAM ATA honor old hw.ata.ata_dma and hw.ata.atapi_dma tunables.

Even having more specific hint.ata.X.mode controls, global ones are
still could be useful from some points, including compatibility.

PR: kern/164651
MFC after: 1 week

Remove unneeded checks for CAM_DEV_QFRZN after cam_periph_runccb() call.
cam_periph_runccb() since the beginning checks it and releases device queue.
After r203108 it even clears CAM_DEV_QFRZN flag after that to avoid double
release, so removed code is unreachable now.

MFC after: 1 month

Update list of 4K physical sector hard drives.

Mark all SYSCTL_NODEs static that have no corresponding SYSCTL_DECLs.

The SYSCTL_NODE macro defines a list that stores all child-elements of
that node. If there's no SYSCTL_DECL macro anywhere else, there's no
reason why it shouldn't be static.

Mark MALLOC_DEFINEs static that have no corresponding MALLOC_DECLAREs.

This means that their use is restricted to a single C file.

Higher-priority initialization request can eat request scheduling done from
adaclose(). Add immediate_priority check into adaschedule() to restore it.

Approved by: re (kib)
MFC after: 1 week

Add control for ATA disk read-ahead, alike to the previously added write
cache control. Some controller BIOS'es tend to disable read-ahead, that
dramatically reduces read performance. Previously ata(4) always enabled
read-ahead unconditionally.

Approved by: re (kib)

Do not try to execute FLUSHCACHE on close and print extra messages for
invalidated (considered lost) ada device. Exactly same already done for
the da devices.

Approved by: re (kib)
MFC after: 1 week

Fix ATAPI breakage introduced by r223443. It made SCSI commands to ATAPI
device to never complete, that caused probe process (system boot) stuck.

Plumb support for the device advanced information CCB in the ATA XPT.

This was previously done only for SCSI XPT in r223081, on which the change
in r223089 depended in order to respond to serial number requests. As a
result of r223089, da(4) and ada(4) devices register a d_getattr for geom to
use to obtain the information.

Reported by: ache
Reviewed by: ken

Fix a typo in adagetattr() from r223089. In particular, this restores
the ability to use ahci(4) for kernel dumps.

Plumb device physical path reporting from CAM devices, through GEOM and
DEVFS, and make it accessible via the diskinfo utility.

Extend GEOM's generic attribute query mechanism into generic disk consumers.
sys/geom/geom_disk.c:
sys/geom/geom_disk.h:
sys/cam/scsi/scsi_da.c:
sys/cam/ata/ata_da.c:
- Allow disk providers to implement a new method which can override
the default BIO_GETATTR response, d_getattr(struct bio *). This
function returns -1 if not handled, otherwise it returns 0 or an
errno to be passed to g_io_deliver().

sys/cam/scsi/scsi_da.c:
sys/cam/ata/ata_da.c:
- Don't copy the serial number to dp->d_ident anymore, as the CAM XPT
is now responsible for returning this information via
d_getattr()->(a)dagetattr()->xpt_getatr().

sys/geom/geom_dev.c:
- Implement a new ioctl, DIOCGPHYSPATH, which returns the GEOM
attribute "GEOM::physpath", if possible. If the attribute request
returns a zero-length string, ENOENT is returned.

usr.sbin/diskinfo/diskinfo.c:
- If the DIOCGPHYSPATH ioctl is successful, report physical path
data when diskinfo is executed with the '-v' option.

Submitted by: will
Reviewed by: gibbs
Sponsored by: Spectra Logic Corporation

Add generic attribute change notification support to GEOM.

sys/sys/geom/geom.h:
Add a new attrchanged method field to both g_class
and g_geom.

sys/sys/geom/geom.h:
sys/geom/geom_event.c:
- Provide the g_attr_changed() function that providers
can use to advertise attribute changes.
- Perform delivery of attribute change notifications
from a thread context via the standard GEOM event
mechanism.

sys/geom/geom_subr.c:
Inherit the attrchanged method from class to geom (class instance).

sys/geom/geom_disk.c:
Provide disk_attr_changed() to provide g_attr_changed() access
to consumers of the disk API.

sys/cam/scsi/scsi_pass.c:
sys/cam/scsi/scsi_da.c:
sys/geom/geom_dev.c:
sys/geom/geom_disk.c:
Use attribute changed events to track updates to physical path
information.

sys/cam/scsi/scsi_da.c:
Add AC_ADVINFO_CHANGED to the registered asynchronous CAM
events for this driver. When this event occurs, and
the updated buffer type references our physical path
attribute, emit a GEOM attribute changed event via the
disk_attr_changed() API.

sys/cam/scsi/scsi_pass.c:
Add AC_ADVINFO_CHANGED to the registered asynchronous CAM
events for this driver. When this event occurs, update
the physical patch devfs alias for this pass instance.

Submitted by: gibbs
Sponsored by: Spectra Logic Corporation

Do not report CFA devices as ATAPI, even though IDENTIFY data look alike.

Increase maximum supported number of ranges per TRIM command from 256 to 512
to use full potential of Intel X25-M SSDs. On synthetic test with 32K ranges
it gives about 20% speedup, which probably costs more then 2K of RAM.

When possible, join ranges of subsequest BIO_DELETE requests to handle more
(up to 2048 instead of 256 or even 64) of them with single TRIM request.

OCZ Vertex2/Vertex3 SSDs can handle no more then 64 ranges per TRIM request.
Due to lack of BIO_DELETE clustering now, it means that we could delete no
more then 2MB per request (on FS with 32K block) with limited request rate.
This change increases delete rate on Vertex2 from 250MB/s to 950MB/s.

Add quirks to hint 4K physical sector (Advanced Format) for ATA disks not
reporting it properly (none? of known disks now).

Hitachi and WDC AF disks seem could be identified more or less formally.
For Seagate and Samsung enumerate some found models/series.
For other disks it can be forced with kern.cam.ada.X.quirks=1 tunable.

- Add shim to simplify migration to the CAM-based ATA. For each new adaX
device in /dev/ create symbolic link with adY name, trying to mimic old ATA
numbering. Imitation is not complete, but should be enough in most cases to
mount file systems without touching /etc/fstab.
- To know what behavior to mimic, restore ATA_STATIC_ID option in cases
where it was present before.
- Add some more details to UPDATING.

Add basic support for DMA-capable ATA disks on DMA-incapable controller.
This is really rare situation these days, but still may happen in embedded.

Remove some used variables.

Found with: Clang Static Analyzer

Make ada(4) driver put ATA disks into sleep state on suspend.

Submitted by: jkim (original version)

Make CAM report devices with ATA/SATA transport to devstat(9) as IDE.

Fix typo in write_cache tunable name and move TUNABLE_INT_FETCH() out of
the SIM lock to fix WITNESS warning.

Reported by: jh

Properly log few more ATA commands used by the kernel.

Improve SATA Asynchronous Notification feature support in CAM:
- make SATA SIMs announce capabilities to handle SDB with Notification bit;
- make PMP driver honor this SIMs capability;
- make SATA XPT to negotiate and enable this feature for ATAPI devices.

This feature allows supporting SATA ATAPI devices to inform system about
some events happened, that may require attention. In my case this allows
LG GH22LS50 SATA DVR-RW drive to report tray open/close events. Events
reported to CAM in form of AC_SCSI_AEN async. Further they could be used
as a hints for checking device status and reporting media change to upper
layers, for example, via spoiling mechanism of GEOM.

- Add kern.cam.ada.X.write_cache tunables/sysctls to control write caching
on per-device basis.
- While adding support for per-device sysctls, merge from graid branch
support for ADA_TEST_FAILURE kernel option, which opens few more sysctl,
allowing to simulate read and write errors for testing purposes.

Make ada(4) driver to control device write cache, same as ata(4) does.
Add kern.cam.ada.write_cache sysctl/tunable to control it alike hw.ata.wc.

Add the disk ident and a human-meaningful description (here, the disk model
string) to the geom_disk config XML so that they are easily accessible from
userland.

MFC after: 1 week

In addition to r217444 ignore also ATA status errors on DMA Auto-Activation
enabling request. Some HP disks reported to return ABORT error there while
declaring support for this feature.

Make device initialization sequence shorter when possible. Do not enable/
disable already enabled/disabled SATA features.

Some old WD SATA disks report supported and enabled device-initiated
interface power management, but return ABORT error on attempt to disable
it. Make CAM SATA probe sequence ignore this error, as it is not fatal.

If HBA doesn't report user-enabled SATA capabilies (like ATA_CAM wrapper) -
handle all of them as disabled. This was original cause of the problem,
workarounded by r215453.

MFC after: 1 week

Mostly revert r203420, and add similar functionality into ada(4) since the
existing code caused problems with some SCSI controllers.

A new sysctl kern.cam.ada.spindown_shutdown has been added that controls
whether or not to spin-down disks when shutting down.
Spinning down the disks unloads/parks the heads - this is
much better than removing power when the disk is still
spinning because otherwise an Emergency Unload occurs which may cause damage
to the actuator.

PR: kern/140752
Submitted by: olli
Reviewed by: arundel
Discussed with: mav
MFC after: 2 weeks

Correct bioq_disksort so that bioq_insert_tail() offers barrier semantic.
Add the BIO_ORDERED flag for struct bio and update bio clients to use it.

The barrier semantics of bioq_insert_tail() were broken in two ways:

o In bioq_disksort(), an added bio could be inserted at the head of
the queue, even when a barrier was present, if the sort key for
the new entry was less than that of the last queued barrier bio.

o The last_offset used to generate the sort key for newly queued bios
did not stay at the position of the barrier until either the
barrier was de-queued, or a new barrier (which updates last_offset)
was queued. When a barrier is in effect, we know that the disk
will pass through the barrier position just before the
"blocked bios" are released, so using the barrier's offset for
last_offset is the optimal choice.

sys/geom/sched/subr_disk.c:
sys/kern/subr_disk.c:
o Update last_offset in bioq_insert_tail().

o Only update last_offset in bioq_remove() if the removed bio is
at the head of the queue (typically due to a call via
bioq_takefirst()) and no barrier is active.

o In bioq_disksort(), if we have a barrier (insert_point is non-NULL),
set prev to the barrier and cur to it's next element. Now that
last_offset is kept at the barrier position, this change isn't
strictly necessary, but since we have to take a decision branch
anyway, it does avoid one, no-op, loop iteration in the while
loop that immediately follows.

o In bioq_disksort(), bypass the normal sort for bios with the
BIO_ORDERED attribute and instead insert them into the queue
with bioq_insert_tail(). bioq_insert_tail() not only gives
the desired command order during insertion, but also provides
barrier semantics so that commands disksorted in the future
cannot pass the just enqueued transaction.

sys/sys/bio.h:
Add BIO_ORDERED as bit 4 of the bio_flags field in struct bio.

sys/cam/ata/ata_da.c:
sys/cam/scsi/scsi_da.c
Use an ordered command for SCSI/ATA-NCQ commands issued in
response to bios with the BIO_ORDERED flag set.

sys/cam/scsi/scsi_da.c
Use an ordered tag when issuing a synchronize cache command.

Wrap some lines to 80 columns.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c
sys/geom/geom_io.c
Mark bios with the BIO_FLUSH command as BIO_ORDERED.

Sponsored by: Spectra Logic Corporation
MFC after: 1 month

Export PCI IDs of ATA/SATA controllers through CAM and ata(4) layers to
GEOM. This information needed for proper soft-RAID's on-disk metadata
reading and writing.

ATA device reset starts probe sequence from the beginning. If reset caused
by timeout/error of one of probe commands, process may continue infinitely.
Make CAM ATA more robust to faulty devices and false positive detections,
abort probe after two restarts on timeouts or ten on other errors.

Add allocation error hadling.

Found with: Coverity Prevent(tm)
CID: 3897

Fix double free on error.

Found with: Coverity Prevent(tm)
CID: 4573

Add a new primitive, XPT_SCAN_TGT, to cover the range between scanning a
whole bus (XPT_SCAN_BUS) and a single lun on that bus (XPT_SCAN_LUN).

It's less resource comsumptive than scanning a whole bus when the
caller knows only one target has changes.

Reviewed by: scsi@
Sponsored by: Panasas
MFC after: 1 month

Change ad_firmware_geom_adjust() to operate on a struct disk * only and
hook it up to ada(4) also. While at it, rename *ad_firmware_geom_adjust()
to *ata_disk_firmware_geom_adjust() etc now that these are no longer
limited to ad(4).

Reviewed by: mav
MFC after: 3 days

Remove unnecessary headers which were inherited from cam_xpt.c without
reason.

Reviewed by: mav
MFC after: 3 days

Make SATA XPT negotiate and enable some additional SATA features, such as:
- device initiated power management (some devices support only this way);
- Automatic Partial to Slumber Transition (more power saving);
- DMA auto-activation (expected to slightly improve performance).
More features could be added later, when hardware supports.

Report PMP absence using target 15, same as for precence (not a wildcard),
to not confuse target ID checks at SIMs.

Update device identify data and serial number when device change detected.
Reprobe immediately following this should have fresh data.

MFp4:
Move PI_TAG_ABLE check from ada driver to ATA XPT.

Make PUIS detection more strict. Previous implementation caused false
positives on VMWare's virtual CD-ROMs.

Virtualize transport part of periph announcement.

On probe error, if restart requested, skip any retries and recovery.
Just restart probe from the beginning immediately.

MFp4:
Add Power Up In Stand-by feature support. Device with PUIS enabled
require explicit command to do initial spin-up. Mark that command
with CAM_HIGH_POWER flag, to allow CAM manage staggered spin-up.

- Use separate buffer for identify data fetching. We can't use main buffer
here if device already running, as data need to be formatted before use.
- Remove some saved_ccb variables. They are unused now.

- Give ATA/SATA SIMs info about ATAPI packet size, supported by device.
- Make ATA XPT to reject longer SCSI CDBs then supported by device, or
any SCSI CDBs, if device doesn't support ATAPI.

MFp4: Large set of CAM inprovements.

- Unify bus reset/probe sequence. Whenever bus attached at boot or later,
CAM will automatically reset and scan it. It allows to remove duplicate
code from many drivers.
- Any bus, attached before CAM completed it's boot-time initialization,
will equally join to the process, delaying boot if needed.
- New kern.cam.boot_delay loader tunable should help controllers that
are still unable to register their buses in time (such as slow USB/
PCCard/ CardBus devices), by adding one more event to wait on boot.
- To allow synchronization between different CAM levels, concept of
requests priorities was extended. Priorities now split between several
"run levels". Device can be freezed at specified level, allowing higher
priority requests to pass. For example, no payload requests allowed,
until PMP driver enable port. ATA XPT negotiate transfer parameters,
periph driver configure caching and so on.
- Frozen requests are no more counted by request allocation scheduler.
It fixes deadlocks, when frozen low priority payload requests occupying
slots, required by higher levels to manage theit execution.
- Two last changes were holding proper ATA reinitialization and error
recovery implementation. Now it is done: SATA controllers and Port
Multipliers now implement automatic hot-plug and should correctly
recover from timeouts and bus resets.
- Improve SCSI error recovery for devices on buses without automatic sense
reporting, such as ATAPI or USB. For example, it allows CAM to wait, while
CD drive loads disk, instead of immediately return error status.
- Decapitalize diagnostic messages and make them more readable and sensible.
- Teach PMP driver to limit maximum speed on fan-out ports.
- Make boot wait for PMP scan completes, and make rescan more reliable.
- Fix pass driver, to return CCB to user level in case of error.
- Increase number of retries in cd driver, as device may return several UAs.

- Report SATA in legacy emulation mode still as SATA.
- Make ATA XPT able to handle such case.

Add BIO_DELETE support to ada(4):
- For SSDs use TRIM feature of DATA SET MANAGEMENT command, as defined by
ACS-2 specification working draft.
- For CompactFlash use CFA ERASE command, same as ad(4) does.

With this patch, `newfs -E /dev/ada1` was able to restore write speed of
my heavily weared OCZ Vertex SSD (firmware 1.4) up to the initial level
for the most part of it's capacity. Previous 1.3 firmware, even reportiong
TRIM capabilty bit set, was not working, reporting ABORT error for every
DSM command.

I have no idea whether it is normal, but for some reason it takes 200ms
to handle any TRIM command on this drive, that was making delete extremely
slow. But TRIM command is able to accept long list of LBAs and the length of
that list seems doesn't affect it's execution time. Implemented request
clusting algorithm allowed me to rise delete rate up to reasonable numbers,
when many parallel DELETE requests running.

Report stripe size only if physical sector size is not equal to logical.

MFp4;
- Cleanup kernel messages, mostly PMP.
- Took references on devices, while PMP reinitializes them, to not let them
go and distort freeze reference counting.

MFp4:
If we panicked with SIM lock held, do not try to flush caches.
Extra lock recursing will not make debugging easier.

MFp4:
Introduce ATA_CAM kernel option, turning ata(4) controller drivers into
cam(4) interface modules. When enabled, this options deprecates all ata(4)
peripheral drivers (ad, acd, ...) and interfaces and allows cam(4) drivers
(ada, cd, ...) and interfaces to be natively used instead.

As side effect of this, ata(4) mode setting code was completely rewritten
to make controller API more strict and permit above change. While doing
this, SATA revision was separated from PATA mode. It allows DMA-incapable
SATA devices to operate and makes hw.ata.atapi_dma tunable work again.

Also allow ata(4) controller drivers (except some specific or broken ones)
to handle larger data transfers. Previous constraint of 64K was artificial
and is not really required by PCI ATA BM specification or hardware.

Submitted by: nwitehorn (powerpc part)

Add CAM_ATAIO_DMA ATA command flag to mark DMA protocol commands.
It is not needed for SATA controllers, but required for PATA.

MFp4:
Improve ATA mode/SATA revision control.

Fix small copu-paste bug.

MFp4:
- Extend XPT-SIM transfer settings control API. Now it allows to report to
SATA SIM number of tags supported by each device, implement ATA mode and
SATA revision negotiation for both SATA and PATA SIMs.
- Make ahci(4) and siis(4) to use submitted maximum tag number, when
scheduling requests. It allows to support NCQ on devices with lower tags
count then controller supports.
- Make PMP driver to report attached devices connection speeds.
- Implement ATA mode negotiation between user settings, device and
controller capabilities.

Disable PortMultiplier Async Notifications for time of ports reset.
They are useless at that time, but confuse Marvell AHCI.

Add quirk for SiI57XX Port Multipliers, to hide extra port.

MFp4:
Do not enable tagged queueing if controller reports 0 tags support.

MFp4:
- Move tagged queueing control from ADA to ATA XPT. It allows to control
device command queue length correctly. First step to support < 32 tags.
- Limit queue for non-tagged devices by 2 slots for ahci(4) and siis(4).
- Implement quirk matching for ATA devices.
- Move xpt_schedule_dev_sendq() from header to source file.
- Move delayed queue shrinking to the more expected place - element freeing.
- Remove some SCSIsms in ATA.

Fix protype.

PMP commands use short format. PMP write doesn't return result.

MFp4:
- Remove CAM_PERIPH_POLLED flag. It is broken by design. Polling can't be
periph flag. May be SIM, may be CCB, but now it works fine just without it.
- Remove check unused for at least five years. If we will ever have non-BIO
devices in CAM, this check is smallest of what we will need.
- If several controllers complete requests same time, call swi_sched()
only once.

MFp4:
- Add support for sector size > 512 bytes and physical sector of several
logical sectors, introduced by ATA-7 specification.
- Remove some obsoleted code.

MFp4:
Improve reporting ATA Status error details.

MFp4:
Fix reference counting bug, when device unreferenced before then
invalidated. To do it, do not handle validity flag as another
reference, but explicitly modify reference count each time flag is
modified.

Discovered by: thompsa

MFp4:
- Reduce code duplication in ATA XPT and PMP driver.
- Move PIO size setting from ada driver to ATA XPT. It is XPT business
to negotiate transfer details. ada driver is now stateless.
- Report PIO size to SIM. It is required for correct PATA SIM operation.
- Tune PMP scan timings. It workarounds some problems with SiI.
- If reset hapens during PMP initialization - restart it.
- Introduce early-initialized periph drivers, which are used during initial
scan process. Use it for xpt, probe, aprobe and pmp. It gives pmp chance
to finish scan before mountroot and numerate devices in right order.

MFp4:
Move Port Multiplier support code out of ATA XPT into pmp periph driver.
This is convinient, as PMP itself is a bus target and has own state.

Replace most of priority numbers with defines. No logical changes.

MFp4:
Separate CAM_DEV_IDENTIFY_DATA_VALID flag from CAM_DEV_INQUIRY_DATA_VALID.
Add workaround for very old devices without support for mode setting.
Add some PATA bus scanning support.
Remove some SCSIsms.

MFp4:
Add support for PIO-only devices.
Fix maxio values and 256 sectors transactions for 28bits commands.
Implement periodic ordered commands insertion, sames as da driver does.
Remove some SCSIsms.

MFp4:
Report real max_target = 15. SIM doesn't need to know that target 15 is PMP.
It is XPT business.

Export disk serial numbers for adaX disks.

Reviewed by: mav
MFC after: 3 days

Report SATA 3.x devices.

Short ATA command format has 28bit address, not 36bit.
Rename ata_36bit_cmd() into ata_28bit_cmd(), while it didn't become legacy.

MFC after: 2 days

ATA_FLUSHCACHE is a 36bit format command, not 48.

Fix copy/paste bug, that requests data read during ATA device probe sequence
for ATA_SETFEATURES/ATA_SF_SETXFER command which by definition transfers no
data. Most of controllers are irrelevant to this bug, but some nVidia's
doesn't.

Tested on: current@
Approved by: re (kib)

Fix copy-paste bug. Use regular non-polled mode for executing FLUSHCACHE
command on disk close.

Approved by: re (implicitly)

Fix copy-paste bug, enabling SIM PMP support, when it was not really found.

Approved by: re (implicitly)

Rename ATA probe driver to "aprobe" to resolve name conflict with SCSI
and fix loading cam as module.

Approved by: re (implicitly)

Separate the parallel scsi knowledge out of the core of the XPT, and
modularize it so that new transports can be created.

Add a transport for SATA

Add a periph+protocol layer for ATA

Add a driver for AHCI-compliant hardware.

Add a maxio field to CAM so that drivers can advertise their max
I/O capability. Modify various drivers so that they are insulated
from the value of MAXPHYS.

The new ATA/SATA code supports AHCI-compliant hardware, and will override
the classic ATA driver if it is loaded as a module at boot time or compiled
into the kernel. The stack now support NCQ (tagged queueing) for increased
performance on modern SATA drives. It also supports port multipliers.

ATA drives are accessed via 'ada' device nodes. ATAPI drives are
accessed via 'cd' device nodes. They can all be enumerated and manipulated
via camcontrol, just like SCSI drives. SCSI commands are not translated to
their ATA equivalents; ATA native commands are used throughout the entire
stack, including camcontrol. See the camcontrol manpage for further
details. Testing this code may require that you update your fstab, and
possibly modify your BIOS to enable AHCI functionality, if available.

This code is very experimental at the moment. The userland ABI/API has
changed, so applications will need to be recompiled. It may change
further in the near future. The 'ada' device name may also change as
more infrastructure is completed in this project. The goal is to
eventually put all CAM busses and devices until newbus, allowing for
interesting topology and management options.

Few functional changes will be seen with existing SCSI/SAS/FC drivers,
though the userland ABI has still changed. In the future, transports
specific modules for SAS and FC may appear in order to better support
the topologies and capabilities of these technologies.

The modularization of CAM and the addition of the ATA/SATA modules is
meant to break CAM out of the mold of being specific to SCSI, letting it
grow to be a framework for arbitrary transports and protocols. It also
allows drivers to be written to support discrete hardware without
jeopardizing the stability of non-related hardware. While only an AHCI
driver is provided now, a Silicon Image driver is also in the works.
Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware
is possible and encouraged. Help with new transports is also encouraged.

Submitted by: scottl, mav
Approved by: re