Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r264020 (by trasz):
Remove the homegrown ctl_be_block_io allocator, replacing it with UMA.
There is no performance difference.

MFC r263979 (by trasz):
Hide CTL messages about SCSI error responses. Too many users take
them for actual target errors. They can be enabled back by setting
kern.cam.ctl.verbose=1, or booting with bootverbose.

MFC r263811 (by trasz):
Remove ctl_mem_pool.{c,h}.

MFC r262782 (by trasz):
Fix missing unlock in persistent reservations code, which resulted in panics
with Hyper-V Failover Cluster.

MFC r258871:
Properly report an error instead of panicing when user tries to create
LUN backed by non-disk device, e.g. /dev/null.

MFC r257946:
Introduce seperate mutex lock to protect protect CTL I/O pools, slightly
reducing global CTL lock scope and congestion.

While there, simplify CTL I/O pools KPI, hiding implementation details.

MFC r264191:
Report stripe size and offset of the backing device in READ CAPACITY (16)
as physical sector size and offset.

MFC r261538:
Make CTL block backend return proper error code for operations unsupposed
by the underlying device.

MFC r258622,258675: dtrace sdt: remove the ugly sname parameter of
SDT_PROBE_DEFINE

MFC r256995:
Remove 128KB bzero() call done for every block I/O data buffer.

MFC r254378 (by trasz):
Turn comments about locking into actual lock assertions.

MFC r249065 (by trasz):
Fix locking problem in ctl_maintenance_in() - one cannot use M_WAITOK or call
ctl_done() with mutex held.

MFC r253987 (by trasz):
Remove dead code.

MFC r250460 (by eadler):
Fix a bunch of typos.

MFC r250131 (by eadler):
Correct a few sizeof()s

MFC r249466, r249481, r250025:
Remove multilevel freezing mechanism, implemented to handle specifics of
the ATA/SATA error recovery, when post-reset recovery commands should be
allocated when queues are already full of payload requests. Instead of
removing frozen CCBs with specified range of priorities from the queue
to provide free openings, use simple hack, allowing explicit CCBs over-
allocation for requests with priority higher (numerically lower) then
CAM_PRIORITY_OOB threshold.

Simplify CCB allocation logic by removing SIM-level allocation queue.
After that SIM-level queue manages only CCBs execution, while allocation
logic is localized within each single device.

MFC r249256 (by ken):
Fix a memory leak that showed up when we delete LUNs. The memory used for
the LUN was never freed.

ctl.c: Adjust ctl_alloc_lun() to make sure we don't clear the
CTL_LUN_MALLOCED flag.

MFC r249102 (by trasz):
Since the CTL version in FreeBSD does not support High Availability,
ctl_is_single should always be set to 1. Make it so. Previously
it was always 0, because ctl_isc_start() never got to run.

MFC 244015:
------------------------------------------------------------------------
r244015 | ken | 2012-12-07 21:16:07 -0700 (Fri, 07 Dec 2012) | 17 lines

Fix the CTL OOA queue dumping code so that it does not hold a mutex
while doing a copyout. That can cause a panic, because copyout
can trigger VM faults, and we can't handle VM faults while holding
a mutex.

The solution here is to malloc a separate buffer to hold the OOA
queue entries, so that we don't risk a VM fault while filling up
the buffer and we don't have to drop the lock. The other solution
would be to wire the user's memory while filling their buffer with
copyout, but that would have been a little more complex.

Also fix a debugging parenthesis issue in ctl_abort_task() pointed
out by Chuck Tuffli.

Sponsored by: Spectra Logic Corporation

MFC 249028:

The CTL block backend is broken without this.

Fix sending virtual scatter/gather lists from the CTL CAM frontend
peripheral.

Sponsored by: Spectra Logic

MFC r255117:
Fix the build with CTLFEDEBUG, broken by unmapped I/O support changes.

MFC r249194 (by trasz):
Make SYNCHRONIZE CACHE work with LUNs backed by device files (as opposed
to regular files, which already worked fine). With this change, it's no
longer neccessary to use "ctladm realsync off" workaround.

MFC r252569:
Fix NULL-deference kernel panic on attempt of destroying non-existing
ramdisk-backed CTL LUN.

MFC r249468:
Stop abusing xpt_periph in random plases that really have no periph related
to CCB, for example, bus scanning. NULL value is fine in such cases and it
is correctly logged in debug messages as "noperiph". If at some point we
need some real XPT periphs (alike to pmpX now), quite likely they will be
per-bus, and not a single global instance as xpt_periph now.

Big MFC of the physbio changes necessary for unmapped I/O. These changes
have been in production at Netflix for several months with significant
success.

MFC r246713:

Reform the busdma API so that new types may be added without modifying
every architecture's busdma_machdep.c. It is done by unifying the
bus_dmamap_load_buffer() routines so that they may be called from MI
code. The MD busdma is then given a chance to do any final processing
in the complete() callback.

MFC r249538:
Some compilers issue a warning when wider integer is casted to narrow
pointer. Supposedly shut down the warning by casting through
uintptr_t.

MFC r251479:
Simplify the checking of flags for cam_periph_mapmem(). This gets rid of
a lot of code redundancy and grossness at very minor expense.

MFC r251837:
MFC r251842:
Add infrastructure for doing compatibility shims, as has been sorely
needed for the last 10 years. Far too much of the internal API is
exposed, and every small adjustment causes applications to stop working.
To kick this off, bump the API version to 0x17 as should have been done
with r246713, but add shims to compensate. Thanks to the shims, there
should be no visible change in application behavior.

Submitted by: kib, jeffr
Approved by: kib
Obtained from: Netflix

MFC r249026:

Don't directly dereference userland pointer; instead use kernel pointer
copied in from userspace. This fixes instant panic when creating CTL LUN
on sparc64. Not a security problem, since the API is root-only.

MFC r249019:

Fix comment formatting.

MFC r249012:

Remove unused code.

MFC r249009:

Make it possible to build CTL as a module.

MFC r249022 by glebius@:

- Remove extra $FreeBSD$
- Touch options headers to make module buildable.

MFC r244052 and r245288:

r244052 | ken | 2012-12-09 12:53:21 -0700 (Sun, 09 Dec 2012) | 20 lines

Fix a couple of CTL locking issues and clean up some duplicated code.

ctl_frontend_cam_sim.c: Coalesce cfcs_online() and cfcs_offline()
into a single function since these were
identical except for one line.

Make sure we hold the SIM lock around path
creation, and calling xpt_rescan().

scsi_ctl.c: In ctlfe_onoffline(), make sure we hold the
SIM lock around path creation and free
calls, as well as xpt_action().

In ctlfe_lun_enable(), hold the SIM lock
around path and peripheral operations that
require it.

Sponsored by: Spectra Logic Corporation
MFC after: 1 week

------------------------------------------------------------------------
r245228 | ken | 2013-01-09 10:02:08 -0700 (Wed, 09 Jan 2013) | 43 lines

Make CTL work a little better with loading and unloading drivers.

Previously CTL would leave individual LUNs enabled in the target
driver, whether or not the port as a whole was enabled. It would
also leave the wildcard LUN enabled indefinitely.

This change means that CTL will enable and disable any active LUNs,
as well as the wildcard LUN, when enabling and disabling a port.

Also, fix a bug that could crop up due to an uninitialized CCB
type.

ctl.c: Before calling ctl_frontend_online(), run through
the LUN list and enable all active LUNs.

After calling ctl_frontend_offline(), run through
the LUN list and disble all active LUNs.

scsi_ctl.c: Before bringing a port online, allocate the
wildcard peripheral for that bus. And after taking
a port offline, invalidate the wildcard peripheral
for that bus.

Make sure that we hold the SIM lock around all
calls to xpt_action() and other transport layer
interfaces that require it.

Use CAM_SIM_{LOCK|UNLOCK} consistently to acquire
and release the SIM lock.

Update a number of outdated comments. Some of
these should have been fixed long ago.

Actually do LUN disbables now. The newer drivers
in the tree work correctly for this as far as I
know.

Initialize the CCB type to CTLFE_CCB_DEFAULT to
avoid a panic due to uninitialized memory.

Submitted by: Chuck Tuffli (partially)
MFC after: 1 week

MFC r236426 (by mjacob):
Print FC PortID as a hex number. This makes it easy to
figure out domain, etc..

Zero ATIO and INOTify allocations. It makes for much
less guesswork when looking at the structure and
seeing 'deadc0de' present.

MFC r244016 (by ken):
Make sure we hold the SIM lock when calling xpt_free_path().

Sponsored by: Spectra Logic Corporation

MFC r242358 (by trasz):
Use M_ZERO instead of explicit memsets and bzeros.

MFC r242174:
Remove several uses of numeric priorities from immediate CCB setups.

MFC r240993 (by trasz):
Remove useless NULL checks after M_WAITOK allocations.

MFC r240948 (by trasz):
Fix panic in CTL caused by trying to free invalid pointers passed
by the userland process via the IOCTL interface.

MFC r241844:
remove duplicate semicolons where possible.

Approved by: cperciva (implicit)

MFC r241508:
Extend SIM lock coverage during the bus registration process.

MFC of 238870.

Handle a case where we had an SRR that pushed back the..

MFC 237726:

r237726 | ken | 2012-06-28 13:39:30 -0600 (Thu, 28 Jun 2012) | 5 lines

Add a loader tunable, kern.cam.ctl.disable, that will disable
loading CTL. This may be useful in very low memory installations.

MFC r237601:
r237601 | ken | 2012-06-26 08:51:35 -0600 (Tue, 26 Jun 2012) | 38 lines

Fix an issue that caused the kernel to panic inside CTL when trying
to attach to target capable HBAs that implement the old immediate
notify (XPT_IMMED_NOTIFY) and notify acknowledge (XPT_NOTIFY_ACK)
CCBs. The new API has been in place since SVN change 196008 in
2009.

The solution is two-fold: fix CTL to handle the responses from the
HBAs, and convert the HBA drivers in question to use the new API.

These drivers have not been tested with CTL, so how well they will
interoperate with CTL is unknown.

scsi_target.c:Update the userland target example code to use the
new immediate notify API.

scsi_ctl.c: Detect when an immediate notify CCB is returned
with CAM_REQ_INVALID or CAM_PROVIDE_FAIL status,
and just free it.

Fix a duplicate assignment.

aic79xx.c,
aic79xx_osm.c:Update the aic79xx driver to use the new API.
Target mode is not enabled on for this driver, so
the changes will have no practical effect.

aic7xxx.c,
aic7xxx_osm.c:Update the aic7xxx driver to use the new API.

sbp_targ.c: Update the firewire target code to work with the
new API.

mpt_cam.c: Update the mpt(4) driver to work with the new API.
Target mode is only enabled for Fibre Channel
mpt(4) devices.

MFC r233963:
r233963 | ken | 2012-04-06 16:23:13 -0600 (Fri, 06 Apr 2012) | 27 lines

Change the SCSI INQUIRY peripheral qualifier that CTL reports for LUNs
that don't exist.

Anecdotal evidence indicates that it is better to return 011b (bad LUN)
than 001b (LUN offline). However, this change also gives the user a
sysctl/tunable, kern.cam.ctl.inquiry_pq_no_lun, to override the change
and return to the previous behavior. (The previous behavior was to
return 001b, or LUN offline.)

ctl.c: Change the default inquiry peripheral qualifier to 011b,
and add a sysctl and tunable to allow the user to change
it back to 001b if needed.

Don't insert a Copan copyright statement in the inquiry
data. The copyright statements on the files are
sufficient.

ctl_private.h:Add sysctl variable context to the CTL softc.

ctl_cmd_table.c,
ctl_frontend_internal.c,
ctl_frontend.c,
ctl_backend.c,
ctl_error.c: Include sys/sysctl.h.

MFC r232604:
r232604 | trasz | 2012-03-06 06:43:57 -0700 (Tue, 06 Mar 2012) | 6 lines

Add LUN resizing to CTL. Also make it possible to explicitly set
size when creating file-backed or device-backed LUN.

Reviewed by: ken (earlier version)
Sponsored by: The FreeBSD Foundation

MFC r232074:

Use a better way to silence unneeded internal declaration warnings in
several sys/cam/ctl files.

Suggested by: ed
Reviewed by: ken

MFC r231092

r231092 | emaste | 2012-02-06 11:11:00 -0700 (Mon, 06 Feb 2012) | 4 lines

Avoid panic from unlocking a not locked mutex (in some error cases).

Reviewed by: ken@

MFC r229997, r230033, and r230334

Bring the CAM Target Layer into stable/9.

r230334 | ken | 2012-01-19 11:42:03 -0700 (Thu, 19 Jan 2012) | 19 lines

Quiet some clang warnings when compiling CTL.

ctl_error.c,
ctl_error.h: Take out the ctl_sense_format enumeration, and use
scsi_sense_data_type instead.

Remove ctl_get_sense_format() and switch ctl_build_ua()
over to using scsi_sense_data_type.

ctl_backend_ramdisk.c,
ctl_backend_block.c:
Use C99 structure initializers instead of GNU initializers.

ctl.c: Switch over to using the SCSI sense format enumeration
instead of the CTL-specific enumeration.

Submitted by: dim (partially)
MFC after: 1 month

r230033 | ken | 2012-01-12 15:08:33 -0700 (Thu, 12 Jan 2012) | 5 lines

Silence some unnecessary verbosity.

Reported by: mav
MFC after: 1 month

r229997 | ken | 2012-01-11 17:34:33 -0700 (Wed, 11 Jan 2012) | 170 lines

Add the CAM Target Layer (CTL).

CTL is a disk and processor device emulation subsystem originally written
for Copan Systems under Linux starting in 2003. It has been shipping in
Copan (now SGI) products since 2005.

It was ported to FreeBSD in 2008, and thanks to an agreement between SGI
(who acquired Copan's assets in 2010) and Spectra Logic in 2010, CTL is
available under a BSD-style license. The intent behind the agreement was
that Spectra would work to get CTL into the FreeBSD tree.

Some CTL features:

- Disk and processor device emulation.
- Tagged queueing
- SCSI task attribute support (ordered, head of queue, simple tags)
- SCSI implicit command ordering support. (e.g. if a read follows a mode
select, the read will be blocked until the mode select completes.)
- Full task management support (abort, LUN reset, target reset, etc.)
- Support for multiple ports
- Support for multiple simultaneous initiators
- Support for multiple simultaneous backing stores
- Persistent reservation support
- Mode sense/select support
- Error injection support
- High Availability support (1)
- All I/O handled in-kernel, no userland context switch overhead.

(1) HA Support is just an API stub, and needs much more to be fully
functional.

ctl.c: The core of CTL. Command handlers and processing,
character driver, and HA support are here.

ctl.h: Basic function declarations and data structures.

ctl_backend.c,
ctl_backend.h: The basic CTL backend API.

ctl_backend_block.c,
ctl_backend_block.h: The block and file backend. This allows for using
a disk or a file as the backing store for a LUN.
Multiple threads are started to do I/O to the
backing device, primarily because the VFS API
requires that to get any concurrency.

ctl_backend_ramdisk.c: A "fake" ramdisk backend. It only allocates a
small amount of memory to act as a source and sink
for reads and writes from an initiator. Therefore
it cannot be used for any real data, but it can be
used to test for throughput. It can also be used
to test initiators' support for extremely large LUNs.

ctl_cmd_table.c: This is a table with all 256 possible SCSI opcodes,
and command handler functions defined for supported
opcodes.

ctl_debug.h: Debugging support.

ctl_error.c,
ctl_error.h: CTL-specific wrappers around the CAM sense building
functions.

ctl_frontend.c,
ctl_frontend.h: These files define the basic CTL frontend port API.

ctl_frontend_cam_sim.c: This is a CTL frontend port that is also a CAM SIM.
This frontend allows for using CTL without any
target-capable hardware. So any LUNs you create in
CTL are visible in CAM via this port.

ctl_frontend_internal.c,
ctl_frontend_internal.h:
This is a frontend port written for Copan to do
some system-specific tasks that required sending
commands into CTL from inside the kernel. This
isn't entirely relevant to FreeBSD in general,
but can perhaps be repurposed.

ctl_ha.h: This is a stubbed-out High Availability API. Much
more is needed for full HA support. See the
comments in the header and the description of what
is needed in the README.ctl.txt file for more
details.

ctl_io.h: This defines most of the core CTL I/O structures.
union ctl_io is conceptually very similar to CAM's
union ccb.

ctl_ioctl.h: This defines all ioctls available through the CTL
character device, and the data structures needed
for those ioctls.

ctl_mem_pool.c,
ctl_mem_pool.h: Generic memory pool implementation used by the
internal frontend.

ctl_private.h: Private data structres (e.g. CTL softc) and
function prototypes. This also includes the SCSI
vendor and product names used by CTL.

ctl_scsi_all.c,
ctl_scsi_all.h: CTL wrappers around CAM sense printing functions.

ctl_ser_table.c: Command serialization table. This defines what
happens when one type of command is followed by
another type of command.

ctl_util.c,
ctl_util.h: CTL utility functions, primarily designed to be
used from userland. See ctladm for the primary
consumer of these functions. These include CDB
building functions.

scsi_ctl.c: CAM target peripheral driver and CTL frontend port.
This is the path into CTL for commands from
target-capable hardware/SIMs.

README.ctl.txt: CTL code features, roadmap, to-do list.

usr.sbin/Makefile: Add ctladm.

ctladm/Makefile,
ctladm/ctladm.8,
ctladm/ctladm.c,
ctladm/ctladm.h,
ctladm/util.c: ctladm(8) is the CTL management utility.
It fills a role similar to camcontrol(8).
It allow configuring LUNs, issuing commands,
injecting errors and various other control
functions.

usr.bin/Makefile: Add ctlstat.

ctlstat/Makefile
ctlstat/ctlstat.8,
ctlstat/ctlstat.c: ctlstat(8) fills a role similar to iostat(8).
It reports I/O statistics for CTL.

sys/conf/files: Add CTL files.

sys/conf/NOTES: Add device ctl.

sys/cam/scsi_all.h: To conform to more recent specs, the inquiry CDB
length field is now 2 bytes long.

Add several mode page definitions for CTL.

sys/cam/scsi_all.c: Handle the new 2 byte inquiry length.

sys/dev/ciss/ciss.c,
sys/dev/ata/atapi-cam.c,
sys/cam/scsi/scsi_targ_bh.c,
scsi_target/scsi_cmds.c,
mlxcontrol/interface.c: Update for 2 byte inquiry length field.

scsi_da.h: Add versions of the format and rigid disk pages
that are in a more reasonable format for CTL.

amd64/conf/GENERIC,
i386/conf/GENERIC,
ia64/conf/GENERIC,
sparc64/conf/GENERIC: Add device ctl.

i386/conf/PAE: The CTL frontend SIM at least does not compile
cleanly on PAE.

Sponsored by: Copan Systems, SGI and Spectra Logic
MFC after: 1 month

Add the CAM Target Layer (CTL).

CTL is a disk and processor device emulation subsystem originally written
for Copan Systems under Linux starting in 2003. It has been shipping in
Copan (now SGI) products since 2005.

It was ported to FreeBSD in 2008, and thanks to an agreement between SGI
(who acquired Copan's assets in 2010) and Spectra Logic in 2010, CTL is
available under a BSD-style license. The intent behind the agreement was
that Spectra would work to get CTL into the FreeBSD tree.

Some CTL features:

- Disk and processor device emulation.
- Tagged queueing
- SCSI task attribute support (ordered, head of queue, simple tags)
- SCSI implicit command ordering support. (e.g. if a read follows a mode
select, the read will be blocked until the mode select completes.)
- Full task management support (abort, LUN reset, target reset, etc.)
- Support for multiple ports
- Support for multiple simultaneous initiators
- Support for multiple simultaneous backing stores
- Persistent reservation support
- Mode sense/select support
- Error injection support
- High Availability support (1)
- All I/O handled in-kernel, no userland context switch overhead.

(1) HA Support is just an API stub, and needs much more to be fully
functional.

ctl.c: The core of CTL. Command handlers and processing,
character driver, and HA support are here.

ctl.h: Basic function declarations and data structures.

ctl_backend.c,
ctl_backend.h: The basic CTL backend API.

ctl_backend_block.c,
ctl_backend_block.h: The block and file backend. This allows for using
a disk or a file as the backing store for a LUN.
Multiple threads are started to do I/O to the
backing device, primarily because the VFS API
requires that to get any concurrency.

ctl_backend_ramdisk.c: A "fake" ramdisk backend. It only allocates a
small amount of memory to act as a source and sink
for reads and writes from an initiator. Therefore
it cannot be used for any real data, but it can be
used to test for throughput. It can also be used
to test initiators' support for extremely large LUNs.

ctl_cmd_table.c: This is a table with all 256 possible SCSI opcodes,
and command handler functions defined for supported
opcodes.

ctl_debug.h: Debugging support.

ctl_error.c,
ctl_error.h: CTL-specific wrappers around the CAM sense building
functions.

ctl_frontend.c,
ctl_frontend.h: These files define the basic CTL frontend port API.

ctl_frontend_cam_sim.c: This is a CTL frontend port that is also a CAM SIM.
This frontend allows for using CTL without any
target-capable hardware. So any LUNs you create in
CTL are visible in CAM via this port.

ctl_frontend_internal.c,
ctl_frontend_internal.h:
This is a frontend port written for Copan to do
some system-specific tasks that required sending
commands into CTL from inside the kernel. This
isn't entirely relevant to FreeBSD in general,
but can perhaps be repurposed.

ctl_ha.h: This is a stubbed-out High Availability API. Much
more is needed for full HA support. See the
comments in the header and the description of what
is needed in the README.ctl.txt file for more
details.

ctl_io.h: This defines most of the core CTL I/O structures.
union ctl_io is conceptually very similar to CAM's
union ccb.

ctl_ioctl.h: This defines all ioctls available through the CTL
character device, and the data structures needed
for those ioctls.

ctl_mem_pool.c,
ctl_mem_pool.h: Generic memory pool implementation used by the
internal frontend.

ctl_private.h: Private data structres (e.g. CTL softc) and
function prototypes. This also includes the SCSI
vendor and product names used by CTL.

ctl_scsi_all.c,
ctl_scsi_all.h: CTL wrappers around CAM sense printing functions.

ctl_ser_table.c: Command serialization table. This defines what
happens when one type of command is followed by
another type of command.

ctl_util.c,
ctl_util.h: CTL utility functions, primarily designed to be
used from userland. See ctladm for the primary
consumer of these functions. These include CDB
building functions.

scsi_ctl.c: CAM target peripheral driver and CTL frontend port.
This is the path into CTL for commands from
target-capable hardware/SIMs.

README.ctl.txt: CTL code features, roadmap, to-do list.

usr.sbin/Makefile: Add ctladm.

ctladm/Makefile,
ctladm/ctladm.8,
ctladm/ctladm.c,
ctladm/ctladm.h,
ctladm/util.c: ctladm(8) is the CTL management utility.
It fills a role similar to camcontrol(8).
It allow configuring LUNs, issuing commands,
injecting errors and various other control
functions.

usr.bin/Makefile: Add ctlstat.

ctlstat/Makefile
ctlstat/ctlstat.8,
ctlstat/ctlstat.c: ctlstat(8) fills a role similar to iostat(8).
It reports I/O statistics for CTL.

sys/conf/files: Add CTL files.

sys/conf/NOTES: Add device ctl.

sys/cam/scsi_all.h: To conform to more recent specs, the inquiry CDB
length field is now 2 bytes long.

Add several mode page definitions for CTL.

sys/cam/scsi_all.c: Handle the new 2 byte inquiry length.

sys/dev/ciss/ciss.c,
sys/dev/ata/atapi-cam.c,
sys/cam/scsi/scsi_targ_bh.c,
scsi_target/scsi_cmds.c,
mlxcontrol/interface.c: Update for 2 byte inquiry length field.

scsi_da.h: Add versions of the format and rigid disk pages
that are in a more reasonable format for CTL.

amd64/conf/GENERIC,
i386/conf/GENERIC,
ia64/conf/GENERIC,
sparc64/conf/GENERIC: Add device ctl.

i386/conf/PAE: The CTL frontend SIM at least does not compile
cleanly on PAE.

Sponsored by: Copan Systems, SGI and Spectra Logic
MFC after: 1 month