cam: Remove left-over sys/cdefs.h in sys/cam

These weren't removed when $FreeBSD$ was removed. They aren't needed and
now are a style(9) nonconformity.

Sponsored by: Netflix

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

cam: Migrate to modern uintXX_t from u_intXX_t

As per https://lists.freebsd.org/archives/freebsd-scsi/2023-July/000257.html
move to the modern uintXX_t.

MFC After: 3 days
Sponsored by: Netflix

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

cam: Remove redunant static __inline forward decls

Sponsored by: Netflix

cam(4): Fix a few typos in source code comments

- s/trafer/transfer/
- s/failes/fails/

Obtained from: NetBSD
MFC after: 3 days

cam: fix UB behavior

The trick of subtracting one from the poitner returned from malloc
results in undefined behavior:

>>C89: 3.3.6 Unless both the pointer operand and the result point to a
>>member of the same array object, or one past the last member of the
>>array object, the behavior is undefined.

Instead, allocate 1 extra element and stop adjusting the pointer. While
a little wasteful, the extra is in the noise on today's systems.

Reviewed by: scottl@
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D30847

cam: clean up empty lines in .c and .h files

Eliminate camq_alloc() and camq_free()

These are no longer needed now that it's embedded in cam_ccbq. They are also
unused.

Reviewed by: ken, chuck
Differential Revision: https://reviews.freebsd.org/D24008

Revert r327828, r327949, r327953, r328016-r328026, r328041:
Uses of mallocarray(9).

The use of mallocarray(9) has rocketed the required swap to build FreeBSD.
This is likely caused by the allocation size attributes which put extra pressure
on the compiler.

Given that most of these checks are superfluous we have to choose better
where to use mallocarray(9). We still have more uses of mallocarray(9) but
hopefully this is enough to bring swap usage to a reasonable level.

Reported by: wosch
PR: 225197

cam: make some use of mallocarray(9).

Focus on code where we are doing multiplications within malloc(9). None of
these ire likely to overflow, however the change is still useful as some
static checkers can benefit from the allocation attributes we use for
mallocarray.

This initial sweep only covers malloc(9) calls with M_NOWAIT. No good
reason but I started doing the changes before r327796 and at that time it
was convenient to make sure the sorrounding code could handle NULL values.

X-Differential revision: https://reviews.freebsd.org/D13837

sys/cam: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

CAM ccbq sanity: checks on insert and remove

KASSERT in cam_ccbq_insert_ccb that only XPT_FC_QUEUED ops are queued,
and XPT_FC_USER_CCB ops are not. Otherwise cam_ccbq_ccb_done may be
skipped.

Bounds check the index used for camq_remove in order to panic instead
of scribble on removal of an out-of-bounds index (e.g. consider the
effect of camq_remove of CAM_UNQUEUED_INDEX).

KASSERT in cam_ccbq_remove_ccb that the ccb removed by index was the
one sought.

Submitted by: Ryan Libby <rlibby@gmail.com>
Reviewed by: imp, mav
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D8151

sys/cam: spelling fixes.

Affects a debug message.

MFC after: 2 weeks

Update CAM CCB accounting for the new status quo.

devq_openings counter lost its meaning after allocation queues has gone.
held counter is still meaningful, but problematic to update due to separate
locking of CCB allocation and queuing.

To fix that replace devq_openings counter with allocated counter. held is
now calculated on request as difference between number of allocated, queued
and active CCBs.

MFC after: 1 month

Merge CAM locking changes from the projects/camlock branch to radically
reduce lock congestion and improve SMP scalability of the SCSI/ATA stack,
preparing the ground for the coming next GEOM direct dispatch support.

Replace big per-SIM locks with bunch of smaller ones:
- per-LUN locks to protect device and peripheral drivers state;
- per-target locks to protect list of LUNs on target;
- per-bus locks to protect reference counting;
- per-send queue locks to protect queue of CCBs to be sent;
- per-done queue locks to protect queue of completed CCBs;
- remaining per-SIM locks now protect only HBA driver internals.

While holding LUN lock it is allowed (while not recommended for performance
reasons) to take SIM lock. The opposite acquisition order is forbidden.
All the other locks are leaf locks, that can be taken anywhere, but should
not be cascaded. Many functions, such as: xpt_action(), xpt_done(),
xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM
lock to be held.

To keep compatibility and solve cases where SIM lock can't be dropped, all
xpt_async() calls in addition to xpt_done() calls are queued to completion
threads for async processing in clean environment without SIM lock held.

Instead of single CAM SWI thread, used for commands completion processing
before, use multiple (depending on number of CPUs) threads. Load balanced
between them using "hash" of the device B:T:L address.

HBA drivers that can drop SIM lock during completion processing and have
sufficient number of completion threads to efficiently scale to multiple
CPUs can use new function xpt_done_direct() to avoid extra context switch.
Make ahci(4) driver to use this mechanism depending on hardware setup.

Sponsored by: iXsystems, Inc.
MFC after: 2 months

MFprojects/camlock r249505:
Change CCB queue resize logic to be able safely handle overallocations:
- (re)allocate queue space in power of 2 chunks with 64 elements minimum
and never shrink it; with only 4/8 bytes per element size is insignificant.
- automatically reallocate the queue to double size if it is overflowed.
- if queue reallocation failed, store extra CCBs in unsorted TAILQ,
fetching them back as soon as some queue element is freed.

To free space in CCB for TAILQ linking, change highpowerq from keeping
high-power CCBs to keeping devices frozen due to high-power CCBs.

This encloses all pieces of queue resize logic inside of cam_queue.[ch],
removing some not obvious duties from xpt_release_ccb().

MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905,
r248917, r248918, r248978, r249001, r249014, r249030:

Remove multilevel freezing mechanism, implemented to handle specifics of
the ATA/SATA error recovery, when post-reset recovery commands should be
allocated when queues are already full of payload requests. Instead of
removing frozen CCBs with specified range of priorities from the queue
to provide free openings, use simple hack, allowing explicit CCBs over-
allocation for requests with priority higher (numerically lower) then
CAM_PRIORITY_OOB threshold.

Simplify CCB allocation logic by removing SIM-level allocation queue.
After that SIM-level queue manages only CCBs execution, while allocation
logic is localized within each single device.

Suggested by: gibbs

Increase device CCB queue array size by CAM_RL_VALUES - 1 (4) elements.
It is required to store extra recovery requests in case of bus resets.
On ATA/SATA this fixes assertion panics on HEAD with INVARIANTS enabled or
possible memory corruptions otherwise if timeout/reset happens when device
CCB queue is already full.

Reported by: gibbs@
MFC after: 1 week

Change queue overflow checks from DIAGNOSTIC+panic() to KASSERT() to make
them enabled on HEAD by default. It is probably better to do single compare
then hunt for unexpected memory corruption.

Mark MALLOC_DEFINEs static that have no corresponding MALLOC_DECLAREs.

This means that their use is restricted to a single C file.

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

MFC r199279, r199280, r199281:
- Fix several device freeze counting bugs.
- Remove code that years ago was closing race between request submission
to SIM and device/SIM freeze. That race become impossible after moving from
spl to mutex locking, while this workaround causes some unexpected effects.

MFC r198372, r198377:
Implement cam_ccbq_fini().
This is effectively NULL change, but makes this API a bit more consistent.

MFp4:
Some more missed parts from previous commits.

After thinking again, implement cam_ccbq_fini().
This is effectively NULL change, but makes this API a bit more consistent.

- Import the HEAD csup code which is the basis for the cvsmode work.

- Providing fine-grained malloc statistic by replacing M_DEVBUF with
module-specific malloc types. These should help us to pinpoint the
possible memory leakage in the future.
- Implementing xpt_alloc_ccb_nowait() and replacing all malloc/free based
CCB memory management with xpt_alloc_ccb[_nowait]/xpt_free_ccb. Hopefully
this would be helpful if someday we move the CCB allocator to use UMA
instead of malloc().

Encouraged by: jeffr, rwatson
Reviewed by: gibbs, scottl
Approved by: re (scottl)

Start each of the license/copyright comments with /*-

Use __FBSDID().

Fix typos, mostly s/ an / a / where appropriate and a few s/an/and/
Add FreeBSD Id tag where missing.

$Id$ -> $FreeBSD$

Fix a long standing bug in the camq teardown code.

cam_periph.c:
Move handling of CAM_AUTOSENSE_FAIL into block dealing with
all other scsi status errors.

cam_queue.c:
cam_queue.h:
Fix 'off by one' heap bug in a more efficient manner. Since
heap algorithms like to deal with indexes started from 1,
offset our heap array pointer at allocation time to make this
so for a C environment. This makes the implementation of the
algorithm a bit more efficient.

cam_xpt.c:
Use macros for accessing the head of the heap so that code
is isolated from implementation details of the heap.

Remove camq_regen(). We already perform modular comparisons
for generation counts, so no further steps to deal with generation
count wrap are required.

Fix an off by one problem in the camq heap code.

CAM Transport Layer (XPT).

Submitted by: The CAM Team