subr_unit.c: another attempt to fix the build

Reported by: cy
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

kern/subr_unit.c: fix non-debug build

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

unr(9) iterator: add naive test

To use, compile userspace code e.g. into the subr_unit binary, then do
$ while ./subr_unit -iv >|/tmp/subr_unit.log ; do :; done
The loop should be left run for as long as possible.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D40089

unr(9): add 'show unrhdr_iter' command

Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D40089

unr(9): add iterator interface

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D40089

unr(9): add 'show unrhdr' ddb command

Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D40089

subr_unit.c: explain first/last special ranges

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D40089

kern/subr_unit.c: more uses for is_bitmap()

To facilitate it, move the is_bitmap() definition earlier.

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D40089

kern/subr_unit.c: some style

Use NBBY instead of spelling '8' literally.
Put space into the type specifier.

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D40089

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

unr: remove UNR64_LOCKED

All platforms support 64-bit atomics now.

subr_unit.c: make userspace tests buildable

by defining a placeholder for UNR_NO_MTX

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

unr(9): allow to avoid internal locking

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D35014

init_unrhdr(): make it usable by initializing everything

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D35014

kern: clean up empty lines in .c and .h files

Remove duplicated empty lines from kern/*.c

No functional changes.

kern/subr_unit: Rip srandomdev, random(3) out of dead code

The simulation cannot be reproduced, so the value of using a deterministic PRNG
like random(3) is dubious. The number of repitions used in the sample isn't a
problem for the Chacha implementation of arc4random we have today. (Also, no
one actually runs this code; it was provided as an example of the work the
author did validating the implementation. It's not even test code.)

Implement unr64

Important users of unr like tmpfs or pipes can get away with just
ever-increasing counters, making the overhead of managing the state
for 32 bit counters a pessimization.

Change it to an atomic variable. This can be further sped up by making
the counts variable "allocate" ranges and store them per-cpu.

Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D18054

sys/kern: adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Properly reset the fields in clean_unrhdr.

In r324542 I neglected to reset the first and last fields of struct
unrhdr. This causes a tmpfs to fail the unr(9) consistency checks with
DIAGNOSTIC on. Fix this by resetting the fields by calling init_unrhdr.
While here, change a loop to use TAILQ_FOREACH_SAFE since it is more
readable and equally fast.

Reported by: David Wolfskill <david@catwhisker.org>
Approved by: rstone (mentor)
Sponsored by: Dell EMC Isilon

Add clearing function for unr(9).

Previously before you could call unrhdr_delete you needed to
individually free every allocated unit. It is useful to be able to tear
down the unr without having to go through this process, as it is
significantly faster than freeing the individual units.

Reviewed by: cem, lidl
Approved by: rstone (mentor)
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12591

Clean up trailing whitespace

MFC after: 3 days
Sponsored by: Dell EMC Isilon

Fix -Wunused on gcc 4.9 (x was set but not used)

MFC after: 3 days
Sponsored by: Dell EMC Isilon

Fix build of kern/subr_unit.c, broken by r300539

Reported by: peter
Pointyhat to: asomers
Sponsored by: Spectra Logic Corp

Add bit_count to the bitstring(3) api

Add a bit_count function, which efficiently counts the number of bits set in
a bitstring.

sys/sys/bitstring.h
tests/sys/sys/bitstring_test.c
share/man/man3/bitstring.3
Add bit_alloc

sys/kern/subr_unit.c
Use bit_count instead of a naive counting loop in check_unrhdr, used
when INVARIANTS are enabled. The userland test runs about 6x faster
in a generic build, or 8.5x faster when built for Nehalem, which has
the POPCNT instruction.

sys/sys/param.h
Bump __FreeBSD_version due to the addition of bit_alloc

UPDATING
Add a note about the ABI incompatibility of the bitstring(3)
changes, as suggested by lidl.

Suggested by: gibbs
Reviewed by: gibbs, ngie
MFC after: 9 days
X-MFC-With: 299090, 300538
Relnotes: yes
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D6255

Add explicit cast to fix mips and powerpc build after r299090

Sponsored by: The FreeBSD Foundation

Improve performance and functionality of the bitstring(3) api

Two new functions are provided, bit_ffs_at() and bit_ffc_at(), which allow
for efficient searching of set or cleared bits starting from any bit offset
within the bit string.

Performance is improved by operating on longs instead of bytes and using
ffsl() for searches within a long. ffsl() is a compiler builtin in both
clang and gcc for most architectures, converting what was a brute force
while loop search into a couple of instructions.

All of the bitstring(3) API continues to be contained in the header file.
Some of the functions are large enough that perhaps they should be uninlined
and moved to a library, but that is beyond the scope of this commit.

sys/sys/bitstring.h:
Convert the majority of the existing bit string implementation from
macros to inline functions.

Properly protect the implementation from inadvertant macro expansion
when included in a user's program by prefixing all private
macros/functions and local variables with '_'.

Add bit_ffs_at() and bit_ffc_at(). Implement bit_ffs() and
bit_ffc() in terms of their "at" counterparts.

Provide a kernel implementation of bit_alloc(), making the full API
usable in the kernel.

Improve code documenation.

share/man/man3/bitstring.3:
Add pre-exisiting API bit_ffc() to the synopsis.

Document new APIs.

Document the initialization state of the bit strings
allocated/declared by bit_alloc() and bit_decl().

Correct documentation for bitstr_size(). The original code comments
indicate the size is in bytes, not "elements of bitstr_t". The new
implementation follows this lead. Only hastd assumed "elements"
rather than bytes and it has been corrected.

etc/mtree/BSD.tests.dist:
tests/sys/Makefile:
tests/sys/sys/Makefile:
tests/sys/sys/bitstring.c:
Add tests for all existing and new functionality.

include/bitstring.h
Include all headers needed by sys/bitstring.h

lib/libbluetooth/bluetooth.h:
usr.sbin/bluetooth/hccontrol/le.c:
Include bitstring.h instead of sys/bitstring.h.

sbin/hastd/activemap.c:
Correct usage of bitstr_size().

sys/dev/xen/blkback/blkback.c
Use new bit_alloc.

sys/kern/subr_unit.c:
Remove hard-coded assumption that sizeof(bitstr_t) is 1. Get rid of
unrb.busy, which caches the number of bits set in unrb.map. When
INVARIANTS are disabled, nothing needs to know that information.
callapse_unr can be adapted to use bit_ffs and bit_ffc instead.
Eliminating unrb.busy saves memory, simplifies the code, and
provides a slight speedup when INVARIANTS are disabled.

sys/net/flowtable.c:
Use the new kernel implementation of bit-alloc, instead of hacking
the old libc-dependent macro.

sys/sys/param.h
Update __FreeBSD_version to indicate availability of new API

Submitted by: gibbs, asomers
Reviewed by: gibbs, ngie
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D6004

Automate the subr_unit test.

Build and install the subr_unit test program originally written by phk, and
run it with the other ATF tests.

tests/sys/kern/Makefile
* Build and install the subr_unit test as a plain test

sys/kern/subr_unit.c
* Reduce the default number of repetitions from 100 to 1, and add a
command-line parser to override it.
* Don't be so noisy by default
* Fix an include problem for the test build

Reviewed by: ngie
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D6038

Move the definition of the struct unrhdr into a separate header file,
to allow embedding the struct. Add init_unrhdr(9) initializer, which
sets up preallocated unrhdr.

Reviewed by: alc
Tested by: pho, bf

Correct double "the the"

Approved by: cperciva
MFC after: 3 days

Fix typos - remove duplicate "the".

PR: bin/154928
Submitted by: Eitan Adler <lists at eitanadler.com>
MFC after: 3 days

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Remove redundant high >= 0.

Reported by: rstone

Assert that low and high are >= 0. The allocator doesn't support the
negative range.

Extend the kernel unit number allocator for allocating specific unit
numbers. This change adds a new function alloc_unr_specific() which
returns the requested unit number if it is free. If the number is
already allocated or out of the range, -1 is returned.

Update alloc_unr(9) manual page accordingly and add a MLINK for
alloc_unr_specific(9).

Discussed on: freebsd-hackers

Correct a comment typo.

- Fix compilation of the subr_unit.c user space test program.
- Use %zu for size_t in a few format strings.

- Import the HEAD csup code which is the basis for the cvsmode work.

Since cdev mutex is after system map mutex in global lock order, free()
shall not be called while holding cdev mutex. devfs_inos unrhdr has cdev as
mutex, thus creating this LOR situation.

Postpone calling free() in kern/subr_unit.c:alloc_unr() and nested functions
until the unrhdr mutex is dropped. Save the freed items on the ppfree list
instead, and provide the clean_unrhdrl() and clean_unrhdr() functions to
clean the list.
Call clean_unrhdrl() after devfs_create() calls immediately before
dropping cdev mutex. devfs_create() is the only user of the alloc_unrl()
in the tree.

Reviewed by: phk
Tested by: Peter Holm
LOR: 80
Approved by: re (kensmith)

Add the witness warning for free_unr. Function could sleep, thus callers
shall not have any non-sleepable locks held.

Submitted by: Hans Petter Selasky <hselasky c2i net>
Approved by: re (kensmith)

Remove debugging printfs.

Reengineer subr_unit

Add support for passing in a mutex. If NULL is passed a global
subr_unit mutex is used.

Add alloc_unrl() which expects the mutex to be held.

Allocating a unit will never sleep as it does not need to allocate
memory.

Cut possible range in half so we can use -1 to mean "out of number".

Collapse first and last runs into the head by means of counters.
This saves memory in the common case(s).

Add placeholder mutex argument to new_unrhdr().

/* -> /*- for copyright notices, minor format tweaks as necessary

Add delete_unrhdr() function.

It will fail fatally if all allocated numbers have not been returned first.

Fix a typo to fix the !DIAGNOSTIC build.

Submitted by: many

Add a new API for allocating unit number (-like) resources.

Allocation is always lowest free unit number.

A mixed range/bitmap strategy for maximum memory efficiency. In
the typical case where no unit numbers are freed total memory usage
is 56 bytes on i386.

malloc is called M_WAITOK but no locking is provided (yet). A bit of
experience will be necessary to determine the best strategy. Hopefully
a "caller provides locking" strategy can be maintained, but that may
require use of M_NOWAIT allocation and failure handling.

A userland test driver is included.