bitset: Add ORNOT macros

Macros to ANDNOT a bitset currently exist, but there are no ORNOT
equivalents. Introduce ORNOT macros for bitset(9), cpuset(9), and
domainset(9).

Approved by: markj (mentor)
Reviewed by: markj
MFC after: 1 week
Sponsored by: NIKSUN, Inc.
Differential Revision: https://reviews.freebsd.org/D44976

accf_tls: accept filter that waits for TLS handshake header

Add SYSCTL_ADD_CONST_STRING(9)

Catch up with 10a1e981d (2019-03-19) so that one can do
'man SYSCTL_ADD_CONST_STRING' and not get
"No manual entry for SYSCTL_ADD_CONST_STRING".

Reviewed by: sjg

share/man/man9/Makefile: fix buildworld

cdevpriv(9): document devfs_foreach_cdevpriv()

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

man9: mlink for counter_ratecheck(9)

rman: Add rman_get/set_type

This permits associating a resource type (e.g. SYS_RES_MEMORY) with a
struct resource.

I considered adding a new field to struct rman to store the type and
only providing rman_get_type as an accessor. However, changing
'struct rman' is an ABI breakage. I might revisit this in main, but
the current approach is MFC'able.

Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D44122

callout: retire callout_async_drain()

This function was used only in TCP before 446ccdd08e2a. It was born in
pain in 2016 to plug different complex panics in TCP timers. It wasn't
warmly accepted in phabricator by all of the reviewers and my recollection
of overall agreement was that "if you need this KPI, then you'd better fix
your code to not need it". However, the function served its duty well all
the way to FreeBSD 14. But now that TCP doesn't need it anymore, let's
retire it to reduce complexity of callout code and also to avoid its
further use.

Reviewed by: jhb, markj, kib, rrs
Differential Revision: https://reviews.freebsd.org/D43546

subr_bus: introduce device_set_descf() and modify allocation logic

device_set_descf() is a printf-like version of device_set_desc().

Allocation code has been transferred from device_set_desc_internal() to
device_set_desc_copy() and device_set_descf() to avoid complicating
device_set_desc_internal(). The "copy" argument in
device_set_desc_internal() has been replaced with a flag which is set
when the description string has been allocated with M_BUS.

Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Reviewed by: imp, markj
Differential Revision: https://reviews.freebsd.org/D43370

pmap_init(9): drop MLINKS reference

fix the build

pmap_init(9): sweep references to pmap_init2()

gone since 2005

pmap_pinit(9): sweep references to pmap_pinit2()

gone since 2004

mbuf.9: Document mtodo

mtodo() accepts an mbuf and offset and returns a void * pointer to the
requested offset into the mbuf's associated data. Similar to mtod(),
no bounds checking is performed.

Reviewed by: imp, markj
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D43215

kmsan: Add kmsan_check_uio()

This was handy for some ad-hoc debugging and fits in with other
kmsan_check_*() routines which operate on some kind of data container.

MFC after: 1 week
Sponsored by: The FreeBSD Foundation

pmap: add pmap_kextract(9) man page

Add a man page for pmap_kextract(9), with alias to vtophys(9). This man
page is based on pmap_extract(9).

Add it as cross reference in pmap(9), and add comments above the
function implementations.

Co-authored-by: Graham Perrin <grahamperrin@gmail.com>
Co-authored-by: mhorne
Sponsored by: The FreeBSD Foundation
Pull Request: https://github.com/freebsd/freebsd-src/pull/827

net80211: remove ieee80211_unref_node()

ieee80211_unref_node() was only used in two error cases in
ieee80211_send_nulldata(). There we do not need to guard against
ni pointer reuse after decrementing the refcount of the ni as we
only update the stats and return.

Update the man page and remove the link for the now gone function.

Sponsored by: The FreeBSD Foundation
X-MFC: never
Reviewed by: adrian, emaste
Differential Revision: https://reviews.freebsd.org/D42035

realgroupmember(9): Link to groupmember(9), document the function

Reviewed by: bcr (older version), mhorne
MFC after: 2 weeks
Sponsored by: Kumacom SAS
Differential Revision: https://reviews.freebsd.org/D40643

cr_bsd_visible(9): New man page

Reviewed by: bcr, pauamma_gundo.com
MFC after: 2 weeks
Sponsored by: Kumacom SAS
Differential Revision: https://reviews.freebsd.org/D40632

cr_canseejailproc(9): New man page

Reviewed by: pauamma_gundo.com, mhorne
MFC after: 2 weeks
Sponsored by: Kumacom SAS
Differential Revision: https://reviews.freebsd.org/D40631

cr_canseeotheruids(), cr_canseeothergids(): Man pages: Impacts of rename

When these functions were renamed 7 years ago, their man pages were not.
Rename the latter in accordance and fix the names inside them. Fix
references to them as well. Add the old man pages to the list of
obsolete files.

Reviewed by: mhorne
MFC after: 2 weeks
Sponsored by: Kumacom SAS
Differential Revision: https://reviews.freebsd.org/D40630

kqueue(9): sweep references to knlist_remove_inevent()

knlist_remove_inevent() gone in 9e590ff04b687e910579a5851d95cedf9eb10bfd

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

g_bio(9): add link to bio(9)

This page documents 'struct bio', so this can be helpful with finding or
referencing it.

Reviewed by: imp, markj
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D41252

signal: Remove gsignal references from manuals

Reported by: Mina Galić, mhorne
Reviewed by: Mina Galić, mhorne
Differential Revision: https://reviews.freebsd.org/D41020
MFC after: 1 week

ddb: Rework macros to make it easier to add new command tables.

- Add new DB_DEFINE_TABLE and DB_DECLARE_TABLE macros to define new
command tables. DB_DECLARE_TABLE is intended for use in headers
similar to MALLOC_DECLARE and SYSCTL_DECL.

DB_DEFINE_TABLE takes three arguments, the name of the parent table,
the command name, and the name of the table itself, e.g.
DB_DEFINE_TABLE(show, foo, show_foo) defines a new "show foo" table.

- DB_TABLE_COMMAND, DB_TABLE_COMMAND_FLAGS, DB_TABLE_ALIAS, and
DB_ALIAS_FLAGS allow new commands and aliases to be defined. These
are similar to the existing DB_COMMAND, etc. except that they take
an initial argument giving the name of the parent table, e.g.:

DB_TABLE_COMMAND(show_foo, bar, db_show_foo_bar)

defines a new "show foo bar" command.

This provides a cleaner interface than the ad-hoc use of internal
macros like _DB_SET that was required previously (e.g. in cxgbe(4)).

This retires DB_FUNC macro as well as the internal _DB_FUNC macro.

Reviewed by: melifaro, kib, markj
Differential Revision: https://reviews.freebsd.org/D40819

man9: Add a manual page for kstack_contains(9).

Approved by: gnn (mentor)
Reviewed by: mhorne
Differential Revision: https://reviews.freebsd.org/D39933

printf(9): add MLINKS for vlog(9) and vprintf(9)

MFC after: 3 days
Sponsored by: The FreeBSD Foundation

kern_reboot(9): document shutdown_nice()

Reviewed by: markj, Pau Amma <pauamma@gundo.com>
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39134

kern_reboot(9): some updates

- This function no longer disables interrupts
- MLINK to reboot.9
- The mentions of autoconfiguration is more about shutdown_nice(),
coming in the next commit.
- Describe the RB_* flags relevant to this function
- Describe behaviour when shutdown hooks fail the reset
- Describe expected execution contexts
- Add FF copyright
- xref panic(9)
- xref this page in reboot(2)

Reviewed by: markj
Discussed with: rpokala, Pau Amma <pauamma@gundo.com>
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39133

panic(9): some updates

- Better description of what the panic() function does
- Document KERNEL_PANICKED()
- Add a section describing panic execution context
- Add SEE ALSO

Reviewed by: kib, markj, rpokala
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39132

KASSERT(9): some updates

- Add a little bit of introductory text
- Improve the existing example: ANSI C, use a better assertion than a
NULL check (which is discouraged)
- Document the widely used MPASS macro in this page
- Drop the cross-reference to config(8)

Reviewed by: kib, markj, rpokala, Pau Amma <pauamma@gundo.com>
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39131

critical(9): small updates

- Document CRITICAL_ASSERT() in this man page.
- Clarify that a thread may also handle interrupts in a critical
section, not only faults/exceptions.
- Note the negative effects of critical section abuse
- Some other minor clarifications
- Add short SEE ALSO

Reviewed by: kib, markj, rpokala, Pau Amma <pauamma@gundo.com>
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39130

vcount.9: remove obsolete man page

These functions, vcount() and count_dev(), no longer exist in the source
tree.

Reviewed by: rpokala
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39128

man9: Add an smr(9) manual page

Also update the UMA manual page to mention its SMR-enabled
functionality, and update locking.9 to mention both epoch and SMR.
Details of its usage are provided in the SMR manual page.

Reviewed by: Olivier Certner, mhorne, kib
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D38108

man9: Add man page for kern_yield() and friends

The page is fairly simple and will be referenced by mi_switch(9).
Provide some usage notes so that the broader implications of how/when to
use these functions are understood.

Reviewed by: kib, markj
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D38182

rtalloc.9: remove obsolete man page

This KPI was removed in d22337254564. Note that there are a handful of
references remaining in the src tree to these rtalloc functions that
could be cleaned up by someone with more domain knowledge.

Reviewed by: pauamma (manpages), glebius, melifaro
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D38188

PCBGROUP.9: remove obsolete man page

The PCBGROUP option and KPI were removed entirely in 93c67567e015.

Reviewed by: pauamma (manpages), glebius, melifaro
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D38187

man9: Add MLINKS for dpcpu macros

MFC after: 1 week

man9: add vm_map_unwire link

Reported by: markj (in D37416)
Sponsored by: The FreeBSD Foundation

man9: correct vm_page_wire.9 links

PR: 244596
Reviewed by: markj
Fixes: fee2a2fa3983 ("Change synchonization rules for ...")
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D37416

man9: remove duplicate superio_find_dev link

PR: 244596
Fixes: c812bea351021 ("add superio.4 and superio.9 ...")
Sponsored by: The FreeBSD Foundation

intr_event(9): document intr_event_handle()

Reviewed by: pauamma
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D36934

ithread(9): update functions to current day

The public KPI is now intr_event_**,
- Convert existing documented functions to their equivalents.
- Fix up the function arguments
- Fix up the possible error return values for each
- Remove ithread_schedule() completely
- Rename man page to intr_event(9)
- Update cross-references

Future changes will update the descriptive text for these functions.

PR: 100803
Based on work by: trhodes
Reviewed by: jhb
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33475

malloc(9): update names and hardlinks

Give all documented functions a .Nm entry in the man page, following the
order they are listed in the synopsis. Create MLINKs for each of the
functions as well.

While here, add a missing include directive to the synopsis, and appease
mandoc by wrapping a long line.

Reviewed by: markj, imp (previous version), jhb (previous version)
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D36940

ofw_graph(9): hook it up to the Makefile

Otherwise, the man page is not installed. Add appropriate MLINKS.

Reviewed by: manu
MFC after: 3 days
Fixes: 9a4eed0be20c ("ofw_graph: Add functions for graph bindings")
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D36932

man9: Add MLINKs for bus_space_{peek,poke}

MFC after: 1 week

Unbreak the build after b3ee318b79d54a59190d35b8c76a63a8fb81b903

domains: rewrite documentation to describe present state

Reviewed by: debdrup, pauamma
Differential revision: https://reviews.freebsd.org/D36513

m_get3: add an MLINK for the manpage

Reviewed by: kp
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D36319

kqueue: retire knlist_init_rw_reader()

Last usage was removed in afa85850e79c1839ec33efa1138206687b952cfa.

Reviewed by: pauamma, melifaro, kib
Differential Revision: https://reviews.freebsd.org/D36205

DB_COMMAND(9): update to mention additional macros

Document the existing alias definitions, and augment the example with
one of these. Also, describe the purpose of the newly added _FLAGS
variations of these command definitions.

Make some small style improvements to appease mandoc -Tlint.

Reviewed by: markj
MFC after: 3 days
Sponsored by: Juniper Networks, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D35664

OCF: Add crypto_clonereq().

This function clones an existing crypto request, but associates the
new request with a specified session. The intended use case is for
drivers to be able to fall back to software by cloning a request and
dispatch it to an internally allocated software session.

Reviewed by: markj
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D33607

mount: retire kernel_vmount()

The last usage of this function was removed in e3b1c847a4237ad9.

There are no in-tree consumers of kernel_vmount().

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D32607

vm_page: Remove vm_page_sbusy() and vm_page_xbusy()

They are unused today and cannot be safely used in the face of unlocked
lookup, in which pages may be busied without the object lock held.

Obtained from: jeff (object_concurrency patches)
Reviewed by: kib
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D32948

vm_page: Consolidate page busy sleep mechanisms

- Modify vm_page_busy_sleep() and vm_page_busy_sleep_unlocked() to take
a VM_ALLOC_* flag indicating whether to sleep on shared-busy, and fix
up callers.
- Modify vm_page_busy_sleep() to return a status indicating whether the
object lock was dropped, and fix up callers.
- Convert callers of vm_page_sleep_if_busy() to use vm_page_busy_sleep()
instead.
- Remove vm_page_sleep_if_(x)busy().

No functional change intended.

Obtained from: jeff (object_concurrency patches)
Reviewed by: kib
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D32947

Rewrite the vm_page_alloc manual page

Document the new allocator variants and flesh out the description of
some details of the page allocator interface.

Reviewed by: kib, alc
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32035

boot(9): update to match reality

This function was renamed to kern_reboot() in 2010, but the man page has
failed to keep in sync. Bring it up to date on the rename, add the
shutdown hooks to the synopsis, and document the (obvious) fact that
kern_reboot() does not return.

Fix an outdated reference to the old name in kern_reboot(), and leave a
reference to the man page so future readers might find it before any
large changes.

Reviewed by: imp, markj
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32085

bitset(9): Introduce BIT_FOREACH_ISSET and BIT_FOREACH_ISCLR

These allow one to non-destructively iterate over the set or clear bits
in a bitset. The motivation is that we have several code fragments
which iterate over a CPU set like this:

while ((cpu = CPU_FFS(&cpus)) != 0) {
cpu--;
CPU_CLR(cpu, &cpus);
<do something>;
}

This is slow since CPU_FFS begins the search at the beginning of the
bitset each time. On amd64 and arm64, CPU sets have size 256, so there
are four limbs in the bitset and we do a lot of unnecessary scanning.

A second problem is that this is destructive, so code which needs to
preserve the original set has to make a copy. In particular, we have
quite a few functions which take a cpuset_t parameter by value, meaning
that each call has to copy the 32 byte cpuset_t.

The new macros address both problems.

Reviewed by: cem, kib
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32028

device: add device_get_property and device_has_property

Generialize bus specific property accessors. Those functions allow driver code
to access device specific information.

Currently there is only support for FDT and ACPI buses.

Reviewed by: manu, mw
Sponsored by: Semihalf
Differential revision: https://reviews.freebsd.org/D31597

man: A trailing space cleanup in man9's Makefile

Sponsored by: The FreeBSD Foundation

Add gone_in(9) man page

Reviewed by: imp
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D30703

domain: give domains a chance to probe for availability

This gives any given domain a chance to indicate that it's not actually
supported on the current system. If dom_probe isn't supplied, we assume
the domain is universally applicable as most of them are. Keeping
fully-initialized and registered domains around that physically can't
work on a large majority of FreeBSD deployments is sub-optimal and leads
to errors that aren't consistent with the reality of why the socket
can't be created (e.g. ESOCKTNOSUPPORT) because such scenario has to be
caught upon pru_attach, at which point kicking back the more-appropriate
EAFNOSUPPORT would seem weird.

The initial consumer of this will be hvsock, which is only available on
HyperV guests.

Reviewed by: cem (earlier version), bcr (manpages)
Differential Revision: https://reviews.freebsd.org/D25062

kmsan: Add a manual page

Sponsored by: The FreeBSD Foundation

man: Install more man pages unconditionally

Add more manual pages which were not spotted previously in 0a0f7486413c

Ideally to be MFH'ed with:

8539518055d0 - Remove manpages from OLD_FILES
8b487b8292e4 - Fix bsd.subdir.mk-related issues after 0a0f7486413c
f6043a672135 - ObsoleteFiles.inc: Remove manpages from OLD_FILES
0a0f7486413c - man: Build manpages for all architectures

There is at least one pending issue when building with -DNO_ROOT.

Reported by: ceri@
MFH: 4 weeks
Discussed with: wosch
Differential Revision: https://reviews.freebsd.org/D31018

Add fspacectl(2), vn_deallocate(9) and VOP_DEALLOCATE(9).

fspacectl(2) is a system call to provide space management support to
userspace applications. VOP_DEALLOCATE(9) is a VOP call to perform the
deallocation. vn_deallocate(9) is a public KPI for kmods' use.

The purpose of proposing a new system call, a KPI and a VOP call is to
allow bhyve or other hypervisor monitors to emulate the behavior of SCSI
UNMAP/NVMe DEALLOCATE on a plain file.

fspacectl(2) comprises of cmd and flags parameters to specify the
space management operation to be performed. Currently cmd has to be
SPACECTL_DEALLOC, and flags has to be 0.

fo_fspacectl is added to fileops.
VOP_DEALLOCATE(9) is added as a new VOP call. A trivial implementation
of VOP_DEALLOCATE(9) is provided.

Sponsored by: The FreeBSD Foundation
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D28347

Add vnode_pager_purge_range(9) KPI

This KPI is created in addition to the existing vnode_pager_setsize(9)
KPI. The KPI is intended for file systems that are able to turn a range
of file into sparse range, also known as hole-punching.

Sponsored by: The FreeBSD Foundation
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D27194

Add some missing osd.9 MLINKs

MFC after: 1 week
Sponsored by: The FreeBSD Foundation

nvmem(9): install the manpage

This is being installed on all architectures in line with the OF_*
pages.

Discussed with: fernape, manu

newbus: Move from bus_child_{pnpinfo,location}_src to bus_child_{pnpinfo,location} with sbuf

Now that the upper layers all go through a layer to tie into these
information functions that translates an sbuf into char * and len. The
current interface suffers issues of what to do in cases of truncation,
etc. Instead, migrate all these functions to using struct sbuf and these
issues go away. The caller is also in charge of any memory allocation
and/or expansion that's needed during this process.

Create a bus_generic_child_{pnpinfo,location} and make it default. It
just returns success. This is for those busses that have no information
for these items. Migrate the now-empty routines to using this as
appropriate.

Document these new interfaces with man pages, and oversight from before.

Reviewed by: jhb, bcr
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D29937

regulator: Add manpage for this framework

Reviewed by: imp, mmel
Differential Revision: https://reviews.freebsd.org/D30762
Sponsored by: Diablotin Systems

man9: add hz(9) and hardclock(9)

Document aspects of system time keeping. Hz is the nominal rate that we
interrupt the system and is known and the 'tick' period of 1 / hz.
hardclock is the routine that does various bits of timekeeping. stathz
and profhz are documented as historical relics that are deprecated
and replaced by hwpmc.4 and others.

Reviewed by: phk@, mav@ and gnn@ (previous version)
Obtained from: hardclock.9 from NetBSD (with FreeBSD adjustments)
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D30802

man9: Sort unr in Makefile MLINKS

Sort unr entries in alphabetical order, like everything else in the
MLINKS section.

Sponsored by: Netflix

crypto: Remove now-unused crypto_cursor_seg{base,len}.

Callers should use crypto_cursor_segment() instead.

Reviewed by: markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D30448

man: document ether_gen_addr(9)

This KPI is used to assign a MAC address to an interface that doesn't
already have one assigned.

Reviewed by: bcr, gnn, imp, kbowling, kp
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D29787

crypto: Add crypto_cursor_segment() to fetch both base and length.

This function combines crypto_cursor_segbase() and
crypto_cursor_seglen() into a single function. This is mostly
beneficial in the unmapped mbuf case where back to back calls of these
two functions have to iterate over the sub-components of unmapped
mbufs twice.

Bump __FreeBSD_version for crypto drivers in ports.

Suggested by: markj
Reviewed by: markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D30445

sglist: Add sglist_append_single_mbuf().

This function appends the contents of a single mbuf to an sglist
rather than an entire mbuf chain.

Reviewed by: gallatin, markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D30135

Add the KASAN runtime

KASAN enables the use of LLVM's AddressSanitizer in the kernel. This
feature makes use of compiler instrumentation to validate memory
accesses in the kernel and detect several types of bugs, including
use-after-frees and out-of-bounds accesses. It is particularly
effective when combined with test suites or syzkaller. KASAN has high
CPU and memory usage overhead and so is not suited for production
environments.

The runtime and pmap maintain a shadow of the kernel map to store
information about the validity of memory mapped at a given kernel
address.

The runtime implements a number of functions defined by the compiler
ABI. These are prefixed by __asan. The compiler emits calls to
__asan_load*() and __asan_store*() around memory accesses, and the
runtime consults the shadow map to determine whether a given access is
valid.

kasan_mark() is called by various kernel allocators to update state in
the shadow map. Updates to those allocators will come in subsequent
commits.

The runtime also defines various interceptors. Some low-level routines
are implemented in assembly and are thus not amenable to compiler
instrumentation. To handle this, the runtime implements these routines
on behalf of the rest of the kernel. The sanitizer implementation
validates memory accesses manually before handing off to the real
implementation.

The sanitizer in a KASAN-configured kernel can be disabled by setting
the loader tunable debug.kasan.disable=1.

Obtained from: NetBSD
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D29416

Remove reference to the pfctlinput2() from domain(9) after 237c1f932b.

Reviewed by: glebius
MFC After: 1 week
Differential Revision: https://reviews.freebsd.org/D29751

OCF: Remove support for asymmetric cryptographic operations.

There haven't been any non-obscure drivers that supported this
functionality and it has been impossible to test to ensure that it
still works. The only known consumer of this interface was the engine
in OpenSSL < 1.1. Modern OpenSSL versions do not include support for
this interface as it was not well-documented.

Reviewed by: cem
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D29736

Document vnode_pager_setsize(9)

MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Reviewed by: bcr
Approved by: philip (mentor)
Differential Revision: https://reviews.freebsd.org/D29408

config_intrhook: provide config_intrhook_drain

config_intrhook_drain will remove the hook from the list as
config_intrhook_disestablish does if the hook hasn't been called. If it has,
config_intrhook_drain will wait for the hook to be disestablished in the normal
course (or expedited, it's up to the driver to decide how and when
to call config_intrhook_disestablish).

This is intended for removable devices that use config_intrhook and might be
attached early in boot, but that may be removed before the kernel can call the
config_intrhook or before it ends. To prevent all races, the detach routine will
need to call config_intrhook_train.

Sponsored by: Netflix, Inc
Reviewed by: jhb, mav, gde (in D29006 for man page)
Differential Revision: https://reviews.freebsd.org/D29005

Implement mallocarray_domainset(9) variant of mallocarray(9).

Reviewed by: kib @
MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking

share/man/man9: document zero_region(9)

The zero_region() kernel interface was previously undocumented.
Add a new zero_region(9) manual page to document it.

Submitted by: Ka Ho Ng <khng@freebsdfoundation.org>
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D28914

Add VOP_READ_PGCACHE(9)

PR: 253894
Reviewed by: gbe, rwatson
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D28980

bitset.9: add missing MLINKS

Add MLINKS for new bitset(9) APIs in r364796 /
f8782001804f674dc3aff53053af5ebc3c199f02 and
ae4a8e52072a87bfd49d553b9b14450c626269f8.

Reported by: trasz
Reviewed by: cem
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D27944

Add missing refcount.9 MLINKS

Add some missing nv(9) MLINKS

MFC after: 1 week

Add backlight subsystem

This is a simple subsystem that allow drivers to register as a backlight.
Each backlight creates a device node under /dev/backlight/backlightX and
an alias based on the name provided.

Relnotes: yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D26250

Remove stray line

Add devctl_notify(9) man page

Document the calls to send messages to userland via devctl.
devctl_notify will create a message for the specified system,
subsystem and type, optionally adding additional information.

Reviewed by: bcr
Differential Revision: https://reviews.freebsd.org/D26520

Document devctl_safe_quote_sb

This routine centralizes the knowledge needed for properly quoting
'value' in all key="value" items that appear in devctl messages.

Reviewed by: bcr
Differential Revision: https://reviews.freebsd.org/D26520

Add a devctl_process_running man page.

Reviewed by: bcr
Differential Revision: https://reviews.freebsd.org/D26520

Add getenv(9) boolean parsing functions

This adds the getenv_bool() function, to parse a boolean value from a
kernel environment variable or tunable. This works for traditional
boolean values like "0" and "1", and also "true" and "false"
(case-insensitive). These semantics do not yet apply to sysctls declared
using SYSCTL_BOOL with CTLFLAG_TUN (they still only parse 1 and 0).

Also added are two wrapper functions, getenv_is_true() and
getenv_is_false(). These are slightly simpler for callers wishing to
perform a single check of a configuration variable.

Reviewed by: jhb (slightly earlier version)
Sponsored by: NetApp, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D26270

Remove vm_map_create(9) KPI's manpage according to r364302

Submitted by: Ka Ho Ng <khng300@gmail.com>
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D26372

Add prng(9) API

Add prng(9) as a replacement for random(9) in the kernel.

There are two major differences from random(9) and random(3):

- General prng(9) APIs (prng32(9), etc) do not guarantee an
implementation or particular sequence; they should not be used for
repeatable simulations.

- However, specific named API families are also exposed (for now: PCG),
and those are expected to be repeatable (when so-guaranteed by the named
algorithm).

Some minor differences from random(3) and earlier random(9):

- PRNG state for the general prng(9) APIs is per-CPU; this eliminates
contention on PRNG state in SMP workloads. Each PCPU generator in an
SMP system produces a unique sequence.

- Better statistical properties than the Park-Miller ("minstd") PRNG
(longer period, uniform distribution in all bits, passes
BigCrush/PractRand analysis).

- Faster than Park-Miller ("minstd") PRNG -- no division is required to
step PCG-family PRNGs.

For now, random(9) becomes a thin shim around prng32(). Eventually I
would like to mechanically switch consumers over to the explicit API.

Reviewed by: kib, markj (previous version both)
Discussed with: markm
Differential Revision: https://reviews.freebsd.org/D25916

vfs: add VOP_STAT

The current scheme of calling VOP_GETATTR adds avoidable overhead.

An example with tmpfs doing fstat (ops/s):
before: 7488958
after: 7913833

Reviewed by: kib (previous version)
Differential Revision: https://reviews.freebsd.org/D25910

Remove free_domain() and uma_zfree_domain().

These functions were introduced before UMA started ensuring that freed
memory gets placed in domain-local caches. They no longer serve any
purpose since UMA now provides their functionality by default. Remove
them to simplyify the kernel memory allocator interfaces a bit.

Reviewed by: cem, kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D25937

Add missing bitset(9) MLINKS.

MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D25713

devstat(9): Update the man page to reflect the current implementation

- Rename devstat_add_entry to devstat_new_entry
- Update the description of devstat_trans_flags
- Add manpage aliases for devstat_start_transaction_bio and devstat_end_transaction_bio

PR: 157316
Submitted by: novel
Reviewed by: cem, bcr (mentor)
Approved by: bcr (mentor)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D25677

Add crypto_initreq() and crypto_destroyreq().

These routines are similar to crypto_getreq() and crypto_freereq() but
operate on caller-supplied storage instead of allocating crypto
requests from a UMA zone.

Reviewed by: markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D25691

Fixup for r360574: install new mlinks for sglist(9) and remove old ones.

Remove an extraneous line continuation from r361481.

Add support for optional separate output buffers to in-kernel crypto.

Some crypto consumers such as GELI and KTLS for file-backed sendfile
need to store their output in a separate buffer from the input.
Currently these consumers copy the contents of the input buffer into
the output buffer and queue an in-place crypto operation on the output
buffer. Using a separate output buffer avoids this copy.

- Create a new 'struct crypto_buffer' describing a crypto buffer
containing a type and type-specific fields. crp_ilen is gone,
instead buffers that use a flat kernel buffer have a cb_buf_len
field for their length. The length of other buffer types is
inferred from the backing store (e.g. uio_resid for a uio).
Requests now have two such structures: crp_buf for the input buffer,
and crp_obuf for the output buffer.

- Consumers now use helper functions (crypto_use_*,
e.g. crypto_use_mbuf()) to configure the input buffer. If an output
buffer is not configured, the request still modifies the input
buffer in-place. A consumer uses a second set of helper functions
(crypto_use_output_*) to configure an output buffer.

- Consumers must request support for separate output buffers when
creating a crypto session via the CSP_F_SEPARATE_OUTPUT flag and are
only permitted to queue a request with a separate output buffer on
sessions with this flag set. Existing drivers already reject
sessions with unknown flags, so this permits drivers to be modified
to support this extension without requiring all drivers to change.

- Several data-related functions now have matching versions that
operate on an explicit buffer (e.g. crypto_apply_buf,
crypto_contiguous_subsegment_buf, bus_dma_load_crp_buf).

- Most of the existing data-related functions operate on the input
buffer. However crypto_copyback always writes to the output buffer
if a request uses a separate output buffer.

- For the regions in input/output buffers, the following conventions
are followed:
- AAD and IV are always present in input only and their
fields are offsets into the input buffer.
- payload is always present in both buffers. If a request uses a
separate output buffer, it must set a new crp_payload_start_output
field to the offset of the payload in the output buffer.
- digest is in the input buffer for verify operations, and in the
output buffer for compute operations. crp_digest_start is relative
to the appropriate buffer.

- Add a crypto buffer cursor abstraction. This is a more general form
of some bits in the cryptosoft driver that tried to always use uio's.
However, compared to the original code, this avoids rewalking the uio
iovec array for requests with multiple vectors. It also avoids
allocate an iovec array for mbufs and populating it by instead walking
the mbuf chain directly.

- Update the cryptosoft(4) driver to support separate output buffers
making use of the cursor abstraction.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D24545

Retire the CRYPTO_F_IV_GENERATE flag.

The sole in-tree user of this flag has been retired, so remove this
complexity from all drivers. While here, add a helper routine drivers
can use to read the current request's IV into a local buffer. Use
this routine to replace duplicated code in nearly all drivers.

Reviewed by: cem
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D24450

Refactor driver and consumer interfaces for OCF (in-kernel crypto).

- The linked list of cryptoini structures used in session
initialization is replaced with a new flat structure: struct
crypto_session_params. This session includes a new mode to define
how the other fields should be interpreted. Available modes
include:

- COMPRESS (for compression/decompression)
- CIPHER (for simply encryption/decryption)
- DIGEST (computing and verifying digests)
- AEAD (combined auth and encryption such as AES-GCM and AES-CCM)
- ETA (combined auth and encryption using encrypt-then-authenticate)

Additional modes could be added in the future (e.g. if we wanted to
support TLS MtE for AES-CBC in the kernel we could add a new mode
for that. TLS modes might also affect how AAD is interpreted, etc.)

The flat structure also includes the key lengths and algorithms as
before. However, code doesn't have to walk the linked list and
switch on the algorithm to determine which key is the auth key vs
encryption key. The 'csp_auth_*' fields are always used for auth
keys and settings and 'csp_cipher_*' for cipher. (Compression
algorithms are stored in csp_cipher_alg.)

- Drivers no longer register a list of supported algorithms. This
doesn't quite work when you factor in modes (e.g. a driver might
support both AES-CBC and SHA2-256-HMAC separately but not combined
for ETA). Instead, a new 'crypto_probesession' method has been
added to the kobj interface for symmteric crypto drivers. This
method returns a negative value on success (similar to how
device_probe works) and the crypto framework uses this value to pick
the "best" driver. There are three constants for hardware
(e.g. ccr), accelerated software (e.g. aesni), and plain software
(cryptosoft) that give preference in that order. One effect of this
is that if you request only hardware when creating a new session,
you will no longer get a session using accelerated software.
Another effect is that the default setting to disallow software
crypto via /dev/crypto now disables accelerated software.

Once a driver is chosen, 'crypto_newsession' is invoked as before.

- Crypto operations are now solely described by the flat 'cryptop'
structure. The linked list of descriptors has been removed.

A separate enum has been added to describe the type of data buffer
in use instead of using CRYPTO_F_* flags to make it easier to add
more types in the future if needed (e.g. wired userspace buffers for
zero-copy). It will also make it easier to re-introduce separate
input and output buffers (in-kernel TLS would benefit from this).

Try to make the flags related to IV handling less insane:

- CRYPTO_F_IV_SEPARATE means that the IV is stored in the 'crp_iv'
member of the operation structure. If this flag is not set, the
IV is stored in the data buffer at the 'crp_iv_start' offset.

- CRYPTO_F_IV_GENERATE means that a random IV should be generated
and stored into the data buffer. This cannot be used with
CRYPTO_F_IV_SEPARATE.

If a consumer wants to deal with explicit vs implicit IVs, etc. it
can always generate the IV however it needs and store partial IVs in
the buffer and the full IV/nonce in crp_iv and set
CRYPTO_F_IV_SEPARATE.

The layout of the buffer is now described via fields in cryptop.
crp_aad_start and crp_aad_length define the boundaries of any AAD.
Previously with GCM and CCM you defined an auth crd with this range,
but for ETA your auth crd had to span both the AAD and plaintext
(and they had to be adjacent).

crp_payload_start and crp_payload_length define the boundaries of
the plaintext/ciphertext. Modes that only do a single operation
(COMPRESS, CIPHER, DIGEST) should only use this region and leave the
AAD region empty.

If a digest is present (or should be generated), it's starting
location is marked by crp_digest_start.

Instead of using the CRD_F_ENCRYPT flag to determine the direction
of the operation, cryptop now includes an 'op' field defining the
operation to perform. For digests I've added a new VERIFY digest
mode which assumes a digest is present in the input and fails the
request with EBADMSG if it doesn't match the internally-computed
digest. GCM and CCM already assumed this, and the new AEAD mode
requires this for decryption. The new ETA mode now also requires
this for decryption, so IPsec and GELI no longer do their own
authentication verification. Simple DIGEST operations can also do
this, though there are no in-tree consumers.

To eventually support some refcounting to close races, the session
cookie is now passed to crypto_getop() and clients should no longer
set crp_sesssion directly.

- Assymteric crypto operation structures should be allocated via
crypto_getkreq() and freed via crypto_freekreq(). This permits the
crypto layer to track open asym requests and close races with a
driver trying to unregister while asym requests are in flight.

- crypto_copyback, crypto_copydata, crypto_apply, and
crypto_contiguous_subsegment now accept the 'crp' object as the
first parameter instead of individual members. This makes it easier
to deal with different buffer types in the future as well as
separate input and output buffers. It's also simpler for driver
writers to use.

- bus_dmamap_load_crp() loads a DMA mapping for a crypto buffer.
This understands the various types of buffers so that drivers that
use DMA do not have to be aware of different buffer types.

- Helper routines now exist to build an auth context for HMAC IPAD
and OPAD. This reduces some duplicated work among drivers.

- Key buffers are now treated as const throughout the framework and in
device drivers. However, session key buffers provided when a session
is created are expected to remain alive for the duration of the
session.

- GCM and CCM sessions now only specify a cipher algorithm and a cipher
key. The redundant auth information is not needed or used.

- For cryptosoft, split up the code a bit such that the 'process'
callback now invokes a function pointer in the session. This
function pointer is set based on the mode (in effect) though it
simplifies a few edge cases that would otherwise be in the switch in
'process'.

It does split up GCM vs CCM which I think is more readable even if there
is some duplication.

- I changed /dev/crypto to support GMAC requests using CRYPTO_AES_NIST_GMAC
as an auth algorithm and updated cryptocheck to work with it.

- Combined cipher and auth sessions via /dev/crypto now always use ETA
mode. The COP_F_CIPHER_FIRST flag is now a no-op that is ignored.
This was actually documented as being true in crypto(4) before, but
the code had not implemented this before I added the CIPHER_FIRST
flag.

- I have not yet updated /dev/crypto to be aware of explicit modes for
sessions. I will probably do that at some point in the future as well
as teach it about IV/nonce and tag lengths for AEAD so we can support
all of the NIST KAT tests for GCM and CCM.

- I've split up the exising crypto.9 manpage into several pages
of which many are written from scratch.

- I have converted all drivers and consumers in the tree and verified
that they compile, but I have not tested all of them. I have tested
the following drivers:

- cryptosoft
- aesni (AES only)
- blake2
- ccr

and the following consumers:

- cryptodev
- IPsec
- ktls_ocf
- GELI (lightly)

I have not tested the following:

- ccp
- aesni with sha
- hifn
- kgssapi_krb5
- ubsec
- padlock
- safe
- armv8_crypto (aarch64)
- glxsb (i386)
- sec (ppc)
- cesa (armv7)
- cryptocteon (mips64)
- nlmsec (mips64)

Discussed with: cem
Relnotes: yes
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D23677

Sort UMA macros and create MLINKS for them

This patch is a follow-up to r344518.

Reported by: ngie

Reviewed by: hselasky
Differential Revision: https://reviews.freebsd.org/D24165

Remove struct callout_handle. Should have gone with r355732.

Remove the deprecated timeout(9) interface.

All in-tree consumers have been converted to callout(9).

Reviewed by: kib, markj
Differential Revision: https://reviews.freebsd.org/D22602

Create new wrapper function: bus_delayed_attach_children()

Delay the attachment of children, when requested, until after interrutps are
running. This is often needed to allow children to run transactions on i2c or
spi busses. It's a common enough idiom that it will be useful to have its own
wrapper.

Reviewed by: ian
Differential Revision: https://reviews.freebsd.org/D21465

bitset: rename confusing macro NAND to ANDNOT

s/BIT_NAND/BIT_ANDNOT/, and for CPU and DOMAINSET too. The actual
implementation is "and not" (or "but not"), i.e. A but not B.
Fortunately this does appear to be what all existing callers want.

Don't supply a NAND (not (A and B)) operation at this time.

Discussed with: jeff
Reviewed by: cem
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D22791

document taskqueue_start_threads_in_proc

While here, fix taskqueue_start_threads_cpuset that was documented under
old name of taskqueue_start_threads_pinned.

MFC after: 4 weeks

add superio.4 and superio.9 manual pages

This adds basic documentation on what the superio driver is and how
other drivers can interact with it. I decided to also document
superio's ivar accessors.

Reviewed by: bcr, brueffer (both manual contents only)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D21958

Change synchonization rules for vm_page reference counting.

There are several mechanisms by which a vm_page reference is held,
preventing the page from being freed back to the page allocator. In
particular, holding the page's object lock is sufficient to prevent the
page from being freed; holding the busy lock or a wiring is sufficent as
well. These references are protected by the page lock, which must
therefore be acquired for many per-page operations. This results in
false sharing since the page locks are external to the vm_page
structures themselves and each lock protects multiple structures.

Transition to using an atomically updated per-page reference counter.
The object's reference is counted using a flag bit in the counter. A
second flag bit is used to atomically block new references via
pmap_extract_and_hold() while removing managed mappings of a page.
Thus, the reference count of a page is guaranteed not to increase if the
page is unbusied, unmapped, and the object's write lock is held. As
a consequence of this, the page lock no longer protects a page's
identity; operations which move pages between objects are now
synchronized solely by the objects' locks.

The vm_page_wire() and vm_page_unwire() KPIs are changed. The former
requires that either the object lock or the busy lock is held. The
latter no longer has a return value and may free the page if it releases
the last reference to that page. vm_page_unwire_noq() behaves the same
as before; the caller is responsible for checking its return value and
freeing or enqueuing the page as appropriate. vm_page_wire_mapped() is
introduced for use in pmap_extract_and_hold(). It fails if the page is
concurrently being unmapped, typically triggering a fallback to the
fault handler. vm_page_wire() no longer requires the page lock and
vm_page_unwire() now internally acquires the page lock when releasing
the last wiring of a page (since the page lock still protects a page's
queue state). In particular, synchronization details are no longer
leaked into the caller.

The change excises the page lock from several frequently executed code
paths. In particular, vm_object_terminate() no longer bounces between
page locks as it releases an object's pages, and direct I/O and
sendfile(SF_NOCACHE) completions no longer require the page lock. In
these latter cases we now get linear scalability in the common scenario
where different threads are operating on different files.

__FreeBSD_version is bumped. The DRM ports have been updated to
accomodate the KPI changes.

Reviewed by: jeff (earlier version)
Tested by: gallatin (earlier version), pho
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20486

Revert a portion of r351628 that I did not mean to commit.

Reported by: mjg
MFC with: r351628

Extend uma_reclaim() to permit different reclamation targets.

The page daemon periodically invokes uma_reclaim() to reclaim cached
items from each zone when the system is under memory pressure. This
is important since the size of these caches is unbounded by default.
However it also results in bursts of high latency when allocating from
heavily used zones as threads miss in the per-CPU caches and must
access the keg in order to allocate new items.

With r340405 we maintain an estimate of each zone's usage of its
(per-NUMA domain) cache of full buckets. Start making use of this
estimate to avoid reclaiming the entire cache when under memory
pressure. In particular, introduce TRIM, DRAIN and DRAIN_CPU
verbs for uma_reclaim() and uma_zone_reclaim(). When trimming, only
items in excess of the estimate are reclaimed. Draining a zone
reclaims all of the cached full buckets (the previous behaviour of
uma_reclaim()), and may further drain the per-CPU caches in extreme
cases.

Now, when under memory pressure, the page daemon will trim zones
rather than draining them. As a result, heavily used zones do not incur
bursts of bucket cache misses following reclamation, but large, unused
caches will be reclaimed as before.

Reviewed by: jeff
Tested by: pho (an earlier version)
MFC after: 2 months
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D16667

Update and clean up the UMA man page.

- Fix warnings from igor and mandoc.
- Provide a brief description of the separation between zones and their
backend slab allocators.
- Document cache zones and secondary zones.
- Document the kernel config options added in r350659.
- Document the uma_zalloc_pcpu() and uma_zfree_pcpu() wrappers.
- Document uma_zone_reserve(), uma_zone_reserve_kva() and
uma_zone_prealloc().
- Document uma_zone_alloc() and uma_zone_freef().
- Add some missing MLINKs and Xrefs.

MFC after: 2 weeks

Follow r350693 to add a link for sbuf_nl_terminate(9)

Sponsored by: The FreeBSD Foundation

GEOM: Reduce unnecessary log interleaving with sbufs

Similar to what was done for device_printfs in r347229.

Convert g_print_bio() to a thin shim around g_format_bio(), which acts on an
sbuf; documented in g_bio.9.

Reviewed by: markj
Discussed with: rlibby
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D21165

seqc: add man page

Reviewed by: markj
Earlier version reviewed by: emaste, mjg, bcr, 0mp
Differential Revision: https://reviews.freebsd.org/D16744

Create a man page for VOP_COPY_FILE_RANGE(9).

r350315 created a Linux compatible copy_file_range(2) syscall.
It uses a VOP method called VOP_COPY_FILE_RANGE so that file systems,
such as the NFSv4.2 client can do file system specific copying.
For NFSv4.2, this allows the copying to be done locally on the NFS server,
avoiding transferring the data across the wire twice.

This is a new man page (content changed).

Reviewed by: kib, asomers
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D20584

pkgbase: move man pages from runtime-manual to runtime

We don't split the other man pages in their own package so do the same for runtime.

Reviewed by: bapt, gjb
Differential Revision: https://reviews.freebsd.org/D20962

Merge the vm_page hold and wire mechanisms.

The hold_count and wire_count fields of struct vm_page are separate
reference counters with similar semantics. The remaining essential
differences are that holds are not counted as a reference with respect
to LRU, and holds have an implicit free-on-last unhold semantic whereas
vm_page_unwire() callers must explicitly determine whether to free the
page once the last reference to the page is released.

This change removes the KPIs which directly manipulate hold_count.
Functions such as vm_fault_quick_hold_pages() now return wired pages
instead. Since r328977 the overhead of maintaining LRU for wired pages
is lower, and in many cases vm_fault_quick_hold_pages() callers would
swap holds for wirings on the returned pages anyway, so with this change
we remove a number of page lock acquisitions.

No functional change is intended. __FreeBSD_version is bumped.

Reviewed by: alc, kib
Discussed with: jeff
Discussed with: jhb, np (cxgbe)
Tested by: pho (previous version)
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D19247

Add an external mbuf buffer type that holds multiple unmapped pages.

Unmapped mbufs allow sendfile to carry multiple pages of data in a
single mbuf, without mapping those pages. It is a requirement for
Netflix's in-kernel TLS, and provides a 5-10% CPU savings on heavy web
serving workloads when used by sendfile, due to effectively
compressing socket buffers by an order of magnitude, and hence
reducing cache misses.

For this new external mbuf buffer type (EXT_PGS), the ext_buf pointer
now points to a struct mbuf_ext_pgs structure instead of a data
buffer. This structure contains an array of physical addresses (this
reduces cache misses compared to an earlier version that stored an
array of vm_page_t pointers). It also stores additional fields needed
for in-kernel TLS such as the TLS header and trailer data that are
currently unused. To more easily detect these mbufs, the M_NOMAP flag
is set in m_flags in addition to M_EXT.

Various functions like m_copydata() have been updated to safely access
packet contents (using uiomove_fromphys()), to make things like BPF
safe.

NIC drivers advertise support for unmapped mbufs on transmit via a new
IFCAP_NOMAP capability. This capability can be toggled via the new
'nomap' and '-nomap' ifconfig(8) commands. For NIC drivers that only
transmit packet contents via DMA and use bus_dma, adding the
capability to if_capabilities and if_capenable should be all that is
required.

If a NIC does not support unmapped mbufs, they are converted to a
chain of mapped mbufs (using sf_bufs to provide the mapping) in
ip_output or ip6_output. If an unmapped mbuf requires software
checksums, it is also converted to a chain of mapped mbufs before
computing the checksum.

Submitted by: gallatin (earlier version)
Reviewed by: gallatin, hselasky, rrs
Discussed with: ae, kp (firewalls)
Relnotes: yes
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20616

Implement API for draining EPOCH(9) callbacks.

The epoch_drain_callbacks() function is used to drain all pending
callbacks which have been invoked by prior epoch_call() function calls
on the same epoch. This function is useful when there are shared
memory structure(s) referred to by the epoch callback(s) which are not
refcounted and are rarely freed. The typical place for calling this
function is right before freeing or invalidating the shared
resource(s) used by the epoch callback(s). This function can sleep and
is not optimized for performance.

Differential Revision: https://reviews.freebsd.org/D20109
MFC after: 1 week
Sponsored by: Mellanox Technologies

Catch up with recent changes in pwmbus(9). The pwm(9) and pwmbus(9)
interfaces were unified into pwmbus(9), and the PWMBUS_CHANNEL_MAX method
was renamed PWMBUS_CHANNEL_COUNT. The pwmbus_attach_bus() function just
went away completely. Also, fix a few typos such as s/is/if/.

Add a VOP_BMAP(9) man page

Reviewed by: mckusick
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D20704

Add wakeup_any(), cheaper wakeup_one() for taskqueue(9).

wakeup_one() and underlying sleepq_signal() spend additional time trying
to be fair, waking thread with highest priority, sleeping longest time.
But in case of taskqueue there are many absolutely identical threads, and
any fairness between them is quite pointless. It makes even worse, since
round-robin wakeups not only make previous CPU affinity in scheduler quite
useless, but also hide from user chance to see CPU bottlenecks, when
sequential workload with one request at a time looks evenly distributed
between multiple threads.

This change adds new SLEEPQ_UNFAIR flag to sleepq_signal(), making it wakeup
thread that went to sleep last, but no longer in context switch (to avoid
immediate spinning on the thread lock). On top of that new wakeup_any()
function is added, equivalent to wakeup_one(), but setting the flag.
On top of that taskqueue(9) is switchied to wakeup_any() to wakeup its
threads.

As result, on 72-core Xeon v4 machine sequential ZFS write to 12 ZVOLs
with 16KB block size spend 34% less time in wakeup_any() and descendants
then it was spending in wakeup_one(), and total write throughput increased
by ~10% with the same as before CPU usage.

Reviewed by: markj, mmacy
MFC after: 2 weeks
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D20669

Make the warning intervals for deprecated crypto algorithms tunable.

New sysctl/tunables can now set the interval (in seconds) between
rate-limited crypto warnings. The new sysctls are:
- kern.cryptodev_warn_interval for /dev/crypto
- net.inet.ipsec.crypto_warn_interval for IPsec
- kern.kgssapi_warn_interval for KGSSAPI

Reviewed by: cem
MFC after: 1 month
Relnotes: yes
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D20555

Document sysctl nodes that translate their values.

This documents the behavior of sysctl_msec_to_ticks and
SYSCTL_{ADD,}_SBINTIME_[UM]SEC.

Reviewed by: cem
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D20596

Hook DEFINE_IFUNC.9 up to the build.

Reported by: pluknet
MFC with: r348003

device_printf: Use sbuf for more coherent prints on SMP

device_printf does multiple calls to printf allowing other console messages to
be inserted between the device name, and the rest of the message. This change
uses sbuf to compose to two into a single buffer, and prints it all at once.

It exposes an sbuf drain function (drain-to-printf) for common use.

Update documentation to match; some unit tests included.

Submitted by: jmg
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D16690

Track device's NUMA domain in ifnet & alloc ifnet from NUMA local memory

This commit adds new if_alloc_domain() and if_alloc_dev() methods to
allocate ifnets. When called with a domain on a NUMA machine,
ifalloc_domain() will record the NUMA domain in the ifnet, and it will
allocate the ifnet struct from memory which is local to that NUMA
node. Similarly, if_alloc_dev() is a wrapper for if_alloc_domain
which uses a driver supplied device_t to call ifalloc_domain() with
the appropriate domain.

Note that the new if_numa_domain field fits in an alignment pad in
struct ifnet, and so does not alter the size of the structure.

Reviewed by: glebius, kib, markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D19930

Revert r346410 and r346411

libkern in .PATH has too many filename conflicts with libc and my -DNO_CLEAN
tinderbox didn't catch that ahead of time. Mea culpa.

libkern: Bring in arc4random_uniform(9) from libc

It is a useful arc4random wrapper in the kernel for much the same reasons as
in userspace. Move the source to libkern (because kernel build is
restricted to sys/, but userspace can include any file it likes) and build
kernel and libc versions from the same source file.

Copy the documentation from arc4random_uniform(3) to the section 9 page.

While here, add missing arc4random_buf(9) symlink.

Sponsored by: Dell EMC Isilon

random(4): Add is_random_seeded(9) KPI

The imagined use is for early boot consumers of random to be able to make
decisions based on whether random is available yet or not. One such
consumer seems to be __stack_chk_init(), which runs immediately after random
is initialized. A follow-up patch will attempt to address that.

Reported by: many
Reviewed by: delphij (except man page)
Approved by: secteam(delphij)
Differential Revision: https://reviews.freebsd.org/D19926

Add man page for VOP_FDATASYNC(9)

Reviewed by: kib
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D19678

New pfil(9) KPI together with newborn pfil API and control utility.

The KPI have been reviewed and cleansed of features that were planned
back 20 years ago and never implemented. The pfil(9) internals have
been made opaque to protocols with only returned types and function
declarations exposed. The KPI is made more strict, but at the same time
more extensible, as kernel uses same command structures that userland
ioctl uses.

In nutshell [KA]PI is about declaring filtering points, declaring
filters and linking and unlinking them together.

New [KA]PI makes it possible to reconfigure pfil(9) configuration:
change order of hooks, rehook filter from one filtering point to a
different one, disconnect a hook on output leaving it on input only,
prepend/append a filter to existing list of filters.

Now it possible for a single packet filter to provide multiple rulesets
that may be linked to different points. Think of per-interface ACLs in
Cisco or Juniper. None of existing packet filters yet support that,
however limited usage is already possible, e.g. default ruleset can
be moved to single interface, as soon as interface would pride their
filtering points.

Another future feature is possiblity to create pfil heads, that provide
not an mbuf pointer but just a memory pointer with length. That would
allow filtering at very early stages of a packet lifecycle, e.g. when
packet has just been received by a NIC and no mbuf was yet allocated.

Differential Revision: https://reviews.freebsd.org/D18951

Remove documentation for the nonexistant cred_update_thread(9).

This was a tangential change submitted as part of D18930.

Submitted by: jack@gandi.net

Remove pbuf(9) manual page that is now nearly 100% outdated.

Add man page for device_delete_children.

Submitted by: Jeroen Ruigrok van der Werven
Differential Revision: https://reviews.freebsd.org/D18452

Add a pwm subsystem so we can configure pwm controller from kernel and userland.

The pwm subsystem consist of API for PWM controllers, pwmbus to register them
and a pwm(8) utility to talk to them from userland.

Reviewed by: oshgobo (capsicum), bcr (manpage), 0mp (manpage)
Differential Revision: https://reviews.freebsd.org/D17938

Add a taskqueue_quiesce(9) KPI.

This is similar to taskqueue_drain_all(9) but will wait for the queue
to become idle before returning instead of only waiting for
already-enqueued tasks to finish. This will be used in the opensolaris
compat layer.

PR: 227784
Reviewed by: cem
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D17975

Add malloc_domainset(9) and _domainset variants to other allocator KPIs.

Remove malloc_domain(9) and most other _domain KPIs added in r327900.
The new functions allow the caller to specify a general NUMA domain
selection policy, rather than specifically requesting an allocation from
a specific domain. The latter policy tends to interact poorly with
M_WAITOK, resulting in situations where a caller is blocked indefinitely
because the specified domain is depleted. Most existing consumers of
the _domain KPIs are converted to instead use a DOMAINSET_PREF() policy,
in which we fall back to other domains to satisfy the allocation
request.

This change also defines a set of DOMAINSET_FIXED() policies, which
only permit allocations from the specified domain.

Discussed with: gallatin, jeff
Reported and tested by: pho (previous version)
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D17418

Add a dev_refthread.9 document.

Reviewed by: kib
Sponsored by: Dell EMC Isilon
Differential: https://reviews.freebsd.org/D16897

Add a man page for ratecheck(9) and ppsratecheck(9).

Add efidev(4)/efirt(9)

Document efidev(4), provider of userland access to EFI Runtime Services. A link is created to efirtc(4), which handles the time-of-day clock side.

efirt(9) is the kernel side of this.

Reviewed by: imp, kib (earlier version)
Differential Revision: https://reviews.freebsd.org/D16696

atomic.9: Add missing MLINK for testandclear, thread_fence routines

Missed in r299912, r326982.

X-MFC-With: r299912, r326982
Sponsored by: Dell EMC Isilon

Add missing MLINK.

MFC after: 3 days

Document multi variants of *prop_alloc functions

Add documentation and symlinks for OF_getprop_alloc_multi
and OF_getencprop_alloc_multi functions.

Also while here fix copy-pasted .Dt value and add one more
failure condition for OF_getencprop_alloc.

Create a symlink for sodtor_set(9) to the socket(9) man page.

Add epoch(9) man page

Reviewed by: gallatin@
Approved by: sbruno@

Add a stub manual page for iflib(9).

Currently 'man -k iflib' would find you the right pages for iflib
documentation, namely iflibdd(9) and iflibdi(9) but 'man iflib' would leave
you in the dark. This allows both approaches to find the relevant
documentation.

Reviewed by: kmacy, shurd
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D15219

Remove the unused fuwintr() and suiwintr() functions.

Half of implementations always failed (returned (-1)) and they were
previously used in only one place.

Reviewed by: kib, andrew
Obtained from: CheriBSD
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D15102

[man] Add documentation for OpenFirmware API

Add man pages for following OpenFirmware functions:

OF_child, OF_parent, OF_peer, OF_device_from_xref,
OF_device_register_xref, OF_xref_from_device, OF_getprop,
OF_getencprop, OF_getencprop_alloc, OF_getprop_alloc, OF_getproplen,
OF_hasprop, OF_nextprop, OF_prop_free, OF_searchencprop,
OF_searchprop, OF_setprop

Edited by: wblock
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D14511

Document ofw_bus_subr helpers "compatible" and "status" properties

Add documentation for following functions:

- ofw_bus_is_compatible
- ofw_bus_is_compatible_strict
- ofw_bus_node_is_compatible
- ofw_bus_search_compatible
- ofw_bus_get_status
- ofw_bus_status_okay
- ofw_bus_node_status_okay

MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D14724

Implement several enhancements to NUMA policies.

Add a new "interleave" allocation policy which stripes pages across
domains with a stride or width keeping contiguity within a multi-page
region.

Move the kernel to the dedicated numbered cpuset #2 making it possible
to assign kernel threads and memory policy separately from user. This
also eliminates the need for the complicated interrupt binding code.

Add a sysctl API for viewing and manipulating domainsets. Refactor some
of the cpuset_t manipulation code using the generic bitset type so that
it can be used for both. This probably belongs in a dedicated subr file.

Attempt to improve the include situation.

Reviewed by: kib
Discussed with: jhb (cpuset parts)
Tested by: pho (before review feedback)
Sponsored by: Netflix, Dell/EMC Isilon
Differential Revision: https://reviews.freebsd.org/D14839

Document new NUMA related syscalls and utility options.

Sponsored by: Netflix, Dell/EMC Isilon

Document pause_sig(9) and update prototypes for existing pause(9) and
pause_sbt(9) functions.

MFC after: 1 week
Suggested by: cem@
Sponsored by: Mellanox Technologies

[fdt_pinctrl] Add man pages for fdt_pinctrl driver

- Add fdt_pinctrl(4) with general information for the driver
- Add fdt_pinctrl(9) with fdt_pinctrl KPI description

Reviewed by: ian, manu, wblock
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D14235

Create style.lua(9)

This covers the lua style guidelines we've generally agreed on so far. It
will be revised as work continues and we run into more scenarios that need
specified.

Discussed with: cem, jilles
Differential Revision: https://reviews.freebsd.org/D14423

Add DECLARE_MODULE_TIED mlink missed in r329244

Reported by: imp

Take a first stab at documenting use of MODULE_PNP_INFO

Sponsored by: Dell EMC Isilon

Document syscall_helper_register(9), syscall_helper_unregister(9)

Add mallocarray.9 to malloc.9's MLINKS.

X-MFC with: r327674

bhnd(4) update inline documentation comments and man pages.

This includes a number of copyedits for the inline code documentation
comments, updates to the existing bhnd(4), bhndb(4), bcma(4), and siba(4)
man pages, and new man pages for bhnd_chipc(4), bhnd_pmu(4), bhndb_pci(4),
bhnd(9), and bhnd_erom(9).

Approved by: adrian (mentor)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D13021

Clean up the SYSINIT_FLAGS definitions for rwlock(9) and rmlock(9).

Avoid duplication in their macro definitions, and document them. No
functional change intended.

MFC after: 1 week

Add missing MLINKS for disk_add_alias(9).

MFC after: 2 weeks

Remove mbpool(9) now that it has no consumers.

mbpool existed to support NICs with memory interfaces and all remaining
comsumers were removed earlier this year with NATM.

Reviewed by: jhb
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D10513

Add clearing function for unr(9).

Previously before you could call unrhdr_delete you needed to
individually free every allocated unit. It is useful to be able to tear
down the unr without having to go through this process, as it is
significantly faster than freeing the individual units.

Reviewed by: cem, lidl
Approved by: rstone (mentor)
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12591

Remove manpage entries about crshared(9)

The function itself was removed years ago in r272546

Submitted by: Paulm <paulm tetrardus.net>
MFC after: 2 weeks

Add missing links to the nv man page.

MFC after: 1 week

Add config_intrhook_oneshot(): schedule an intrhook function and unregister
it automatically after it runs.

The config_intrhook mechanism allows a driver to stall the boot process
until device(s) required for booting are available, by not allowing system
inits to proceed until all intrhook functions have been unregistered.
Virtually all existing code simply unregisters from within the hook function
when it gets called.

This new function makes that common usage more convenient. Instead of
allocating and filling in a struct, passing it to a function that might (in
theory) fail, and checking the return code, now a driver can simply call
this cannot-fail routine, passing just the intrhook function and its arg.

Differential Revision: https://reviews.freebsd.org/D11963

Add the infrastructure to support loading multiple versions of TCP
stack modules.

It adds support for mangling symbols exported by a module by prepending
a string to them. (This avoids overlapping symbols in the kernel linker.)

It allows the use of a macro as the module name in the DECLARE_MACRO()
and MACRO_VERSION() macros.

It allows the code to register stack aliases (e.g. both a generic name
["default"] and version-specific name ["default_10_3p1"]).

With these changes, it is trivial to compile TCP stack modules with
the name defined in the Makefile and to load multiple versions of the
same stack simultaneously. This functionality can be used to enable
side-by-side testing of an old and new version of the same TCP stack.
It also could support upgrading the TCP stack without a reboot.

Reviewed by: gnn, sjg (makefiles only)
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D11086

getenv(9): rename to "kern_getenv", etc.

Update the documentation to catch up with r273174, which renamed
getenv -> kern_getenv
setenv -> kern_setenv
unsetenv -> kern_unsetenv

Leave the old links in place to support finger memory.

MFC after: 3 days
Sponsored by: Dell EMC

Add missing MLINKS for functions decribed in iflibdd(9), iflibdi(9), and
iflibtxrx(9).

Add sglist_append_sglist().

This function permits a range of one scatter/gather list to be appended to
another sglist. This can be used to construct a scatter/gather list that
reorders or duplicates ranges from one or more existing scatter/gather
lists.

Sponsored by: Chelsio Communications

More ATM and NATM removal

Submitted by: ak
Reviewed by: ngie
Differential Revision: https://reviews.freebsd.org/D10511

UMA_ZONE_REFCNT was removed.

PR: 209715
Submitted by: Fabian Keil <fk fabiankeil.de>
MFC after: 3 days

sbuf(9): add MLINKS for sbuf_{clear,get,set}_flags(9)

These functions were added in r279992.

MFC after: 2 months
Sponsored by: Dell EMC Isilon

Document EVENTHANDLER_DEFINE(9).

MFC after: 2 weeks
Sponsored by: DARPA, AFRL

Add a man page for the kernel's dynamic per-CPU memory allocator.

MFC after: 3 days

Document uiomove_frombuf(9).

add missing MLINKS for functions that this man page documents.

This page should be renamed to eliminate an unneeded inode.

Forgotten by: markm

Correct MLINKS for sbuf_hexdump(9)

sbuf_hexdump(9) should be linked to sbuf(9), not hexdump(3). Another
review will be posted to deduplicate the sbuf_hexdump reference in
in hexdump(3) or at the very least make the information less duplicative.

MFC after: 1 week
X-MFC with: r313437
Sponsored by: Dell EMC Isilon

Add iflib man pages for developers.

Doc review is probably waranted here for editing.

Submitted by: Nicole Graziano

Add support for attaching aggregation labels to sysctl objects.

I'm currently working on writing a metrics exporter for the Prometheus
monitoring system to provide access to sysctl metrics. Prometheus and
sysctl have some structural differences:

- sysctl is a tree of string component names.
- Prometheus uses a flat namespace for its metrics, but allows you to
attach labels with values to them, so that you can do aggregation.

An initial version of my exporter simply translated

hw.acpi.thermal.tz1.temperature

to

sysctl_hw_acpi_thermal_tz1_temperature_celcius

while we should ideally have

sysctl_hw_acpi_thermal_temperature_celcius{thermal_zone="tz1"}

allowing you to graph all thermal zones on a system in one go.

The change presented in this commit adds support for accomplishing this,
by providing the ability to attach labels to nodes. In the example I
gave above, the label "thermal_zone" would be attached to "tz1". As this
is a feature that will only be used very rarely, I decided to not change
the KPI too aggressively.

Discussed on: hackers@
Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D8775

The function vm_page_cache() no longer exists. Remove its man page.

mbuf(9), mbuf_tags(9): fix function prototypes.

- Add m_getclr(9) symlink to ObsoleteFiles.inc (removed in r295481).
- Add const qualifiers in m_dup(), m_dup_pkthdr() and m_tag_copy_chain()
(r286450).
- Fix m_dup_pkthdr() definition (it's not the same as m_move_pkthdr()).

MFC after: 5 days

Add link for vrefl(9).

Sponsored by: Dell EMC Isilon
MFC after: 1 week

Add man page for dnvlist.

Submitted by: Adam Starak <starak.adam@gmail.com>
Reviewed by: cem, wblock

Reset PCI pass through devices via PCI-e FLR during VM start and end.

Add routines to trigger a function level reset (FLR) of a PCI-express
device via the PCI-express device control register. This also includes
support routines to wait for pending transactions to complete as well
as calculating the maximum completion timeout permitted by a device.

Change the ppt(4) driver to reset pass through devices before attaching
to a VM during startup and before detaching from a VM during shutdown.

Reviewed by: imp, wblock (earlier version)
MFC after: 1 month
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D7751

Remove support for idle page zeroing.

Idle page zeroing has been disabled by default on all architectures since
r170816 and has some bugs that make it seemingly unusable. Specifically,
the idle-priority pagezero thread exacerbates contention for the free page
lock, and yields the CPU without releasing it in non-preemptive kernels. The
pagezero thread also does not behave correctly when superpage reservations
are enabled: its target is a function of v_free_count, which includes
reserved-but-free pages, but it is only able to zero pages belonging to the
physical memory allocator.

Reviewed by: alc, imp, kib
Differential Revision: https://reviews.freebsd.org/D7714

Introduce cnv man page.

Submitted by: Adam Starak <starak.adam@gmail.com>
Reviewed by: cem@, wblock@
Differential Revision: https://reviews.freebsd.org/D7249

Remove lockmgr_waiters(9) and BUF_LOCKWAITERS(9); they were not used
for anything.

Reviewed by: kib@
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D7420

Permit the name of the /dev/iov entry to be set by the driver.

The PCI_IOV option creates character devices in /dev/iov for each PF
device driver that registers support for creating VFs. By default the
character device is named after the PF device (e.g. /dev/iov/foo0).
This change adds a variant of pci_iov_attach() called pci_iov_attach_name()
that allows the name of the /dev/iov entry to be specified by the
driver.

Reviewed by: rstone
MFC after: 1 month
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D7400

Extract the calculation of the callout fire time into the new function
callout_when(9). See the man page update for the description of the
intended use.

Tested by: pho
Reviewed by: jhb, bjk (man page updates)
Sponsored by: The FreeBSD Foundation
MFC after: 1 month
X-Differential revision: https://reviews.freebsd.org/D7137

Implement mtx_trylock_spin(9).

Discussed with: bde
Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D7192

Document support for alternate TCP stacks.

Differential Revision: https://reviews.freebsd.org/D6940
Reviewed by: hiren
Approved by: re (gjb)
Sponsored by: Juniper Networks

Add sglist functions for working with arrays of VM pages.

sglist_count_vmpages() determines the number of segments required for
a buffer described by an array of VM pages. sglist_append_vmpages()
adds the segments described by such a buffer to an sglist. The latter
function is largely pulled from sglist_append_bio(), and
sglist_append_bio() now uses sglist_append_vmpages().

Reviewed by: kib
Sponsored by: Chelsio Communications

Add new bus methods for mapping resources.

Add a pair of bus methods that can be used to "map" resources for direct
CPU access using bus_space(9). bus_map_resource() creates a mapping and
bus_unmap_resource() releases a previously created mapping. Mappings are
described by 'struct resource_map' object. Pointers to these objects can
be passed as the first argument to the bus_space wrapper API used for bus
resources.

Drivers that wish to map all of a resource using default settings
(for example, using uncacheable memory attributes) do not need to change.
However, drivers that wish to use non-default settings can now do so
without jumping through hoops.

First, an RF_UNMAPPED flag is added to request that a resource is not
implicitly mapped with the default settings when it is activated. This
permits other activation steps (such as enabling I/O or memory decoding
in a device's PCI command register) to be taken without creating a
mapping. Right now the AGP drivers don't set RF_ACTIVE to avoid using
up a large amount of KVA to map the AGP aperture on 32-bit platforms.
Once RF_UNMAPPED is supported on all platforms that support AGP this
can be changed to using RF_UNMAPPED with RF_ACTIVE instead.

Second, bus_map_resource accepts an optional structure that defines
additional settings for a given mapping.

For example, a driver can now request to map only a subset of a resource
instead of the entire range. The AGP driver could also use this to only
map the first page of the aperture (IIRC, it calls pmap_mapdev() directly
to map the first page currently). I will also eventually change the
PCI-PCI bridge driver to request mappings of the subset of the I/O window
resource on its parent side to create mappings for child devices rather
than passing child resources directly up to nexus to be mapped. This
also permits bridges that do address translation to request suitable
mappings from a resource on the "upper" side of the bus when mapping
resources on the "lower" side of the bus.

Another attribute that can be specified is an alternate memory attribute
for memory-mapped resources. This can be used to request a
Write-Combining mapping of a PCI BAR in an MI fashion. (Currently the
drivers that do this call pmap_change_attr() directly for x86 only.)

Note that this commit only adds the MI framework. Each platform needs
to add support for handling RF_UNMAPPED and thew new
bus_map/unmap_resource methods. Generally speaking, any drivers that
are calling rman_set_bustag() and rman_set_bushandle() need to be
updated.

Discussed on: arch
Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D5237

Remove dangling references to rman_await_resource().

This function was removed when RF_TIMESHARE was removed a couple of years
ago.

MFC after: 3 days

Per Ravi Pokala's suggestion, rewrite the g_reset_bio description to
be clearer. It also describes it with more nuance. Add missing MLINKS
noticed by trasz@. Bump the date.

Re-commit r299467 having fixed the build:

Add a new get_id interface to pci and pcib. This will allow us to both
detect failures, and get different PCI IDs.

For the former the interface returns an int to signal an error. The ID is
returned at a uintptr_t * argument.

For the latter there is a type argument that allows selecting the ID type.
This only specifies a single type, however a MSI type will be added
to handle the need to find the ID the hardware passes to the ARM GICv3
interrupt controller.

A follow up commit will be made to remove pci_get_rid.

Reviewed by: jhb, rstone (previous version)
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D6239

Revert r299467 to fix the kernel build.

$ svn merge -c -299467 .

Approved by: build being broken for six hours

Add a new get_id interface to pci and pcib. This will allow us to both
detect failures, and get different PCI IDs.

For the former the interface returns an int to signal an error. The ID is
returned at a uintptr_t * argument.

For the latter there is a type argument that allows selecting the ID type.
This only specifies a single type, however a MSI type will be added
to handle the need to find the ID the hardware passes to the ARM GICv3
interrupt controller.

A follow up commit will be made to remove pci_get_rid.

Reviewed by: jhb, rstone
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D6239

Add a new bus method to fetch device-specific CPU sets.

bus_get_cpus() returns a specified set of CPUs for a device. It accepts
an enum for the second parameter that indicates the type of cpuset to
request. Currently two valus are supported:

- LOCAL_CPUS (on x86 this returns all the CPUs in the package closest to
the device when DEVICE_NUMA is enabled)
- INTR_CPUS (like LOCAL_CPUS but only returns 1 SMT thread for each core)

For systems that do not support NUMA (or if it is not enabled in the kernel
config), LOCAL_CPUS fails with EINVAL. INTR_CPUS is mapped to 'all_cpus'
by default. The idea is that INTR_CPUS should always return a valid set.

Device drivers which want to use per-CPU interrupts should start using
INTR_CPUS instead of simply assigning interrupts to all available CPUs.
In the future we may wish to add tunables to control the policy of
INTR_CPUS (e.g. should it be local-only or global, should it ignore
SMT threads or not).

The x86 nexus driver exposes the internal set of interrupt CPUs from the
the x86 interrupt code via INTR_CPUS.

The ACPI bus driver and PCI bridge drivers use _PXM to return a suitable
LOCAL_CPUS set when _PXM exists and DEVICE_NUMA is enabled. They also and
the global INTR_CPUS set from the nexus driver with the per-domain set from
_PXM to generate a local INTR_CPUS set for child devices.

Compared to the r298933, this version uses 'struct _cpuset' in
<sys/bus.h> instead of 'cpuset_t' to avoid requiring <sys/param.h>
(<sys/_cpuset.h> still requires <sys/param.h> for MAXCPU even though
<sys/_bitset.h> does not after recent changes).

Revert bus_get_cpus() for now.

I really thought I had run this through the tinderbox before committing,
but many places need <sys/types.h> -> <sys/param.h> for <sys/bus.h> now.

Add a new bus method to fetch device-specific CPU sets.

bus_get_cpus() returns a specified set of CPUs for a device. It accepts
an enum for the second parameter that indicates the type of cpuset to
request. Currently two valus are supported:

- LOCAL_CPUS (on x86 this returns all the CPUs in the package closest to
the device when DEVICE_NUMA is enabled)
- INTR_CPUS (like LOCAL_CPUS but only returns 1 SMT thread for each core)

For systems that do not support NUMA (or if it is not enabled in the kernel
config), LOCAL_CPUS fails with EINVAL. INTR_CPUS is mapped to 'all_cpus'
by default. The idea is that INTR_CPUS should always return a valid set.

Device drivers which want to use per-CPU interrupts should start using
INTR_CPUS instead of simply assigning interrupts to all available CPUs.
In the future we may wish to add tunables to control the policy of
INTR_CPUS (e.g. should it be local-only or global, should it ignore
SMT threads or not).

The x86 nexus driver exposes the internal set of interrupt CPUs from the
the x86 interrupt code via INTR_CPUS.

The ACPI bus driver and PCI bridge drivers use _PXM to return a suitable
LOCAL_CPUS set when _PXM exists and DEVICE_NUMA is enabled. They also and
the global INTR_CPUS set from the nexus driver with the per-domain set from
_PXM to generate a local INTR_CPUS set for child devices.

Reviewed by: wblock (manpage)
Differential Revision: https://reviews.freebsd.org/D5519

Add a new rescan method to the bus interface.

The BUS_RESCAN() method rescans a single bus device checking for devices
that have been added or removed from the bus. A new 'rescan' command is
added to devctl(8) to trigger a rescan.

Differential Revision: https://reviews.freebsd.org/D6016

Improve the implementation and documentation of the
SYSCTL_COUNTER_U64_ARRAY() macro.

- Add proper asserts to the SYSCTL_COUNTER_U64_ARRAY() macro that checks
the size of the first element of the array.
- Add an example to the counter(9) manual page how to use the
SYSCTL_COUNTER_U64_ARRAY() macro.
- Add some missing symbolic links for counter(9) while at it.

Document m_catpkt(), and remove misinformation about m_cat(9).

Since m_cat() may copy data from the second mbuf chain into the last mbuf
of the first chain, it may free the first mbuf of the second chain. Thus,
the second chain is not guaranteed to be valid after m_cat() returns.

Reviewed by: glebius
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D5497

Remove taskqueue_enqueue_fast().

taskqueue_enqueue() was changed to support both fast and non-fast
taskqueues 10 years ago in r154167. It has been a compat shim ever
since. It's time for the compat shim to go.

Submitted by: Howard Su <howard0su@gmail.com>
Reviewed by: sephe
Differential Revision: https://reviews.freebsd.org/D5131

Separate manual pages into their own package.

Sponsored by: The FreeBSD Foundation

Provide yet another KPI for cdev creation, make_dev_s(9).

Immediate problem fixed by the new KPI is the long-standing race
between device creation and assignments to cdev->si_drv1 and
cdev->si_drv2, which allows the window where cdevsw methods might be
called with si_drv1,2 fields not yet set. Devices typically checked
for NULL and returned spurious errors to usermode, and often left some
methods unchecked.

The new function interface is designed to be extensible, which should
allow to add more features to make_dev_s(9) without inventing yet
another name for function to create devices, while maintaining KPI and
even KBI backward-compatibility.

Reviewed by: hps, jhb
Sponsored by: The FreeBSD Foundation
MFC after: 3 weeks
Differential revision: https://reviews.freebsd.org/D4746

Add accessor methods to fetch the BAR holding the MSI-X table and PBA.

While here, explicitly note the requirement that the BAR(s) must be
allocated prior to calling pci_alloc_msix().

Reviewed by: andrew, emaste
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D4688

Add a safety net to reclaim mbufs when one of the mbuf zones become
exhausted.

It is possible for a bug in the code (or, theoretically, even unusual
network conditions) to exhaust all possible mbufs or mbuf clusters.
When this occurs, things can grind to a halt fairly quickly. However,
we currently do not call mb_reclaim() unless the entire system is
experiencing a low-memory condition.

While it is best to try to prevent exhaustion of one of the mbuf zones,
it would also be useful to have a mechanism to attempt to recover from
these situations by freeing "expendable" mbufs.

This patch makes two changes:

a) The patch adds a generic API to the UMA zone allocator to set a
function that should be called when an allocation fails because the
zone limit has been reached. Because of the way this function can be
called, it really should do minimal work.

b) The patch uses this API to try to free mbufs when an allocation
fails from one of the mbuf zones because the zone limit has been
reached. The function schedules a callout to run mb_reclaim().

Differential Revision: https://reviews.freebsd.org/D3864
Reviewed by: gnn
Comments by: rrs, glebius
MFC after: 2 weeks
Sponsored by: Juniper Networks

Add helper functions proc_readmem() and proc_writemem().

These helper functions can be used to read in or write a buffer from or to
an arbitrary process' address space. Without them, this can only be done
using proc_rwmem(), which requires the caller to fill out a uio. This is
onerous and results in code duplication; the new functions provide a simpler
interface which is sufficient for most existing callers of proc_rwmem().

This change also adds a manual page for proc_rwmem() and the new functions.

Reviewed by: jhb, kib
Differential Revision: https://reviews.freebsd.org/D4245

Add an MLINK for m_collapse.

PR: 204205
Submitted by: avos
MFC after: 1 week

Some basic documentation (a man page) on kern_testfrwk

Add the MLINK for async_drain Thanks Edward for the pointer.

MFC after: 1 month

Add a manual page for PHOLD() and friends.

MFC after: 1 week

Round out SYSCTL macros to the full set of fixed-width types

Add S8, S16, S32, and U32 types; add SYSCTL*() macros for them, as well
as for the existing 64-bit types. (While SYSCTL*QUAD and UQUAD macros
already exist, they do not take the same sort of 'val' parameter that
the other macros do.)

Clean up the documented "types" in the sysctl.9 document. (These are
macros and thus not real types, but the manual page documents intent.)

The sysctl_add_oid(9) arg2 has been bumped from intptr_t to intmax_t to
accommodate 64-bit types on 32-bit pointer architectures.

This is just the kernel support piece; the userspace sysctl(1) support
will follow in a later patch.

Submitted by: Ravi Pokala <rpokala@panasas.com>
Reviewed by: cem
Relnotes: no
Sponsored by: Panasas
Differential Revision: https://reviews.freebsd.org/D4091

remove \, it confuses things.

Add a new helper function for PCI devices to locate the upstream
PCI-express root port of a given PCI device.

Reviewed by: kib, imp
MFC after: 1 week
Sponsored by: Chelsio
Differential Revision: https://reviews.freebsd.org/D4089

Add helper routines for PCI device drivers to read, write, and modify
PCI-Express capability registers (that is, PCI config registers in the
standard PCI config space belonging to the PCI-Express capability
register set).

Note that all of the current PCI-e registers are either 16 or 32-bits,
so only widths of 2 or 4 bytes are supported.

Reviewed by: imp
MFC after: 1 week
Sponsored by: Chelsio
Differential Revision: https://reviews.freebsd.org/D4088

sysctl(9): Document U8/U16 types from r289773

Suggested by: ngie
Sponsored by: EMC / Isilon Storage Division

Document cpuset(9)

A follow-up to r289467.

Coerced by: jhb
Sponsored by: EMC / Isilon Storage Division

Document bitset(9)

Remove MLINKS to more non-existent mbuf(9) macros

X-MFC with: r288295
MFC after: 3 days
Sponsored by: EMC / Isilon Storage Division

Posthumously remove all references to MFREE(9)

The macro was removed in r90227

MFC after: 3 days
Sponsored by: EMC / Isilon Storage Division

Document bus_get_resource(9).

Suggested by: Francois Tigeot
Obtained from: DragonFlyBSD 09301a2b29f3ae5edd39a858f909f8770372f71e
Sponsored by: EMC / Isilon Storage Division

Revert r287780 until more developers have their say.

Differential Revision: https://reviews.freebsd.org/D3521
Requested by: gnn

Implement callout_drain_async(), inspired by the projects/hps_head
branch.

This function is used to drain a callout via a callback instead of
blocking the caller until the drain is complete. Refer to the
callout_drain_async() manual page for a detailed description.

Limitation: If a lock is used with the callout, the callout can only
be drained asynchronously one time unless the callout_init_mtx()
function is called again. This limitation is not present in
projects/hps_head and will require more invasive changes to the
timeout code, which was not in the scope of this patch.

Differential Revision: https://reviews.freebsd.org/D3521
Reviewed by: wblock
MFC after: 1 month

New 1-Wire bus implementation. 1-Wire controller is abstracted, though
only gpiobus configured via FDT is supported. Bus enumeration is
supported. Devices are created for each device found. 1-Wire
temperature controllers are supported, but other drivers could be
written. Temperatures are polled and reported via a sysctl. Errors
are reported via sysctl counters. Mis-wired bus detection is included
for more trouble shooting. See ow(4), owc(4) and ow_temp(4) for
details of what's supported and known issues.

This has been tested on Raspberry Pi-B, Pi2 and Beagle Bone Black
with up to 7 devices.

Differential Revision: https://reviews.freebsd.org/D2956
Relnotes: yes
MFC after: 2 weeks
Reviewed by: loos@ (with many insightful comments)

Add link for rw_unlock(9) to rwlock(9).

MFC after: 3 days
Sponsored by: EMC / Isilon Storage Division

Create man page for pmap_quick_enter_page(9) and pmap_quick_remove_page(9)

Reviewed by: kib, brueffer, wblock
Approved by: kib (mentor)
Differential Revision: https://reviews.freebsd.org/D3312

Revert r286236; vgonel() is a static function.

Sponsored by: The FreeBSD Foundation

Document vgonel(9).

MFC after: 1 month
Sponsored by: The FreeBSD Foundation

Implement the CloudABI random_get() system call.

The random_get() system call works similar to getentropy()/getrandom()
on OpenBSD/Linux. It fills a buffer with random data.

This change introduces a new function, read_random_uio(), that is used
to implement read() on the random devices. We can call into this
function from within the CloudABI compatibility layer.

Approved by: secteam
Reviewed by: jmg, markm, wblock
Obtained from: https://github.com/NuxiNL/freebsd
Differential Revision: https://reviews.freebsd.org/D3053

Move nvlist documentation to the FreeBSD Kernel Developer's sections.

Approved by: pjd (mentor)

Create a separate kobj interface for leaf-driver PCI IOV methods.

Leaf drivers should not import the PCI bus interface to add IOV handling.
Instead, move the IOV client methods to a separate kobj interface.

Differential Revision: https://reviews.freebsd.org/D2584
Reviewed by: rstone

Remove references to Giant in namei(9). This was removed in r241896.

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Make vpanic() externally visible so that it can be called as part of the
DTrace panic() action.

Differential Revision: https://reviews.freebsd.org/D2349
Reviewed by: avg
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Document the interface for defining a configuration schema

Differential Revision: https://reviews.freebsd.org/D89
Reviewed by: wblock, emaste, allanjude
MFC after: 1 month
Sponsored by: Sandvine Inc.

Add manpages for SR-IOV enable/disable driver interface

Differential Revision: https://reviews.freebsd.org/D75
Reviewed by: wblock
MFC after: 1 month
Sponsored by: Sandvine Inc.

Document pci_iov_attach/detach in pci.9

Differential Revision: https://reviews.freebsd.org/D74
Reviewed by: bcr, wblock, emaste
MFC after: 1 month
Sponsored by: Sandvine Inc.

spl man page hasn't been relevant for a while, retire it.

Revert for r277213:

FreeBSD developers need more time to review patches in the surrounding
areas like the TCP stack which are using MPSAFE callouts to restore
distribution of callouts on multiple CPUs.

Bump the __FreeBSD_version instead of reverting it.

Suggested by: kmacy, adrian, glebius and kib
Differential Revision: https://reviews.freebsd.org/D1438

Major callout subsystem cleanup and rewrite:
- Close a migration race where callout_reset() failed to set the
CALLOUT_ACTIVE flag.
- Callout callback functions are now allowed to be protected by
spinlocks.
- Switching the callout CPU number cannot always be done on a
per-callout basis. See the updated timeout(9) manual page for more
information.
- The timeout(9) manual page has been updated to reflect how all the
functions inside the callout API are working. The manual page has
been made function oriented to make it easier to deduce how each of
the functions making up the callout API are working without having
to first read the whole manual page. Group all functions into a
handful of sections which should give a quick top-level overview
when the different functions should be used.
- The CALLOUT_SHAREDLOCK flag and its functionality has been removed
to reduce the complexity in the callout code and to avoid problems
about atomically stopping callouts via callout_stop(). If someone
needs it, it can be re-added. From my quick grep there are no
CALLOUT_SHAREDLOCK clients in the kernel.
- A new callout API function named "callout_drain_async()" has been
added. See the updated timeout(9) manual page for a complete
description.
- Update the callout clients in the "kern/" folder to use the callout
API properly, like cv_timedwait(). Previously there was some custom
sleepqueue code in the callout subsystem, which has been removed,
because we now allow callouts to be protected by spinlocks. This
allows us to tear down the callout like done with regular mutexes,
and a "td_slpmutex" has been added to "struct thread" to atomically
teardown the "td_slpcallout". Further the "TDF_TIMOFAIL" and
"SWT_SLEEPQTIMO" states can now be completely removed. Currently
they are marked as available and will be cleaned up in a follow up
commit.
- Bump the __FreeBSD_version to indicate kernel modules need
recompilation.
- There has been several reports that this patch "seems to squash a
serious bug leading to a callout timeout and panic".

Kernel build testing: all architectures were built
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D1438
Sponsored by: Mellanox Technologies
Reviewed by: jhb, adrian, sbruno and emaste

Replace CC_VAR with CCV, since the CC_VAR macro doesn't exist.
While here, add MLINK for CCV.9 and DECLARE_CC_MODULE.9.

Clean up more usb related files when MK_USB == no when dealing with
manpages, libraries, and binaries

callout(9): add sbt flavors of callout_schedule

Differential Revision: https://reviews.freebsd.org/D1161 (partial)
Reviewed by: jhb, markj
MFC after: 1 week

Create a vimage.9 link to vnet.9

Requested by: glebius

MFp4: @179066

Add page which describes VNET network stack virtualization infrastructure.

Submitted by: bz
Sponsored by: The FreeBSD Foundation
Sponsored by: CK Software GmbH

Actually install casuword(9) to fix build.

Add fueword(9) and casueword(9) functions. They are like fuword(9)
and casuword(9), but do not mix value read and indication of fault.

I know (or remember) enough assembly to handle x86 and powerpc. For
arm, mips and sparc64, implement fueword() and casueword() as wrappers
around fuword() and casuword(), which means that the functions cannot
distinguish between -1 and fault.

On architectures where fueword() and casueword() are native, implement
fuword() and casuword() using fueword() and casuword(), to reduce
assembly code duplication.

Sponsored by: The FreeBSD Foundation
Tested by: pho
MFC after: 2 weeks (ia64 needs treating)

Fix multiple incorrect SYSCTL arguments in the kernel:

- Wrong integer type was specified.

- Wrong or missing "access" specifier. The "access" specifier
sometimes included the SYSCTL type, which it should not, except for
procedural SYSCTL nodes.

- Logical OR where binary OR was expected.

- Properly assert the "access" argument passed to all SYSCTL macros,
using the CTASSERT macro. This applies to both static- and dynamically
created SYSCTLs.

- Properly assert the the data type for both static and dynamic
SYSCTLs. In the case of static SYSCTLs we only assert that the data
pointed to by the SYSCTL data pointer has the correct size, hence
there is no easy way to assert types in the C language outside a
C-function.

- Rewrote some code which doesn't pass a constant "access" specifier
when creating dynamic SYSCTL nodes, which is now a requirement.

- Updated "EXAMPLES" section in SYSCTL manual page.

MFC after: 3 days
Sponsored by: Mellanox Technologies

Reflect the chanages in sleepqueue.h and subr_sleepqueue.c

- Priority argument is introduced to sleepq_*wait* in r177085
- sleepq_calc_signal_retval is removed from implementation
- sleepq_catch_signals is internal now

Differential Revision: https://reviews.freebsd.org/D794
Reviewed by: jhb
Approved by: jhb

Add missing mlinks for section 9.

MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation

Add missing link to TIMEOUT_TASK_INIT(9).

MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation

Add missing links to taskqueue(9).

MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation

Add description of "sysctl_remove_name()" function.

Retire pmap_change_wiring(). We have never used it to wire virtual pages.
We continue to use pmap_enter() for that. For unwiring virtual pages, we
now use pmap_unwire(), which unwires a range of virtual addresses instead
of a single virtual page.

Sponsored by: EMC / Isilon Storage Division

- Updated SYSCTL manual pages after recent changes to the kernel
SYSCTL code. Added description of new macros and functions.
- Merged dynamic and static SYSCTL related content into a single
manual page, hence parameters and functionality is very much the same.
- Uppercased all occurrences of "OID".
- Updated all SYSCTL examples.

PR: 192101

Remove man page for non-existent VOPs.

Sponsored by: The FreeBSD Foundation
MFC after: 3 days

s/PCBGROUPS/PCBGROUP/ manpage to match "options PCBGROUP".

Add the PCBGROUPS manpage.

Thanks to wblock for helping me with this manpage.

Remove obsolete man pages. The corresponding code was removed in r255608.

Remove an obsolete man page.

Add a man page for the new function pmap_unwire().

Remove the man page for a pmap function that no longer exists.

Correct the name of the man page for pmap_ts_referenced().

Sponsored by: EMC / Isilon Storage Division

Add fpu_kern.9 man page to the install.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Add m_pulldown and m_unshare to MLINKS.

Document some more socket features

Add some mention of the functions used by protocol implementations,
upcalls, and other general routines.

Not all functionality is documented; in particular:
o the *at() variants, which are useful only for implementing the
corresponding syscalls.
o soconnect2(), also only used to implement a syscall (socketpair()).
o sockargs(), which is essentually unused and only tangentially
socket-related.
o selsocket(), which is commented as being present solely for use by
netncp and netsmb.
o getsockaddr(), which is just a convenience shortcut for copyin().

Reviewed by: jhb (previous version)
Approved by: hrs (mentor)

Documented the pget(9) system kernel interface.

Reviewed by: trociny, kib
MFC after: 1 week

Document m_get2().

Reviewed by: glebius

update taskqueue(9) manual page

Many thanks to bjk, gjb and pluknet for improvements and suggestions.

MFC after: 5 days
Sponsored by: HybridCluster

add a few missing links...

Submitted by: J David
MFC after: 1 week

link mbuf to m_append so it can be found...

Pointed out by: J David
MFC after: 1 week

Add sglist_append_bio(9) to append a struct bio's data to a sglist

Reviewed by: jhb, kib (long ago)

document the m_getjcl function...

Add missing symlinks for the sbuf man page

Add a manpage for the getenv() family of functions in the kernel.

Add manpages for VOP_ADVISE() and VOP_ALLOCATE().

Approved by: re (gjb)

- Document the UQUAD sysctl variants.
- Clarify that exactly one of the "access" flags is required and
list the optional flags in a separate list. Prefer bundling
CTLFLAG_TUN into the access flag by not documenting it as an
optional flag to set.

Approved by: re (glebius)
MFC after: 1 week

add links for the various vmem functions...

Hook vm_page_busy.9 to the build

The soft and hard busy mechanism rely on the vm object lock to work.
Unify the 2 concept into a real, minimal, sxlock where the shared
acquisition represent the soft busy and the exclusive acquisition
represent the hard busy.
The old VPO_WANTED mechanism becames the hard-path for this new lock
and it becomes per-page rather than per-object.
The vm_object lock becames an interlock for this functionality:
it can be held in both read or write mode.
However, if the vm_object lock is held in read mode while acquiring
or releasing the busy state, the thread owner cannot make any
assumption on the busy state unless it is also busying it.

Also:
- Add a new flag to directly shared busy pages while vm_page_alloc
and vm_page_grab are being executed. This will be very helpful
once these functions happen under a read object lock.
- Move the swapping sleep into its own per-object flag

The KPI is heavilly changed this is why the version is bumped.
It is very likely that some VM ports users will need to change
their own code.

Sponsored by: EMC / Isilon storage division
Discussed with: alc
Reviewed by: jeff, kib
Tested by: gavin, bapt (older version)
Tested by: pho, scottl

Document the sbinuptime() and getsbinuptime() functions introduced in
r247452.

Sponsored by: DARPA, AFRL

Add manual page for vmem(9). Obtained from NetBSD, modified to match
our implementation.

Obtained from: NetBSD

Add a man page for the SDT_* macros, which can be used to define new static
DTrace probes and providers in kernel code.

MFC after: 1 week

Several improvements to rmlock(9). Many of these are based on patches
provided by Isilon.
- Add an rm_assert() supporting various lock assertions similar to other
locking primitives. Because rmlocks track readers the assertions are
always fully accurate unlike rw_assert() and sx_assert().
- Flesh out the lock class methods for rmlocks to support sleeping via
condvars and rm_sleep() (but only while holding write locks), rmlock
details in 'show lock' in DDB, and the lc_owner method used by
dtrace.
- Add an internal destroyed cookie so that API functions can assert
that an rmlock is not destroyed.
- Make use of rm_assert() to add various assertions to the API (e.g.
to assert locks are held when an unlock routine is called).
- Give RM_SLEEPABLE locks their own lock class and always use the
rmlock's own lock_object with WITNESS.
- Use THREAD_NO_SLEEPING() / THREAD_SLEEPING_OK() to disallow sleeping
while holding a read lock on an rmlock.

Submitted by: andre
Obtained from: EMC/Isilon

Remove vfs_mount(9), it have been died since 2004.

Noticed by: git via alfred
MFC after: 2 weeks

Merge from projects/counters: counter(9).

Introduce counter(9) API, that implements fast and raceless counters,
provided (but not limited to) for gathering of statistical data.

See http://lists.freebsd.org/pipermail/freebsd-arch/2013-April/014204.html
for more details.

In collaboration with: kib
Reviewed by: luigi
Tested by: ae, ray
Sponsored by: Nginx, Inc.

MFcalloutng:
Dcoument the new functions added to condvar(9), sleep(9), sleepqueue(9)
KPIs. Also document recent changes in timeout(9) and eventtimers(4).

Document pffinddomain().

Reviewed by: glebius

Document disk_resize(9).

Remove the support for using non-mpsafe filesystem modules.

In particular, do not lock Giant conditionally when calling into the
filesystem module, remove the VFS_LOCK_GIANT() and related
macros. Stop handling buffers belonging to non-mpsafe filesystems.

The VFS_VERSION is bumped to indicate the interface change which does
not result in the interface signatures changes.

Conducted and reviewed by: attilio
Tested by: pho

Add a manpage for BUS_CHILD_DETACHED().

Add a BUS_CHILD_DELETED() method that a bus can hook to allow it to cleanup
any bus-specific state (such as ivars) when a child device is deleted.

Requested by: kan
MFC after: 1 month

Document DRIVER_MODULE_ORDERED, EARLY_DRIVER_MODULE, and
EARLY_DRIVER_MODULE_ORDERED.

Document swi_remove(9).

Reviewed by: jhb
MFC after: 1 week

- Add knlist_init_rw_reader() function to kqueue(9).
Function acquired reader lock if needed.
Assert check for reader or writer lock (RA_LOCKED / RA_UNLOCKED)
- While here, add knlist_init_mtx.9 to MLINKS and fix some style(9) issues

Reviewed by: glebius
Approved by: ae(mentor)

MFC after: 2 weeks

Document pci_find_extcap() and pci_find_htcap().

Expand and reorganize the pci(9) manpage a bit:
- Document the following routines: pci_alloc_msi(), pci_alloc_msix(),
pci_find_cap(), pci_get_max_read_req(), pci_get_vpd_ident(),
pci_get_vpd_readonly(), pci_msi_count(), pci_msix_count(),
pci_pending_msix(), pci_release_msi(), pci_remap_msix(), and
pci_set_max_read_req().
- Group the functions into five sub-sections: raw configuration access,
locating devices, device information, device configuration, and
message signaled interrupts.
- Discourage use of pci_disable_io() and pci_enable_io() in device drivers.
The PCI bus driver handles this automatically as resources are activated.

MFC after: 2 weeks

Add pci_save_state() and pci_restore_state() wrappers around
pci_cfg_save() and pci_cfg_restore() for device drivers to use when
saving and restoring state (e.g. to handle device-specific resets).

Reviewed by: imp
MFC after: 2 weeks

Catch up with r195837 (2.5 years ago) which renamed net_add_domain() to domain_add().

PR: 165424
Submitted by: Lachlan Kang
MFC after: 1 week

Manpages for the buf_ring and drbr interfaces.

Submitted by: kmacy (aeons ago)
MFC after: 1 week

Fix sloppyness in memcchr() man page.

I was considering adding it to libc as well, but last minute I thought
it would be good enough to add it to libkern exclusively. I forgot to
rename the man page and hook it up.

Add eventtimers(9) man page, describing related KPIs.

Sponsored by: iXsystems, Inc.
MFC after: 1 month

Add a TASK_INITIALIZER() macro that can be used to statically
initialize a task structure.

Reviewed by: gj
MFC after: 2 weeks

Add a helper API to allow in-kernel code to map portions of shared memory
objects created by shm_open(2) into the kernel's address space. This
provides a convenient way for creating shared memory buffers between
userland and the kernel without requiring custom character devices.

More MLINKS for rtalloc.9

As it turns out, r186347 actually is insufficient to avoid the use of the
curthread-accessing part of mtx_{,un}lock(9) when using a r210623-style
curthread implementation on sparc64, crashing the kernel in its early
cycles as PCPU isn't set up, yet (and can't be set up as OFW is one of the
things we need for that, which leads to a chicken-and-egg problem). What
happens is that due to the fact that the idea of r210623 actually is to
allow the compiler to cache invocations of curthread, it factors out
obtaining curthread needed for both mtx_lock(9) and mtx_unlock(9) to
before the branch based on kobj_mutex_inited when compiling the kernel
without the debugging options. So change kobj_class_compile_static(9)
to just never acquire kobj_mtx, effectively restricting it to its
documented use, and add a kobj_init_static(9) for initializing objects
using a class compiled with the former and that also avoids using mutex(9)
(and malloc(9)). Also assert in both of these functions that they are
used in their intended way only.
While at it, inline kobj_register_method() and kobj_unregister_method()
as there wasn't much point for factoring them out in the first place
and so that a reader of the code has to figure out the locking for
fewer functions missing a KOBJ_ASSERT.
Tested on powerpc{,64} by andreast.

Reviewed by: nwhitehorn (earlier version), jhb
MFC after: 3 days

Mention tdsignal(9).

MFC after: 1 week

Rename the cc.4 and cc.9 modular congestion control related man pages to
mod_cc.4 and mod_cc.9 respectively to avoid any possible confusion with the cc.1
gcc man page. Update references to these man pages where required.

Requested by: Grenville Armitage
Approved by: re (kib)
MFC after: 3 days

Document vm_page_aflag_set(9), vm_page_aflag_clear(9) and vm_page_reference(9).
Retire vm_page_flag_set() and vm_page_flag_clear() functions.

Reviewed by: alc
Approved by: re (bz)

Remove information and MLINK for MULTI_DRIVER_MODULE().
The macro was removed in r121129 almost 8 years ago.

PR: 150244
Submitted by: Gireesh Nagabhushana <dngireesh@avrita.com>
Approved by: re (kib)

Fix an error in the generation of the manpage after r225177.

Sponsored by: Sandvine Incorporated
Submitted by: jkim
Approved by: re (bz)
MFC after: 3 weeks
X-MFC: r225177

Fix a deficiency in the selinfo interface:
If a selinfo object is recorded (via selrecord()) and then it is
quickly destroyed, with the waiters missing the opportunity to awake,
at the next iteration they will find the selinfo object destroyed,
causing a PF#.

That happens because the selinfo interface has no way to drain the
waiters before to destroy the registered selinfo object. Also this
race is quite rare to get in practice, because it would require a
selrecord(), a poll request by another thread and a quick destruction
of the selrecord()'ed selinfo object.

Fix this by adding the seldrain() routine which should be called
before to destroy the selinfo objects (in order to avoid such case),
and fix the present cases where it might have already been called.
Sometimes, the context is safe enough to prevent this type of race,
like it happens in device drivers which installs selinfo objects on
poll callbacks. There, the destruction of the selinfo object happens
at driver detach time, when all the filedescriptors should be already
closed, thus there cannot be a race.
For this case, mfi(4) device driver can be set as an example, as it
implements a full correct logic for preventing this from happening.

Sponsored by: Sandvine Incorporated
Reported by: rstone
Tested by: pluknet
Reviewed by: jhb, kib
Approved by: re (bz)
MFC after: 3 weeks

vm_map_clean() was long ago renamed to vm_map_sync(). Also, it no longer
acquires Giant.

MFC after: 1 week

Remove manual pages for functions that no longer exist.

MFC after: 1 week

Document copyin_nofault, copyout_nofault, uiomove_nofault.

Submitted by: alc

Install symlinks for m_tag_* mbuf tag functions to the mbuf_tags.9 man page.

MFC after: 1 week

Add make_dev_alias_p to MLINKS.

Pointed out by: pjd
MFC after: 2 weeks

Add a new bus method, BUS_ADJUST_RESOURCE() that is intended to be a
wrapper around rman_adjust_resource(). Include a generic implementation,
bus_generic_adjust_resource() which passes the request up to the parent
bus. There is currently no default implementation. A
bus_adjust_resource() wrapper is provided for use in drivers.

Extend the rman(9) API to support altering an existing resource.
Specifically, these changes allow a resource to back a relocatable and
resizable resource such as the I/O window decoders in PCI-PCI bridges.
- rman_adjust_resource() can adjust the start and end address of an
existing resource. It only succeeds if the newly requested address
space is already free. It also supports shrinking a resource in
which case the freed space will be marked unallocated in the rman.
- rman_first_free_region() and rman_last_free_region() return the
start and end addresses for the first or last unallocated region in
an rman, respectively. This can be used to determine by how much
the resource backing an rman must be adjusted to accomodate an
allocation request that does not fit into the existing rman.

While here, document the rm_start and rm_end fields in struct rman,
rman_is_region_manager(), the bound argument to
rman_reserve_resource_bound(), and rman_init_from_resource().

Rename alloc_unr(9) to unr(9) and adjust the links accordingly.

MFC after: 3 weeks

Final commit to round out the "Five New TCP Congestion Control Algorithms for
FreeBSD" FreeBSD Foundation funded project.

- Add new man pages for the modular congestion control, Khelp and Hhook
frameworks (cc.4, cc.9, khelp.9 and hhook.9).

- Add new man pages for each available congestion control algorithm (cc_chd.4,
cc_cubic.4, cc_hd.4, cc_htcp.4, cc_newreno.4 and cc_vegas.4).

- Add a new man page for the Enhanced Round Trip Time (ERTT) Khelp module
(h_ertt.4).

- Update the TCP (tcp.4) man page to mention the TCP_CONGESTION socket option,
cross reference to cc.4 and remove references to the retired
"net.inet.tcp.newreno" sysctl MIB variable.

In collaboration with: David Hayes <dahayes at swin edu au> and
Grenville Armitage <garmitage at swin edu au>
Sponsored by: FreeBSD Foundation
MFC after: 3 months

Document sbuf_new_for_sysctl(9).

Pointed out by: lstewart

sysctl(8) should use the CTLTYPE to determine the type of data when
reading. (This was already done for writing to a sysctl). This
requires all SYSCTL setups to specify a type. Most of them are now
checked at compile-time.

Remove SYSCTL_*X* sysctl additions as the print being in hex should be
controlled by the -x flag to sysctl(8).

Succested by: bde

Add a man page for the Object Specific Data (OSD) KPI.

Although not directly related the FreeBSD Foundation funded "Five New TCP
Congestion Control Algorithms for FreeBSD" project, the understanding and
inspiration required to write this documentation was significantly bolstered
by the Foundation's support.

Reviewed by: pjd
MFC after: 1 week

Add a manpage for SYSINIT() and SYSUNINIT().

PR: docs/132884
Submitted by: pluknet, hmp

Stop documenting vgonel() after its converting to the static function:
svn r147332 (by jeff): "Don't make vgonel() globally visible".
While here, specify the vnode locking scheme for vgone().

Discussed on: freebsd-hackers@
Approved by: kib (mentor)
MFC after: 10 days

Add a taskqueue_cancel(9) to cancel a pending task without waiting for
it to run as taskqueue_drain(9) does.

Requested by: hselasky
Original code: jeff
Reviewed by: jhb
MFC after: 2 weeks

Document make_dev_p(9).

Reviewed by: brueffer, kib

catch up manual pages with rename of vm_page_sleep_busy to vm_page_sleep_if_busy

Suggested by: alc
MFC after: 4 days

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Update links for taskqueue(9) functions.

Document vunref(9), add some important notes for vrele(9) and vput(9).
Merge all three manpages to one, removing separate file for vput(9).

MFC after: 1 week

Replace sbuf_overflowed() with sbuf_error(), which returns any error
code associated with overflow or with the drain function. While this
function is not expected to be used often, it produces more information
in the form of an errno that sbuf_overflowed() did.

Add drain functionality to sbufs. The drain is a function that is
called when the sbuf internal buffer is filled. For kernel sbufs with a
drain, the internal buffer will never be expanded. For userland sbufs
with a drain, the internal buffer may still be expanded by
sbuf_[v]printf(3).

Sbufs now have three basic uses:
1) static string manipulation. Overflow is marked.
2) dynamic string manipulation. Overflow triggers string growth.
3) drained string manipulation. Overflow triggers draining.

In all cases the manipulation is 'safe' in that overflow is detected and
managed.

Reviewed by: phk (the previous version)

rmlock(9) two additions and one change/fix:
- add rm_try_rlock().
- add RM_SLEEPABLE to use sx(9) as the back-end lock in order to sleep while
holding the write lock.
- change rm_noreadtoken to a cpu bitmask to indicate which CPUs need to go
through the lock/unlock in order to synchronize. As a side effect, this
also avoids IPI to CPUs without any readers during rm_wlock.

Discussed with: ups@, rwatson@ on arch@
Sponsored by: Isilon Systems, Inc.