SysV IPC: provide in-kernel helpers to obtain ipcs(8)-like information

PR: 278949
Reviewed by: markj
Tested by: Ricardo Branco <rbranco@suse.de>
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D45175

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

sysv: ansify

Reported by: clang 15
Sponsored by: Rubicon Communications, LLC ("Netgate")

sysvsem: Fix a typo

Per jamie@ rpr can be NULL if the jail is created with sysvsem=disable.
But at least it doesn't appear to be fatal, since rpr is never dereferenced
but is only compared to other prison pointers.

Reviewed by: jamie
Differential revision: https://reviews.freebsd.org/D35198
MFC after: 2 weeks

sysvsem: Style(9)

MFC after: 2 weeks

sysvsem: Trim traiing whitespace

MFC after: 2 weeks

sysvsem: Add a timeout argument to the semop.

For future use in the Linux emulation layer for the semtimedop syscall
split the sys_semop syscall into two counterparts and add
struct timespec *timeout argument to the last one.

Reviewed by: jhb, kib
Differential revision: https://reviews.freebsd.org/D35121
MFC after: 2 weeks

freebsd32: include `__` in semctl names

This mirrors sys/kern/syscall.master and will simplify generation
of freebsd32 files.

Reviewed by: kevans

freebsd32: rename old SysV IPC types

Move the 32 from ...32_old to ..._old32 to aid automatic generation.

Reviewed by: kevans

freebsd32: add a union semun_old32

Use this for COMPAT7 support. In practice it's the same as
union semun32 since the pointers become uint32_t's the it's more
symetric and is the logical thing to generate from semun_old.

Reviewed by: kevans

jail: Change the locking around pr_ref and pr_uref

Require both the prison mutex and allprison_lock when pr_ref or
pr_uref go to/from zero. Adding a non-first or removing a non-last
reference remain lock-free. This means that a shared hold on
allprison_lock is sufficient for prison_isalive() to be useful, which
removes a number of cases of lock/check/unlock on the prison mutex.

Expand the locking in kern_jail_set() to keep allprison_lock held
exclusive until the new prison is valid, thus making invalid prisons
invisible to any thread holding allprison_lock (except of course the
one creating or destroying the prison). This renders prison_isvalid()
nearly redundant, now used only in asserts.

Differential Revision: https://reviews.freebsd.org/D28419
Differential Revision: https://reviews.freebsd.org/D28458

jail: Add prison_isvalid() and prison_isalive()

prison_isvalid() checks if a prison record can be used at all, i.e.
pr_ref > 0. This filters out prisons that aren't fully created, and
those that are either in the process of being dismantled, or will be
at the next opportunity. While the check for pr_ref > 0 is simple
enough to make without a convenience function, this prepares the way
for other measures of prison validity.

prison_isalive() checks not only validity as far as the useablity of
the prison structure, but also whether the prison is visible to user
space. It replaces a test for pr_uref > 0, which is currently only
used within kern_jail.c, and not often there.

Both of these functions also assert that either the prison mutex or
allprison_lock is held, since it's generally the case that unlocked
prisons aren't guaranteed to remain useable for any length of time.
This isn't entirely true, for example a thread can assume its own
prison is good, but most exceptions will exist inside of kern_jail.c.

sysv_sem: semusz depends on semume.

Size of the per-process semaphore undo structure (semusz) depends on
the number of the per-process undos. If kern.ipc.semume is adjusted,
semusz must be adjusted as well, and it makes no sense to delegate
adjustment to user. Make it automatic.

Reported and tested by: Olef <o.vandestadt@gmail.com>
PR: 250361
Reviewed by: jhb, markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D26826

Don't invoke semunload() if seminit() fails during MOD_LOAD.

The module handler code invokes a MOD_UNLOAD event immediately if
MOD_LOAD fails. The result was that if seminit() failed, semunload()
was invoked twice. semunload() is not idempotent however and would
try to remove it's process_exit eventhandler twice resulting in a
panic.

Reviewed by: kib, markj
Obtained from: CheriBSD
MFC after: 1 month
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D26696

Apply the logic from r363051 to semctl(2) and __sem_base field.

Reported by: Jeffball <jeffball@grimm-co.com>
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D25600

Centralize compatability translation macros.

Copy the CP, PTRIN, etc macros from freebsd32.h into a sys/abi_compat.h
and replace existing definitation with includes where required. This
eliminates duplicate code and allows Linux and FreeBSD compatability
headers to be included in the same files.

Input from: cem, jhb
Obtained from: CheriBSD
MFC after: 2 weeks
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D24275

sem_remove(): fix the loop that compacts sem array on semaphores removal.

As written now, it copies random kernel memory from beyond the bounds
of the array.

Reported and tested by: pho
Reviewed by: markj
Sponsored by: The FreeBSD Foundation (kib)
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D23694

sem_remove(): add some asserts.

Assert that sema[idx] allocation from sem[] is sane.
Also assert that sem_mtx is owned, it protects the SEM_ALLOC flag.

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation (kib)
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D23694

Use designated initializers for seminfo.

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation (kib)
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D23694

Correct the malloc tag used when freeing the temporary semop(2) buffer.

MFC after: 1 week
Sponsored by: The FreeBSD Foundation

Move most of the contents of opt_compat.h to opt_global.h.

opt_compat.h is mentioned in nearly 180 files. In-progress network
driver compabibility improvements may add over 100 more so this is
closer to "just about everywhere" than "only some files" per the
guidance in sys/conf/options.

Keep COMPAT_LINUX32 in opt_compat.h as it is confined to a subset of
sys/compat/linux/*.c. A fake _COMPAT_LINUX option ensure opt_compat.h
is created on all architectures.

Move COMPAT_LINUXKPI to opt_dontuse.h as it is only used to control the
set of compiled files.

Reviewed by: kib, cem, jhb, jtl
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D14941

Rename kernel-only members of semid_ds and msgid_ds.

This deliberately breaks the API in preperation for future syscall
revisions which will remove these nonstandard members.

In an exp-run a single port (devel/qemu-user-static) was found to
use them which it did becuase it emulates system calls. This has
been fixed in the ports tree.

PR: 224443 (exp-run)
Reviewed by: kib, jhb (previous version)
Exp-run by: antoine
Sponsored by: DARPA, AFRP
Differential Revision: https://reviews.freebsd.org/D14490

Correct/improve the descriptions if kern.ipc.(shmsegs,sema,msqids).

The description of kern.ipc.shmsegs was wrong since 2005. I updated the
others (which were more correct) to match.

PR: 225933
Reviewed by: cem
MFC after: 3 days
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D14391

Add kern.ipc.{msqids,semsegs,sema} sysctls for FreeBSD32.

Stop leaking kernel pointers though theses sysctls and make sure that the
padding in the structures is zeroed on allocation to avoid other leaks.

Reviewed by: gordon, kib
Obtained from: CheriBSD
MFC after: 1 week
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D13459

sys/kern: adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

sysvsem: check if semu_list has anything on it before grabbing the lock

This should get a process-specific support instead.

MFC after: 1 week

Audit arguments to System V IPC system calls implementing sempahores,
message queues, and shared memory.

Obtained from: TrustedBSD Project
MFC after: 3 weeks
Sponsored by: DARPA, AFRL

When handling msgsys(2), semsys(2), and shmsys(2) multiplex system calls,
map the 'which' argument into a suitable audit event identifier for the
specific operation requested.

Obtained from: TrustedBSD Project
MFC after: 3 weeks
Sponsored by: DARPA, AFRL

sys/kern: spelling fixes in comments.

No functional change.

osd(9): Change array pointer to array pointer type from void*

This is a minor follow-up to r297422, prompted by a Coverity warning. (It's
not a real defect, just a code smell.) OSD slot array reservations are an
array of pointers (void **) but were cast to void* and back unnecessarily.
Keep the correct type from reservation to use.

osd.9 is updated to match, along with a few trivial igor fixes.

Reported by: Coverity
CID: 1353811
Sponsored by: EMC / Isilon Storage Division

Redo the changes to the SYSV IPC sysctl functions from r298585, so they
don't (mis)use sbufs.

PR: 48471

Encapsulate SYSV IPC objects in jails. Define per-module parameters
sysvmsg, sysvsem, and sysvshm, with the following bahavior:

inherit: allow full access to the IPC primitives. This is the same as
the current setup with allow.sysvipc is on. Jails and the base system
can see (and moduly) each other's objects, which is generally considered
a bad thing (though may be useful in some circumstances).

disable: all no access, same as the current setup with allow.sysvipc off.

new: A jail may see use the IPC objects that it has created. It also
gets its own IPC key namespace, so different jails may have their own
objects using the same key value. The parent jail (or base system) can
see the jail's IPC objects, but not its keys.

PR: 48471
Submitted by: based on work by kikuchan98@gmail.com
MFC after: 5 days

sys: use our roundup2/rounddown2() macros when param.h is available.

rounddown2 tends to produce longer lines than the original code
and when the code has a high indentation level it was not really
advantageous to do the replacement.

This tries to strike a balance between readability using the macros
and flexibility of having the expressions, so not everything is
converted.

Indentation issues.

Contract some lines leftover from r298310.

Mea culpa.

kernel: use our nitems() macro when it is available through param.h.

No functional change, only trivial cases are done in this sweep,

Discussed in: freebsd-current

kern: for pointers replace 0 with NULL.

These are mostly cosmetical, no functional change.

Found with devel/coccinelle.

semget(): Check for [EEXIST] error first.

Although POSIX literally permits failing with [EINVAL] if IPC_CREAT and
IPC_EXCL were both passed, the semaphore set already exists and has fewer
semaphores than nsems, this does not allow an application to retry safely:
if the [EINVAL] is actually because of the semmsl limit, an infinite loop
would result.

PR: 206927

Add kern.racct.enable tunable and RACCT_DISABLED config option.
The point of this is to be able to add RACCT (with RACCT_DISABLED)
to GENERIC, to avoid having to rebuild the kernel to use rctl(8).

Differential Revision: https://reviews.freebsd.org/D2369
Reviewed by: kib@
MFC after: 1 month
Relnotes: yes
Sponsored by: The FreeBSD Foundation

Avoid dynamic syscall overhead for statically compiled modules.

The kernel tracks syscall users so that modules can safely unregister them.

But if the module is not unloadable or was compiled into the kernel, there is
no need to do this.

Achieve this by adding SY_THR_STATIC_KLD macro which expands to SY_THR_STATIC
during kernel build and 0 otherwise.

Reviewed by: kib (previous version)
MFC after: 2 weeks

Pull in r267961 and r267973 again. Fix for issues reported will follow.

Revert r267961, r267973:

These changes prevent sysctl(8) from returning proper output,
such as:

1) no output from sysctl(8)
2) erroneously returning ENOMEM with tools like truss(1)
or uname(1)
truss: can not get etype: Cannot allocate memory

Extend the meaning of the CTLFLAG_TUN flag to automatically check if
there is an environment variable which shall initialize the SYSCTL
during early boot. This works for all SYSCTL types both statically and
dynamically created ones, except for the SYSCTL NODE type and SYSCTLs
which belong to VNETs. A new flag, CTLFLAG_NOFETCH, has been added to
be used in the case a tunable sysctl has a custom initialisation
function allowing the sysctl to still be marked as a tunable. The
kernel SYSCTL API is mostly the same, with a few exceptions for some
special operations like iterating childrens of a static/extern SYSCTL
node. This operation should probably be made into a factored out
common macro, hence some device drivers use this. The reason for
changing the SYSCTL API was the need for a SYSCTL parent OID pointer
and not only the SYSCTL parent OID list pointer in order to quickly
generate the sysctl path. The motivation behind this patch is to avoid
parameter loading cludges inside the OFED driver subsystem. Instead of
adding special code to the OFED driver subsystem to post-load tunables
into dynamically created sysctls, we generalize this in the kernel.

Other changes:
- Corrected a possibly incorrect sysctl name from "hw.cbb.intr_mask"
to "hw.pcic.intr_mask".
- Removed redundant TUNABLE statements throughout the kernel.
- Some minor code rewrites in connection to removing not needed
TUNABLE statements.
- Added a missing SYSCTL_DECL().
- Wrapped two very long lines.
- Avoid malloc()/free() inside sysctl string handling, in case it is
called to initialize a sysctl from a tunable, hence malloc()/free() is
not ready when sysctls from the sysctl dataset are registered.
- Bumped FreeBSD version to indicate SYSCTL API change.

MFC after: 2 weeks
Sponsored by: Mellanox Technologies

In order to maximize the re-usability of kernel code in user space this
patch modifies makesyscalls.sh to prefix all of the non-compatibility
calls (e.g. not linux_, freebsd32_) with sys_ and updates the kernel
entry points and all places in the code that use them. It also
fixes an additional name space collision between the kernel function
psignal and the libc function of the same name by renaming the kernel
psignal kern_psignal(). By introducing this change now we will ease future
MFCs that change syscalls.

Reviewed by: rwatson
Approved by: re (bz)

Remove semaphore map entry count "semmap" field and its tuning
option that is highly recommended to be adjusted in too much
documentation while doing nothing in FreeBSD since r2729 (rev 1.1).

ipcs(1) needs to be recompiled as it is accessing _KERNEL private
variables.

Reviewed by: jhb (before comment change on linux code)
Sponsored by: Sandvine Incorporated

All the racct_*() calls need to happen with the proc locked. Fixing this
won't happen before 9.0. This commit adds "#ifdef RACCT" around all the
"PROC_LOCK(p); racct_whatever(p, ...); PROC_UNLOCK(p)" instances, in order
to avoid useless locking/unlocking in kernels built without "options RACCT".

Style fix.

Submitted by: jhb@

Add accounting for SysV-related resources.

Sponsored by: The FreeBSD Foundation
Reviewed by: kib (earlier version)

Add ucred pointer to the SysV-related memory structures. This is required
for racct.

Note that after this commit, ipcs(1) needs to be rebuilt. Otherwise, it will
fail with "ipcs: sysctlbyname: kern.ipc.msqids: Cannot allocate memory".

Sponsored by: The FreeBSD Foundation
Reviewed by: kib (earlier version)

Add some FEATURE macros for various features (AUDIT/CAM/IPC/KTR/MAC/NFS/NTP/
PMC/SYSV/...).

No FreeBSD version bump, the userland application to query the features will
be committed last and can serve as an indication of the availablility if
needed.

Sponsored by: Google Summer of Code 2010
Submitted by: kibab
Reviewed by: arch@ (parts by rwatson, trasz, jhb)
X-MFC after: to be determined in last commit with code from this project

Specify a CTLTYPE_FOO so that a future sysctl(8) change does not need
to rely on the format string.

Add some descriptions to sys/kern sysctls.

PR: kern/148710
Tested by: Chip Camden <sterling at camdensoftware.com>
MFC after: 1 week

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

In another move to join with the age of the Fruitbat, increase SYSV
shared resources defaults beyond absolute minimums.

The new values are chosen mostly by magic. They are still fairly
small and will need increasing for large installations (especially
SHMMAX). However, they are now enough to e.g. start PostgreSQL
installations with ~~300 users and nearly 512 MB of shared buffers.

Reviewed by: A short discussion on hackers@

MFC r205323:
Move SysV IPC freebsd32 compat shims from freebsd32_misc.c to corresponding
sysv_{msg,sem,shm}.c files.

Mark SysV IPC freebsd32 syscalls as NOSTD and add required
SYSCALL_INIT_HELPER/SYSCALL32_INIT_HELPERs to provide auto
register/unregister on module load.

This makes COMPAT_FREEBSD32 functional with SysV IPC compiled and loaded
as modules.

Move SysV IPC freebsd32 compat shims from freebsd32_misc.c to corresponding
sysv_{msg,sem,shm}.c files.

Mark SysV IPC freebsd32 syscalls as NOSTD and add required
SYSCALL_INIT_HELPER/SYSCALL32_INIT_HELPERs to provide auto
register/unregister on module load.

This makes COMPAT_FREEBSD32 functional with SysV IPC compiled and loaded
as modules.

Reviewed by: jhb
MFC after: 2 weeks

Change the ABI of some of the structures used by the SYSV IPC API:
- The uid/cuid members of struct ipc_perm are now uid_t instead of unsigned
short.
- The gid/cgid members of struct ipc_perm are now gid_t instead of unsigned
short.
- The mode member of struct ipc_perm is now mode_t instead of unsigned short
(this is merely a style bug).
- The rather dubious padding fields for ABI compat with SV/I386 have been
removed from struct msqid_ds and struct semid_ds.
- The shm_segsz member of struct shmid_ds is now a size_t instead of an
int. This removes the need for the shm_bsegsz member in struct
shmid_kernel and should allow for complete support of SYSV SHM regions
>= 2GB.
- The shm_nattch member of struct shmid_ds is now an int instead of a
short.
- The shm_internal member of struct shmid_ds is now gone. The internal
VM object pointer for SHM regions has been moved into struct
shmid_kernel.
- The existing __semctl(), msgctl(), and shmctl() system call entries are
now marked COMPAT7 and new versions of those system calls which support
the new ABI are now present.
- The new system calls are assigned to the FBSD-1.1 version in libc. The
FBSD-1.0 symbols in libc now refer to the old COMPAT7 system calls.
- A simplistic framework for tagging system calls with compatibility
symbol versions has been added to libc. Version tags are added to
system calls by adding an appropriate __sym_compat() entry to
src/lib/libc/incldue/compat.h. [1]

PR: kern/16195 kern/113218 bin/129855
Reviewed by: arch@, rwatson
Discussed with: kan, kib [1]

Deprecate the msgsys(), semsys(), and shmsys() system calls by moving
them under COMPAT_FREEBSD[4567]. Starting with FreeBSD 5.0 the SYSV IPC
API was implemented via direct system calls (e.g. msgctl(), msgget(), etc.)
rather than indirecting through the var-args *sys() system calls. The
shmsys() system call was already effectively deprecated for all but
COMPAT_FREEBSD4 already as its implementation for the !COMPAT_FREEBSD4 case
was to simply invoke nosys().

- Move syscall function argument structure types to be just above the
relevenat system call function.
- Whitespace fixes.

Move "options MAC" from opt_mac.h to opt_global.h, as it's now in GENERIC
and used in a large number of files, but also because an increasing number
of incorrect uses of MAC calls were sneaking in due to copy-and-paste of
MAC-aware code without the associated opt_mac.h include.

Discussed with: pjd

Add hierarchical jails. A jail may further virtualize its environment
by creating a child jail, which is visible to that jail and to any
parent jails. Child jails may be restricted more than their parents,
but never less. Jail names reflect this hierarchy, being MIB-style
dot-separated strings.

Every thread now points to a jail, the default being prison0, which
contains information about the physical system. Prison0's root
directory is the same as rootvnode; its hostname is the same as the
global hostname, and its securelevel replaces the global securelevel.
Note that the variable "securelevel" has actually gone away, which
should not cause any problems for code that properly uses
securelevel_gt() and securelevel_ge().

Some jail-related permissions that were kept in global variables and
set via sysctls are now per-jail settings. The sysctls still exist for
backward compatibility, used only by the now-deprecated jail(2) system
call.

Approved by: bz (mentor)

Fix memory leak in semunload().

PR: 133064
Submitted by: Mateusz Guzik <mjguzik@gmail.com>
MFC after: 1 week

Lock the semaphore identifier lock during semaphore initialization to
guarantee atomicity of the operation for other semaphore consumers.
In particular, this should guard against access to the semaphore with
not done or partially done MAC label assignment.

Reviewed by: rwatson
MFC after: 1 month

It seems that there are at least three issues with IPC_RMID operation
on SysV semaphores.

The squeeze of the semaphore array in the kern_semctl() modifies
sem_base for the semaphores with sem_base greater then sem_base of
the removed semaphore, as well as the values of the semaphores,
without locking their mutex. This can lead to (killable) hangs or
unexpected behaviour of the processes performing any sem operations
while other process does IPC_RMID.

The semexit_myhook() eventhandler unlocks SEMUNDO_LOCK() while
accessing *suptr. This allows for IPC_RMID for the sem id to be
performed in parallel with undo hook referenced by the current undo
structure. This leads to the panic("semexit - semid not allocated") [1].

The semaphore creation is protected by Giant, while IPC_RMID is done
while only semaphore mutex is held. This seems to result in invalid
values for semtot, causing random ENOSPC error returns [2].

Redo the locking of the semaphores lifetime cycle. Delegate the
sem_mtx to the sole purpose of protecting semget() and
semctl(IPC_RMID). Introduce new sem_undo_mtx to protect SEM_UNDO
handling. Remove the Giant remnants from the code.
Note that mac_sysvsem_check_semget() and mac_sysvsem_create() are
now called while sem_mtx is held, as well as mac_sysvsem_cleanup() [3].

When semaphore is removed, acquire semaphore locks for all semaphores
with sem_base that is going to be changed by squeeze of the sema
array. The lock order is not important there, because the region is
protected by sem_mtx.

Organize both used and free sem_undo structures into the lists,
protected by sem_undo_mtx. In semexit_myhook(), remove sem_undo
structure that is being processed, from used list, without putting it
onto the free to prevent modifications by other threads. This allows
for sem_undo_lock to be dropped to acquire individial semaphore locks
without violating lock order. Since IPC_RMID may no longer find this
sem_undo, do tolerate references to unallocated semaphores in undo
structure, and check sequential number to not undo unrelated semaphore
with the same id.

While there, convert functions definitions to ANSI C and fix small
style(9) glitches.

Reported by: Omer Faruk Sen <omerfsen gmail com> [1], pho [2]
Reviewed by: rwatson [3]
Tested by: pho
MFC after: 1 month

- Import the HEAD csup code which is the basis for the cvsmode work.

Renew semaphore's pointer after wakeup since during msleep
sem_base may have been modified by destroying one of semaphores
and semptr would not be valid in this case.

PR: kern/123731

Merge first in a series of TrustedBSD MAC Framework KPI changes
from Mac OS X Leopard--rationalize naming for entry points to
the following general forms:

mac_<object>_<method/action>
mac_<object>_check_<method/action>

The previous naming scheme was inconsistent and mostly
reversed from the new scheme. Also, make object types more
consistent and remove spaces from object types that contain
multiple parts ("posix_sem" -> "posixsem") to make mechanical
parsing easier. Introduce a new "netinet" object type for
certain IPv4/IPv6-related methods. Also simplify, slightly,
some entry point names.

All MAC policy modules will need to be recompiled, and modules
not updates as part of this commit will need to be modified to
conform to the new KPI.

Sponsored by: SPARTA (original patches against Mac OS X)
Obtained from: TrustedBSD Project, Apple Computer

Relock the sema_mtxp unconditionally after copyin() for SETALL case in
kern_semctl. Otherwise, later mtx_unlock() can operate on unlocked mutex.

Submitted by: rdivacky
MFC after: 3 days
Approved by: re (kensmith)

Avoid manipulating semu_list outside of the scope of SEMUNDO_LOCK(). This
would lead to an occasional hang with a cycle in semu_list.

X-Discussed-On: hackers@

Further system call comment cleanup:

- Remove also "MP SAFE" after prior "MPSAFE" pass. (suggested by bde)
- Remove extra blank lines in some cases.
- Add extra blank lines in some cases.
- Remove no-op comments consisting solely of the function name, the word
"syscall", or the system call name.
- Add punctuation.
- Re-wrap some comments.

Remove 'MPSAFE' annotations from the comments above most system calls: all
system calls now enter without Giant held, and then in some cases, acquire
Giant explicitly.

Remove a number of other MPSAFE annotations in the credential code and
tweak one or two other adjacent comments.

Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h
begun with a repo-copy of mac.h to mac_framework.h. sys/mac.h now
contains the userspace and user<->kernel API and definitions, with all
in-kernel interfaces moved to mac_framework.h, which is now included
across most of the kernel instead.

This change is the first step in a larger cleanup and sweep of MAC
Framework interfaces in the kernel, and will not be MFC'd.

Obtained from: TrustedBSD Project
Sponsored by: SPARTA

Remove MAC_DEBUG + MPRINTF debugging from System V IPC. This no longer
appears to be serving a useful purpose, as it was used during initial
development of MAC support for System V IPC.

MFC after: 1 month
Obtained from: TrustedBSD Project
Suggested by: Christopher dot Vance at SPARTA dot com

Use proper format specifier for pointers in debug printfs (turned off
by default).

Approved by: pjd (mentor)
MFC after: 2 weeks

Rework kern_semctl a bit to always assume the UIO_SYSSPACE case. This
mostly consists of pushing a few copyin's and copyout's up into
__semctl() as all the other callers were already doing the UIO_SYSSPACE
case. This also changes kern_semctl() to set the return value in a passed
in pointer to a register_t rather than td->td_retval[0] directly so that
callers can only set td->td_retval[0] if all the various copyout's succeed.

As a result of these changes, kern_semctl() no longer does copyin/copyout
(except for GETALL/SETALL) so simplify the locking to acquire the semakptr
mutex before the MAC check and hold it all the way until the end of the
big switch statement. The GETALL/SETALL cases have to temporarily drop it
while they do copyin/malloc and copyout. Also, simplify the SETALL case to
remove handling for a non-existent race condition.

Fix semctl(2) breakage from the previous commit. Previously __semctl() had
a local 'semid' variable which was the array index and used uap->semid
as the original IPC id. During the kern_semctl() conversion those two
variables were collapsed into a single 'semid' variable breaking the
places that needed the original IPC ID. To fix, add a new 'semidx'
variable to hold the array index and leave 'semid' unmolested as the IPC
id. While I'm here, explicitly document that the (undocumented, at least
in semctl(2)) SEM_STAT command curiously expects an array index in the
'semid' parameter rather than an IPC id.

Submitted by: maxim

- Add a kern_semctl() helper function for __semctl(). It accepts a pointer
to a copied-in copy of the 'union semun' and a uioseg to indicate which
memory space the 'buf' pointer of the union points to. This is then used
in linux_semctl() and svr4_sys_semctl() to eliminate use of the stackgap.
- Mark linux_ipc() and svr4_sys_semsys() MPSAFE.

Move some functions and definitions from uipc_socket2.c to uipc_socket.c:

- Move sonewconn(), which creates new sockets for incoming connections on
listen sockets, so that all socket allocate code is together in
uipc_socket.c.

- Move 'maxsockets' and associated sysctls to uipc_socket.c with the
socket allocation code.

- Move kern.ipc sysctl node to uipc_socket.c, add a SYSCTL_DECL() for it
to sysctl.h and remove lots of scattered implementations in various
IPC modules.

- Sort sodealloc() after soalloc() in uipc_socket.c for dependency order
reasons. Statisticize soalloc() and sodealloc() as they are now
required only in uipc_socket.c, and are internal to the socket
implementation.

After this change, socket allocation and deallocation is entirely
centralized in one file, and uipc_socket2.c consists entirely of socket
buffer manipulation and default protocol switch functions.

MFC after: 1 month

Gratuitous renaming of four System V Semaphore MAC Framework entry
points to convert _sema() to _sem() for consistency purposes with
respect to the other semaphore-related entry points:

mac_init_sysv_sema() -> mac_init_sysv_sem()
mac_destroy_sysv_sem() -> mac_destroy_sysv_sem()
mac_create_sysv_sema() -> mac_create_sysv_sem()
mac_cleanup_sysv_sema() -> mac_cleanup_sysv_sem()

Congruent changes are made to the policy interface to support this.

Obtained from: TrustedBSD Project
Sponsored by: SPAWAR, SPARTA

Remove end-of-line tabs.

MFC after: 3 days

check copyin return value

Noticed by: Coverity Prevent analysis tool

Add much needed descriptions for a number of the IPC related sysctl OIDs.
This information will be very useful for people who are tuning applications
which have a dependence on IPC mechanisms.

The following OIDs were documented:

Message queues:
kern.ipc.msgmax
kern.ipc.msgmni
kern.ipc.msgmnb
kern.ipc.msgtlq
kern.ipc.msgssz
kern.ipc.msgseg

Semaphores:
kern.ipc.semmap
kern.ipc.semmni
kern.ipc.semmns
kern.ipc.semmnu
kern.ipc.semmsl
kern.ipc.semopm
kern.ipc.semume
kern.ipc.semusz
kern.ipc.semvmx
kern.ipc.semaem

Shared memory:
kern.ipc.shmmax
kern.ipc.shmmin
kern.ipc.shmmni
kern.ipc.shmseg
kern.ipc.shmall
kern.ipc.shm_use_phys
kern.ipc.shm_allow_removed
kern.ipc.shmsegs

These new descriptions can be viewed using sysctl -d

PR: kern/65219
Submitted by: Dan Nelson <dnelson at allantgroup dot com> (modified)
No objections: developers@
Descriptions reviewed by: gnn
MFC after: 1 week

Invoke label initialization, creation, cleanup, and tear-down MAC
Framework entry points for System V IPC semaphores.

Submitted by: Dandekar Hrishikesh <rishi_dandekar at sbcglobal dot net>
Obtained from: TrustedBSD Project
Sponsored by: DARPA, SPAWAR, McAfee Research

/* -> /*- for copyright notices, minor format tweaks as necessary

Correct two incorrectly merged changes introduced in sysv_sem.c:1.71:
return EINVAL rather than setting error, and don't free sops
unconditionally. The first change was merged accidentally as part of
the larger set of changes to introduce MAC labels and access control,
and potentially lead to continued processing of a request even after
it was determined to be invalid. The second change was due to changes
in the semaphore code since the original work was performed.

Pointed out by: truckman

Second of several commits to allow kernel System V IPC data structures
to be modified and extended without breaking the user space ABI:

Use _kernel variants on _ds structures for System V sempahores, message
queues, and shared memory. When interfacing with userspace, export
only the _ds subsets of the _kernel data structures. A lot of search
and replace.

Define the message structure in the _KERNEL portion of msg.h so that it
can be used by other kernel consumers, but not exposed to user space.

Submitted by: Dandekar Hrishikesh <rishi_dandekar at sbcglobal dot net>
Obtained from: TrustedBSD Project
Sponsored by: DARPA, SPAWAR, McAfee Research

Add missing #include <sys/module.h>

Adjust the number of processes waiting on a semaphore properly if we're
woken up in the middle of sleeping.

PR: misc/64347
Reviewed by: tjr
MFC after: 7 days

Reduce the overhead of semop() by using the kernel stack instead of
malloc'd memory to store the operations array if it is small enough
to fit.

Initialize sequence numbers to 0 in seminit() instead of using whatever
garbage happens to be in memory. This did not seem to cause any problems
except making semaphore ID's unpredictable (and ugly in ipcs(1) output).

When there are no free sem_undo structs available in semu_alloc(), only
free one sem_undo with un_cnt == 0 instead of all of them. This is a
temporary workaround until the SLIST_FOREACH_PREVPTR loop gets fixed so
that it doesn't cause cycles in semu_list when removing multiple adjacent
items. It might be easier to just use (doubly-linked) LISTs here instead
of complicated SLIST code to achieve O(1) removals.

This bug manifested itself as a complete lockup under heavy semaphore use
by multiple processes with the SEM_UNDO flag set.

PR: 58984

Slight whitespace consistency improvement:
Trim trailing whitespace.
Remove unmatched " " before ")".

Change all SYSCTLS which are readonly and have a related TUNABLE
from CTLFLAG_RD to CTLFLAG_RDTUN so that sysctl(8) can provide
more useful error messages.

Update some argument-documenting comments to match reality.

Add an explicit range check to those same arguments to reduce risk of
cardiac arrest in future code readers.

Use __FBSDID().

Replace the at_fork, at_exec, and at_exit functions with the slightly more
flexible process_fork, process_exec, and process_exit eventhandlers. This
reduces code duplication and also means that I don't have to go duplicate
the eventhandler locking three more times for each of at_fork, at_exec, and
at_exit.

Reviewed by: phk, jake, almost complete silence on arch@

Back out M_* changes, per decision of the TRB.

Approved by: trb

Fix two fatal signedness errors introduced when i and j in semop()
were changed from int to size_t in the previous revision.

PR: 47625

Bring semop() closer the the opengroup standards.

PR: 47471
Submitted by: Craig Rodrigues <rodrigc@attbi.com>

Remove M_TRYWAIT/M_WAITOK/M_WAIT. Callers should use 0.
Merge M_NOWAIT/M_DONTWAIT into a single flag M_NOWAIT.

Don't leak memory in semop(2). (Fix a bug I introduced in rev 1.55.)

Detective work by: jake

Make SYSVSEM mpsafe. Each semaphore set gets its own lock, however
there is a global lock over the undo structures because of the way
they are managed.

Switch to using SLIST instead of rolling our own linked list.

Fix several races where a permission check was done before a
copyin/copyout, if the copy happened to fault it may have been
possible to race for access to a semaphore set that one shouldn't
have access to.

Requested by: rwatson
Tested by: NetBSD regression suite.

Cleanup:
Fix line wrapping.
Remove 'register'.
malloc(9) with M_WAITOK can't fail, so remove checks for that.

Cleanup:
Define a debug printf macro rather than wrapping all calls to printf
with #ifdefs.

Remove caddr_t.

Remove __P.

Pacify gcc-3.1+, initialize two variables to avoid -Wuninitialized
warnings.

Fix a typo, unbreak the world.

Thanks to: mux
Approved by: ru

Maximum semid is seminfo.semmni not seminfo.semmsl.

PR: kern/34979
Submitted by: James Gritton <jamie@gritton.org>
Reviewed by: alfred, ru
Approved by: ru
MFC after: 1 week

- Use td_ucred for jail checks.
- Move jail checks and some other checks involving constants and stack
variables out from under Giant. This isn't perfectly safe atm because
jail_sysvipc_allowed is read w/o a lock meaning that its value could be
stale. This global variable will soon become a per-jail flag, however,
at which time it will either not need a lock or will use the prison lock.

Fix Giant leakage in several error cases in __semctl().

Simple p_ucred -> td_ucred changes to start using the per-thread ucred
reference.

Rehash of 1.43: simply remove the comment, since it's highly redundant
and only partially correct.

style(9) prefers formatted comments in '/*' ... '*/' as opposed to
#if 0'd.

Eliminate semexit_hook using at_exit(9) and rm_at_exit(9).

Reviewed by: alfred

Make AIO a loadable module.

Remove the explicit call to aio_proc_rundown() from exit1(), instead AIO
will use at_exit(9).

Add functions at_exec(9), rm_at_exec(9) which function nearly the
same as at_exec(9) and rm_at_exec(9), these functions are called
on behalf of modules at the time of execve(2) after the image
activator has run.

Use a modified version of tegge's suggestion via at_exec(9) to close
an exploitable race in AIO.

Fix SYSCALL_MODULE_HELPER such that it's archetecuterally neutral,
the problem was that one had to pass it a paramater indicating the
number of arguments which were actually the number of "int". Fix
it by using an inline version of the AS macro against the syscall
arguments. (AS should be available globally but we'll get to that
later.)

Add a primative system for dynamically adding kqueue ops, it's really
not as sophisticated as it should be, but I'll discuss with jlemon when
he's around.

Fix SysV Semaphore Handling.
Updated by peter following KSE and Giant pushdown.
I've running with this patch for two week with no ill side effects.

PR: kern/12014: Fix SysV Semaphore handling
Submitted by: Peter Jeremy <peter.jeremy@alcatel.com.au>

PR: kern/29698 (part)
Reviewed by: audit
Implement SEM_STAT (like IPC_STAT but treats semid as sema-index).
The linuxerator will need it.

PR: kern/29698 (part)
Reviewed by: audit
Add tunables for the sem* and shm* syscontrols for tuning on boottime
until they become dynamic.
SAP R/3 doesn't like the compiled in defaults.

KSE Milestone 2
Note ALL MODULES MUST BE RECOMPILED
make the kernel aware that there are smaller units of scheduling than the
process. (but only allow one thread per process at this time).
This is functionally equivalent to teh previousl -current except
that there is a thread associated with each process.

Sorry john! (your next MFC will be a doosie!)

Reviewed by: peter@freebsd.org, dillon@freebsd.org

X-MFC after: ha ha ha ha

Correct a debugging message.

Make the `nsops' variable in `semop' unsigned. This prevents an
overflow if uap->nsops (which is already unsigned) is over INT_MAX;
consequently, the bounds check below becomes valid. Previously, if a
value over INT_MAX was passed in uap->nsops, the bounds check wouldn't
catch it, and the value would be used to compute copyin()'s third
argument.

Obtained from: NetBSD

Giant Pushdown: sysv shm, sem, and msg calls.

Export via sysctl:
* all members of msginfo from sysv_msg.c;
* msqids from sysv_msg.c;
* sema from sysv_sem.c; and
* shmsegs from sysv_shm.c;

These will be used by ipcs(1) in non-kvm mode.

Reviewed by: tmm

o Move per-process jail pointer (p->pr_prison) to inside of the subject
credential structure, ucred (cr->cr_prison).
o Allow jail inheritence to be a function of credential inheritence.
o Abstract prison structure reference counting behind pr_hold() and
pr_free(), invoked by the similarly named credential reference
management functions, removing this code from per-ABI fork/exit code.
o Modify various jail() functions to use struct ucred arguments instead
of struct proc arguments.
o Introduce jailed() function to determine if a credential is jailed,
rather than directly checking pointers all over the place.
o Convert PRISON_CHECK() macro to prison_check() function.
o Move jail() function prototypes to jail.h.
o Emulate the P_JAILED flag in fill_kinfo_proc() and no longer set the
flag in the process flags field itself.
o Eliminate that "const" qualifier from suser/p_can/etc to reflect
mutex use.

Notes:

o Some further cleanup of the linux/jail code is still required.
o It's now possible to consider resolving some of the process vs
credential based permission checking confusion in the socket code.
o Mutex protection of struct prison is still not present, and is
required to protect the reference count plus some fields in the
structure.

Reviewed by: freebsd-arch
Obtained from: TrustedBSD Project

Use predictable internal names for the sysvipc modules, so we have a
chance of getting dependencies working.

sysvipc loadable.

new syscall entry lkmressys - "reserved loadable syscall"

Make syscall_register allow overwriting of such entries (lkmressys).

o Deny access to System V IPC from within jail by default, as in the
current implementation, jail neither virtualizes the Sys V IPC namespace,
nor provides inter-jail protections on IPC objects.
o Support for System V IPC can be enabled by setting jail.sysvipc_allowed=1
using sysctl.
o This is not the "real fix" which involves virtualizing the System V
IPC namespace, but prevents processes within jail from influencing those
outside of jail when not approved by the administrator.

Reported by: Paulo Fragoso <paulo@nlink.com.br>

Convert all users of fldoff() to offsetof(). fldoff() is bad
because it only takes a struct tag which makes it impossible to
use unions, typedefs etc.

Define __offsetof() in <machine/ansi.h>

Define offsetof() in terms of __offsetof() in <stddef.h> and <sys/types.h>

Remove myriad of local offsetof() definitions.

Remove includes of <stddef.h> in kernel code.

NB: Kernelcode should *never* include from /usr/include !

Make <sys/queue.h> include <machine/ansi.h> to avoid polluting the API.

Deprecate <struct.h> with a warning. The warning turns into an error on
01-12-2000 and the file gets removed entirely on 01-01-2001.

Paritials reviews by: various.
Significant brucifications by: bde

Move the MSG* and SEM* options to opt_sysvipc.h
Remove evil allocation macros from machdep.c (why was that there???) and
use malloc() instead.
Move paramters out of param.h and into the code itself.
Move a bunch of internal definitions from public sys/*.h headers (without
#ifdef _KERNEL even) into the code itself.

I had hoped to make some of this more dynamic, but the cost of doing
wakeups on all sleeping processes on old arrays was too frightening.
The other possibility is to initialize on the first use, and allow
dynamic sysctl changes to parameters right until that point. That would
allow /etc/rc.sysctl to change SEM* and MSG* defaults as we presently
do with SHM*, but without the nightmare of changing a running system.

Remove the undocumented, flawed, broken-as-designed semconfig() syscall.

Remove #ifdef for sem_wakeup() - we just use wakeup().

$Id$ -> $FreeBSD$

Change suser_xxx() to suser() where it applies.

Fixed problems with kernel config file overrides of sysv semaphore
parameters. Prior to this fix a kernel config override would effect
only some of the kernel files, resulting in panics.

PR: kern/9068

Eradicate the variable "time" from the kernel, using various measures.
"time" wasn't a atomic variable, so splfoo() protection were needed
around any access to it, unless you just wanted the seconds part.

Most uses of time.tv_sec now uses the new variable time_second instead.

gettime() changed to getmicrotime(0.

Remove a couple of unneeded splfoo() protections, the new getmicrotime()
is atomic, (until Bruce sets a breakpoint in it).

A couple of places needed random data, so use read_random() instead
of mucking about with time which isn't random.

Add a new nfs_curusec() function.

Mark a couple of bogosities involving the now disappeard time variable.

Update ffs_update() to avoid the weird "== &time" checks, by fixing the
one remaining call that passwd &time as args.

Change profiling in ncr.c to use ticks instead of time. Resolution is
the same.

Add new function "tvtohz()" to avoid the bogus "splfoo(), add time, call
hzto() which subtracts time" sequences.

Reviewed by: bde

Move the "retval" (3rd) parameter from all syscall functions and put
it in struct proc instead.

This fixes a boatload of compiler warning, and removes a lot of cruft
from the sources.

I have not removed the /*ARGSUSED*/, they will require some looking at.

libkvm, ps and other userland struct proc frobbing programs will need
recompiled.

Removed unused #includes.

Back out part 1 of the MCFH that changed $Id$ to $FreeBSD$. We are not
ready for it yet.

Make the long-awaited change from $Id$ to $FreeBSD$

This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.

Boy, I'm glad we're not using sup anymore. This update would have been
insane otherwise.

Clean up -Wunused warnings.

Reviewed by: bde

Somehow managed to miss these four files when converting the SYSV IPC
options over to the new style.

Gack - if you're going to call semexit() from elsewhere, it shouldn't
be static.. :-)

Call semexit() from exit(), in order to process `undo vectors'.
This function has actually never been called.

Update sysv_*.c to get their argument definitions from sysproto.h

A Major staticize sweep. Generates a couple of warnings that I'll deal
with later.
A number of unused vars removed.
A number of unused procs removed or #ifdefed.

Start including <sys/sysproto.h> to get the correct args structs and
prototypes for all syscalls. The args structs are still declared in
comments as in VOP implementation functions. I don't like the
duplication for this, but several more layers of changes are required
to get it right. First we need to catch up with 4.4lite2, which uses
macros to handle struct padding. Then we need to catch up with NetBSD,
which passes the args correctly (as void *). Then we need to handle
varargs functions and struct padding better. I think all the details
can be hidden in machine-generated functions so that the args structs
and verbose macros to reference them don't have to appear in the core
sources.

Add prototypes.

Add bogus casts to hide the evil type puns exposed by the previous
steps. &uap[1] was used to get at the args after the first. This
worked because only the first arg in *uap was declared. This broke
when the machine- genenerated args struct declared all the args
(actually it declares extra args in some cases and depends on the
user stack having some accessible junk after the last arg, not to
mention the user args being on the stack. It isn't possible to
declare a correct args struct for a varargs syscall). The msgsys(),
semsys() and shmsys() syscall interfaces are BAD because they
multiplex several syscalls that have different types of args.
There was no reason to duplicate this sysv braindamage but now
we're stuck with it. NetBSD has reimplemented the syscalls properly
as separate syscalls #220-231.

Declare static functions as static in both their prototype and their
implementation (the latter is optional, and this misfeature was used).

Remove gratuitous #includes.

Continue cleaning up new init stuff.

Fixed init functions argument type - caddr_t -> void *. Fixed a couple of
compiler warnings.

Fix several sysinit functions that had the wrong type and unnecessarily
external linkage.

Remove useless comments saying that SYSINIT() does system initialization.

shm.c:
Remove nearly useless comment that gave wrong pseudo-prototypes.

Reviewed by: julian with quick glances by bruce and others
Submitted by: terry (terry lambert)
This is a composite of 3 patch sets submitted by terry.
they are:
New low-level init code that supports loadbal modules better
some cleanups in the namei code to help terry in 16-bit character support
some changes to the mount-root code to make it a little more
modular..

NOTE: mounting root off cdrom or NFS MIGHT be broken as I haven't been able
to test those cases..

certainly mounting root of disk still works just fine..
mfs should work but is untested. (tomorrows task)

The low level init stuff includes a total rewrite of init_main.c
to make it possible for new modules to have an init phase by simply
adding an entry to a TEXT_SET (or is it DATA_SET) list. thus a new module can
be added to the kernel without editing any other files other than the
'files' file.

Eliminate sloppy common-style declarations. There should be none left for
the LINT configuation.

Remove trailing whitespace.

Use tsleep() rather than sleep so that 'ps' is more informative about
the wait.

All of this is cosmetic. prototypes, #includes, printfs and so on. Makes
GCC a lot more silent.

Got rid of compiler warnings.

Added SYSV ipcs.

Obtained from: NetBSD and FreeBSD-1.1.5