sbin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

devd: Warn for deprecated 'kern' system type

One year ago, I deprecated 'kern' in favor of 'kernel' for the system
name for some power events. I'm about to remove it from the kernel, but
realized there's been no warning generated for users. Preserve POLA by
converting on the fly here and issuing a warning for 14.x, and an fatal
error after we branch 15. Make compiling it an error on 16 to remove
the gross hack after we branch.

Sponsored by: Netflix
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D37584

Fix $() handling, broken since the beginning at r108014.

Due to off-by-one error in brackets counting it consumed the rest of the
string, preventing later variables expansions.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

Regularize my copyright notice

o Remove All Rights Reserved from my notices
o imp@FreeBSD.org everywhere
o regularize punctiation, eliminate date ranges
o Make sure that it's clear that I don't claim All Rights reserved by listing
All Rights Reserved on same line as other copyright holders (but not
me). Other such holders are also listed last where it's clear.

We don't need shell protection for when we're expanding matches.
Don't add it. This should fix when we do regepx matches against
variables we've set and fix wifi bring up.

PR: 231441
Approved by: re@ (kib)
Differential Revision: https://reviews.freebsd.org/D17267

Revert "devd: use C++ style casts"

It seems that nullptr is not yet supported by all compilers used by
FreeBSD. :(

devd: use C++ style casts

devd: use static where possible

We're not, yet, at C++11 capable on all our plaforms.

Use a possibly slower, but C++98 compatibe way to iterate through the
string.

Noticed by: g++ 4.2.1 and Mark Millard

Safely quote all variable expansions.

When expanding a variable set by a message from the kernel, safely
quote all arguments expanded when creating a command line for the
shell.

Reviewd by: Shawn Webb, Oliver Pinter, brd@
Sponsored by: Netflix

devd: Move variable declaration to header

Reminder by: imp

devd: compile at WARNS=6

Verified with "make universe TARGETS='amd64 arm arm64 i386 sparc64'"

devd: correct two warnings

- catching a polymorphic type by value
- "output between 16 and 95 bytes into a destination of size 80"

Remove support for FDDI and token ring media types in userland utilities.

Reviewed by: kib
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D15077

devd: don't pass &fds in useless parameters to select(2)

select(2) should be declared as restrict. In addition the only fd in
the fdset is open O_RDONLY, and it's not a socket that can provide OOB
notifications,

Reviewed by: ian, imp, vangyzen

devd: minor nits

- mark usage as noreturn
- config does not need a virtual destructor

General further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

devd(8): Remove pidfile on shutdown

MFC after: 3 weeks
Sponsored by: Spectra Logic Corp

Use a more stream-lined version of fix_value.

Submitted by: ian@

Implement quote escaping. String values may now contain " if you
it is preceded by \.

foo="I \"like\" C++"

gives the value 'I "like" C++' to the variable 'foo'. If a character
other than " follows the \, both the \ and that character are passed
through.

Differential Revision: https://reviews.freebsd.org/D6286
Sponsored by: Netflix

Renumber copyright clause 4

Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96

Fix build of devd with GCC 4.2

Reported by: olivier
Pointy-hat-to: asomers
MFC after: 27 days
X-MFC-with: 312395
Sponsored by: Spectra Logic Corp

Fix several Coverity CIDs in devd

CID 1362055, 1362054: File descriptor leaks during shutdown
CID 1362013: Potential null-termination fail with long network device names
CID 1362097: Uncaught exception during memory pressure
CID 1362017, 1362016: Unchecked errors, possibly resulting in weird behavior
if two devd instances start at the same time.
CID 1362015: Unchecked error that will probably never fail

Reported by: Coverity
CID: 1362055 1362054 1362013 1362097 1362017 1362016 1362015
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp

Simplify test.

Add a timestamp variable to the environment. This is when the event
was read from the kernel by devd.

Make $_ match the docs.

Sometimes, it's useful to export the entire line to an external
program without listening to the devd socket for all events. Define
two new pseudo variables $*, the entire event from devctl and $_,
the entire event without the type character, since it might be easier
to use in some circumstances.

Increase devd's client socket buffer size to 256KB. This is not as large as
it looks, because we'll hit the sockbuf's mbuf limit long before hitting its
data limit. A 256KB data limit allows creating a ZFS pool on about 450
drives without overflowing the client socket buffers.

MFC after: 4 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D4476

Fix a ton of speelling errors

arc lint is helpful

Reviewed By: allanjude, wblock, #manpages, chris@bsdjunk.com
Differential Revision: https://reviews.freebsd.org/D3337

sbin/devd/devd.8
sbin/devd/devd.cc
Create a new socket, of type SOCK_SEQPACKET, for communicating with
clients. SOCK_SEQPACKET sockets preserve record boundaries,
simplying code in the client. The old SOCK_STREAM socket is
retained for backwards-compatibility with existing clients.

Sponsored by: Spectra Logic

Update userspace users of hw.bus.devctl_disable.

This fixes breakage resulting from r263754.

Reported by: AN <andy@neu.net>
Reviewed by: imp
Pointy hat to: me

sbin/devd/devd.8
sbin/devd/devd.cc
Add a -q flag to devd that will suppress syslog logging at
LOG_NOTICE or below.

Requested by: ian@ and imp@
MFC after: 3 weeks
Sponsored by: Spectra Logic Corporation

sbin/devd/devd.cc
Promoting the SIGINFO handler's log message from LOG_INFO to
LOG_NOTICE, and promoting the "Processing event ..." message from
LOG_DEBUG to LOG_INFO. Setting the logfile to LOG_NOTICE with this
change will have the same result as setting it to LOG_INFO without
this change. Setting it to LOG_INFO with this change will include
the useful "Processing event ..." messages that were previously at
LOG_DEBUG, without including useless messages like "Pushing table".

The intent of this change is that one can log "Processing event ..."
without logging "Pushing table" and related messages that are sent
for every event. The number of lines actually logged is reduced by
about 75% by making this change and setting syslog to LOG_INFO vs
setting syslog to LOG_DEBUG.

etc/syslog.conf
Changing the recommended loglevel to notice instead of info.

Sponsored by: Spectra Logic Corp
MFC after: 4 weeks

sbin/devd/devd.cc
Increase the size of devd's client socket's send buffer from the
default (8k) to 128k. This prevents clients from getting
POLLHUPped during event storms. For example, during zpool creation,
the kernel emits a resource.fs.zfs.statechange event for every vdev
in the pool. A 128k buffer is large enough to hold the statechange
events for a pool with nearly 800 drives.

Reviewed by: ian, imp
Approved by: ken (mentor)
Sponsored by: Spectra Logic Corp
MFC after: 4 weeks

devd: lower priority of action execution logging to info

Reviewed by: asomers

devd: fix a typo in a comment

Reviewed by: asomers

Correct the printf format specifier for total_events.
Add __printflike argument checking for devdlog().

Reported by: pjd
Approved by: gibbs (co-mentor)

Explicitly include <cstdarg> to fix compilation with libc++. It is implicitly
included by libstdc++.

Reported By: Oliver Hartmann
Approved by: gibbs (co-mentor, implicit)

style(9) fixes, including the removal of page break characters. No functional
changes.

Approved by: gibbs (co-mentor)

Add a SIGINFO handler to devd. It will send useful statistics to syslog or
stderr as appropriate. Currently, the only statistic printed is the number of
events received.

Reviewed by: eadler
Approved by: gibbs (co-mentor)

Add syslog(3) support to devd(8).

sbin/devd/devd.cc
All output will now go to syslog(3) if devd is daemonized, or stderr
if it's running in the foreground.

sbin/devd/devd.8
Remove the "-D" flag. Filtering messages by priority now
happens in the usual syslog way. For performance reasons, a few
extra-verbose debugging statements are now conditional on the "-d" (do
not daemonize) flag.

etc/syslog.conf
etc/newsyslog.conf
Direct messages from devd(8) to /var/log/devd.log, but leave it
disabled by default

Reviewed by: eadler
Approved by: gibbs (co-mentor)
MFC after: never (removed a command-line option from devd)

Prefer using the C++ version of the standard headers. These place the
names within the std namespace (and possibly within the global
namespace).

The main advantage is that the C++ versions can provide optimized
versions or simplified interfaces.

devd: Correct typo in comment.

Submitted by: Christoph Mallon <christoph.mallon@gmx.de>
Approved by: cperciva (mentor)

devd: Use simpler dst += *x instead of str.append(x, 1).

Submitted by: Christoph Mallon <christoph.mallon@gmx.de>
Approved by: cperciva (mentor)

devd: Use string::empty() instea of string::length() == 0.

Submitted by: Christoph Mallon <christoph.mallon@gmx.de>
Approved by: cperciva (mentor)

devd: Avoid unnecessary temporary objects (and simplify the code) when handling std::string.

Submitted by: Christoph Mallon <christoph.mallon@gmx.de>
Approved by: cperciva (mentor)

devd: Simplify while (1) { if (x) break; } to while (!x) {}.

Submitted by: Christoph Mallon <christoph.mallon@gmx.de>
Approved by: cperciva (mentor)

devd: Remove call to _exit() from signal handler, which also sets a stop flag.

Submitted by: Christoph Mallon <christoph.mallon@gmx.de>
Approved by: cperciva (mentor)

devd: Use volatile sig_atomic_t for the flag set by a signal handler.

Submitted by: Christoph Mallon <christoph.mallon@gmx.de>
Approved by: cperciva (mentor)

Constify where possible.

Approved by: cperciva (mentor)

Improve devd startup time, by tweaking some string handling routines that are
heavily used when parsing config files. Mostly these changes avoid making
temporary copies of the strings, and avoid doing byte at a time append
operations, on the most-used code path.

On a 1.2 GHz ARM processor this reduces the time to parse the config files
from 13 to 6 seconds.

Reviewed by: imp
Approved by: cognet (mentor)

Fix a descriptor leak in devd. Clients reading /var/run/devd.pipe can close
their socket connection any time, and devd only notices that when it gets an
error trying to write an event to the client. On a system with no device
change activity, clients could connect and disappear repeatedly without devd
noticing, leading to an ever-growing list of open socket descriptors in devd.

Now devd uses poll(2) looking for POLLHUP on all existing clients every time
a new client connection is established, and also periodically (once a minute)
to proactively find zombie clients and reap the socket descriptors. It also
now has a connection limit, configurable with a new -l <num> command line arg.
When the maximum number of connections is reached it stops accepting new
connections until some current clients drop off.

Reviewed by: imp
Approved by: cognet (mentor)

Prefer the use of initalizer lists to ctor assignment.

Approved by: cperciva
MFC after: 2 weeks

Avoid the creation of a temporary object by using the prefix operator
for non-primitive types.

Approved by: cperciva
MFC after: 2 weeks

Constify where possible

Approved by: cperciva
MFC after: 2 weeks

Fix an old bug in devd, where it uses std::sort() to sort the various
lists it reads from its configuration files on the priority field.

Because some items in the lists have the same priority, and std::sort()
is not stable, the exact order in which the items are enumerated does
not have to correspond to the order they appear in the configuration
files.

Apparently this was never noticed with libstdc++, but with libc++ it
could cause the "uhid" entry from /etc/devd/usb.conf to be used instead
of the "ums" entry (which is earlier in the file). This caused the
problem described in the PR: the USB mouse module was never loaded, and
the other actions (such as starting moused) were not executed.

To fix the problem, make devd use std:stable_sort() instead.

Reported by: Jan Beich <jbeich@tormail.org>
PR: bin/172958
MFC after: 2 weeks

Use O_CLOEXEC for open instead of separate fcntl(2) call.

Make devd build with libc++.

MFC after: 3 days

Fix build.

- Add support for a "!" character in regex matching in devd(8). It inverts
the logic (true/false) of the matching.

- Add "!usbus[0-9]+" to IFNET ATTACH notification handler in the default
devd.conf to prevent rc.d/netif from running when usbus[0-9]+ is attached.

Reviewed by: imp

Fix missing closedir()

Reviewed by: imp

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Allow the kernel to generate more spacy things and still have devd
cope. Skip multiple spaces in a few contexts.

PR: 96854
Submitted by: Shin'ya Kumabuchi
MFC after: 1 week

- Avoid calling the copy constructor when it is not necessary.

- Remove unused instance of string.

Expand system into my_system, and add the necessary tidyness that we
need. Close the pidfile. Then close all descriptors >= 3 to avoid
information leakage to children.

This solves the problem of not being able to restart devd when you
have, for example, a dhclient forked to configure your network...

MFC after: 3 days

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

Send all debug to stderr.

- Import the HEAD csup code which is the basis for the cvsmode work.

Add -f option to program's usage(), fix manpage's SYNOPSIS.

o Yacc's lineno variable holds a total number of parsed lines.
Reset it to 1 for each devd config file so if the parser finds
a syntax error devd(8) will report a correct line number.

Submitted by: Niki Denev
MFC after: 2 weeks

Use pidfile(3).

OK'ed by: imp

It was always intended that regular expression matching be case
insensitive. Make it so.

Add a -f configfile option to devd(8), based on a patch submitted by
Wojciech A. Koszek.

Submitted by: Wojciech A. Koszek <dunstan@freebsd.czest.pl>

Reuse delete_and_clear() template helper.

Approved by: imp

Make devd WARNS=4 clean and bump WARNS accordingly. This will insure
that future variable shadowing bugs don't compile.

Reviewed by: imp
Compiled on: alpha i386 sparc64

As pointed out on current@, we don't want to declare a variable in a scope
that just uses the variable and throws it away.

This should fix the subsystem keyword wrt media-type.

MFC After: 2 days

media-type predicate is used in contexts where device-name is undefined.
In those contexted, use subsystem instead.

# This causes dhclient to run again when I plug in my ethernet cable to
# my fxp card in my laptop.

* Open permissions on /var/run/devd.pipe so that any user can read devd events
from this socket
* Enable non-blocking I/O on devd.pipe to keep clients from wedging devd.
If a write(2) on devd.pipe would block, the client in question will be
removed [1]

Requested by: rwatson [1]
Approved by: imp

Replace fch{mod,own} with straight ch{mod,own} as the former cannot be used
on socket file descriptors.

Reviewed by: secteam (cperciva)
Approved by: imp

Add the ability to match the on a media type of the device in question.

Submitted by: sam
Approved by: re (scottl)

Ignore sigpipe so we can properly detach clients to the pipe.

Submitted by: Fredrik Lindberg

Fix parsing of nomatch events.
minor debug fix.

Sync program's usage() with manpage's SYNOPSIS.

Create a unix domain socket and write to it all the events that come
in over the socket.

Parse the ! lines that will soon be coming from the kernel. These are
a generalized notification mechanism for subsystems wishing to report
events.

Revieded by: njl

# The kernel side seems like it might be causing panics for me, but should
# be forthcoming shortly.

Remove old workaround

Drop the pid file after we call the final daemon call. w/o -n would
give the wrong pid.

Submitted by: ru and Lukas Ertl
PR: 54113

sysctlbyname takes a size_t as its 5th argument, not a pointer.
However, since NULL was used, and this is C++, this error went
undetected until g++ 3.3 somehow managed to whine about it.

Reported by: obrien

Create a new C++ header file called devd.hh which has all the class
definitions in it. Begin to document the classes that we use, and how
they interrelate (using comments that I can use with doxygen to
automatically generate docs with).

Update to use is_id_char() in the last patch. I should have done this
before the last commit.

Make better use of the string type when parsing things. This switches
expand one from using a fixed buffer to using a string which
dynamically allocates these things.

Submitted by: green@ (against an earlier version of devd)
Ignored for too long by: imp

Change default to be 'process all events before calling daemon.'

Also, put a small work around into devd to prevent a hang on boot this
would cause because select used to return 2 rather than 0 for no
evetnts due to a bug I fixed a few days ago in subr_bus.c. I'll
remove this workaround May 7th. You have until then to upgrade your
kernel if you want to run a new devd with an older kernel.

Kill debug printf that crept in my last commit

Implement a two new flags, and change the meaning of another one.
'd': now means don't do daemon().
'D': Debug
'n': Don't wait to process all pending events before calling daemon.

In the past, devd would call daemon immediately. However, this causes
a race. If anything in the boot process depends on configuring the
devices configured by devd, maybe they would be configured in time,
maybe not. Now we don't call daemon until all pending events are
processed, unless -n is specified.

# -n is actually the default for a while due to the select(2) bug in devctl
# that I just fixed to give people a chance to upgrade.

Put an anti-foot-shooting measure in place. Set hw.bus.devctl_disable
to 0 when we startup. Print a warning in this case. This allows
people that are playing with devd by hand to have something happen.
Otherwise, it appears that devd isn't working because /dev/devctl is
disabled and producing no events.

Suggested by: peter on irc a long time ago.

o Skip white space between variables better. This makes
'a="b" c="d" at loc=1 on busN' properly set 'c' and process the rest of
the stirng. Before it would ignore everything after variable 'a'.
o Parse nomatch and other events differently. They are more different
than the code allowed for, so we weren't properly parsing nomatch
events. It appears this fixes some of the demand loading issues that
I was having with devd.

Noticed by: Gary Palmer

devd.conf already has the examples in the todo list

Terminate the usage message with \n.

Submitted by: joe

MFp4: make it work

o Expand variables correctly.
o Set variables for each event.
o rewrite event loop to execute the commands in the config file, rather
than the hard wired generic command
o better(?) debug when running -d
o sort vectors of actions so that we just have to search for the first
one to match rather than the best one that matches.
o better attempts to clear all resources used on 'restart'
o Remove now bogus comments

MFC After: 1 centiyear

Latest snapshot of devd. This one adds re-matching and string
expansion.

MFp4 (imp_freebsd branch): snapshot of devd work:
o improve parsing and lexing
o create data structures based on the parsed file now.
o Still need to rewrite main loop and add regex (still uses hard coded
devd-generic)
o minor man page updates.

# There should be one more commit before rc2

Approved by: re (blanket)