export $autoboot var when running from $boottrace_cmd

At the moment, if bootrace profiling is enabled, autoboot is not
exported to the rc scripts. This causes fsck to not check the root
filesystem. To fix this, pass _boot, rc_fast and autoboot to the
boottrace process to mimic what diretly sourcing the rc.d scripts.

PR: 278993
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/1261

rc: improve NAME_setup handling

Reload is used for service reconfiguration as well
and lacks a NAME_prepend-like mechanism so it makes
sense to extend the NAME_reload hook into this
action.

precmd may use configuration checks and blocks setup
from doing its designated work (e.g. nginx). In moving
the invoke of the setup script in front allows us to
provide custom scripts for config file generation and
fixing prior to precmd checking configuration integrity.

Also introduce _run_rc_setup to separate the launcher
from the main one. Let it run correctly in the case
of restart_precmd and block further execution as
would be the case in start due to the internal plumbing
of restart being split into calling stop and start
afterwards.

Differential-Revsiion: https://reviews.freebsd.org/D36259
Signed-off-by: Franco Fichtner <franco@opnsense.org>
Reviewed by: imp, oshogbo
Pull Request: https://github.com/freebsd/freebsd-src/pull/1258

rc: add service jails framework

This takes a rc.d-service and starts it in a jail which shares the same
root-path as the host (or parent jail) and may inherit the network from
the host (or parent jail). Per service there is the possibility to
specify some arguments which give more permissions (e.g. netv4, netv6,
sysvipc...).
Reviewed by: bcr (man page)
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D40370

Allow DEBUG_SH=rc:all to debug all rc.d

Usually a bad idea but there are exceptions to every rule.
Allso debugging all rc.d scripts or all with a given arg.

rc.subr add Exists so we can find sed

SED=`Exists -x /usr/bin/sed /rescue/sed`

avoids adding /rescure to $PATH, and allows use of sed
before /usr is mounted (if a separate filesystem).

Reviewed by: jlduran_gmail.com
Differential Revision: https://reviews.freebsd.org/D43826

rc.subr avoid noise if /usr not mounted

basename, sed and tty are all in /usr/bin and not available
until /usr is mounted.

basename and tty we can replace with a function, but sed is more
important. Fix o_verify to just use shell builtins, and
rc_trace should avoid trying to set RC_LEVEL until sed is available.

/etc/rc add trace debug and verify

Debugging boot issues can be helped by
logging each rc.d script as it is run
and being able to selectively enable/disable set -x
debug.sh provides an elaborate framework for debugging shell scripts.

For secure systems, we want to be paranoid about what we read
during boot.

dot() simply reads (.) arg file if it exists
vdot() if mac_veriexec is active, ignore unverified files
otherwise behaves much the same as dot()
safe_dot() in safe_eval.sh allows reading an untrusted file;
limiting the input to simple variable assignments.

In load_rc_config allow caller to provide an option to indicate how to
handle its arg:
-v use vdot()
-s use sdot() which will try to use vdot() and fallback to safe_dot()
The default is to read using dot()

rc_run_scripts()
encapsulate the running of rc.d scripts
so that we can easily call it more than twice.

We vdot local.rc.subr to pick up extensions (like
run_rc_scripts_final) and overrides.

We also allow rc.subr.local or rc.conf to set rc_config_xtra
eg (rc_config_xtra=XXX for historic compatibility)

rc use set -o verify around the reading in of rc.subr
This has no effect if mac_veriexec is not active, but if it is; ensures
rc.subr has not been tampered with.

Reviewed by: imp
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D43671

Spaces to tabs for consistency. No content change.

rc.subr: don't require service to be enabled for `status`

For a service that sets an rcvar, there is a check whether it has been
enabled before the actual command is executed. If the check fails, one
gets a message to enable it and the returned exit status is 0.
However, this is usually undesirable for the status command, which is
a) supposed to check whether the service is running anyway and
b) returns a non-zero exit code if that is not the case.
Thus, skip the check for the status command.

PR: 272282
Reviewed by: emaste
MFC after: 3 days

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

cpuset(8): move to /bin/

Summary:
This change moves /usr/bin/cpuset to /bin/cpuset so it is more readily
available on hosts where the /usr partition might not be mounted at the
time / has been mounted.

Remove some complexity from rc.subr(8) since /bin is assumed to always
be present if/when / is mounted.

MFC after: 2 weeks
MFC with: 0661f93892a2, 271d552379af

Test Plan:
- Test out rc.subr change.
- Confirm that the installation logic does what's needed.

Relnotes: yes (moving cpuset may impact strict file permissions/mode checking)
Reviewers: kevans
Subscribers: imp
Differential Revision: https://reviews.freebsd.org/D40057

/etc/rc.subr: rename sysctl.conf.d to sysctl.kld.d

This better reflects the intent that this directory is indexed by kld
name and removes a conflict with D32128 which aims to split sysctl.conf.

Reviewed by: kevans imp
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D41043

/etc/rc.subr: add support for kld sysctl variables

For kernel modules loaded by scripts in /etc/rc.d and
/usr/local/etc/rc.d, if there is a file in /etc/sysctl.conf.d named <kld
name>.conf, then this will be loaded using the sysctl(8) utility. For
instance, sysctl variable changes for the pf kernel module would be
placed in the file /etc/sysctl.conf.d/pf.conf.

PR: 272129
Reviewed by: imp freebsd_igalic.co
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D40886

rc.subr(8): delete debug cpuset helper

Summary:
The intention of the original author (I assume) was to add this logic
for testing. This removes the debug statement so it no longer shows up
in calls to `status`.

MFC after: 2 weeks
MFC with: 0661f9389

Reviewers: kevans

Subscribers: imp

Differential Revision: https://reviews.freebsd.org/D40055

rc.subr(8): run `trailing-whitespace-fixer`

This change deletes benign trailing whitespace from rc.subr, making
future non-stylistic changes easier to spot.

MFC after: 1 week

rc: add support for cpuset(1)

If ${name}_cpuset is specified (and /usr is mounted), cpuset(1) will be
run to limit the service to the configured cpuset.

PR: 142434
Reviewed by: kevans

rc: ignore .pkgsave files

The local parts of rc already skip .sample files; we add .pkgsave to the
list, and add logic for base.

Thanks to @RhodiumToad for getting this started.

Differential Revision: https://reviews.freebsd.org/D27962
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/662

rc.subr: Minor formatting fix for ac102a2a6

rc.conf(5): add <service>_umask to run the service using this value

None of tools working with login classes change umask(1)
and we had no ways to specify non-default umask for a service
not touching its startup script. This change makes in possible.

Some file-sharing services that create new files may benefit from it.

Differential: https://reviews.freebsd.org/D36309
MFC-after: 3 days

rc: add ${name}_setup script support

Run a service-based setup script before running the start command.
Useful for automatic configuration file generation.

Reviewed by: https://reviews.freebsd.org/D36006

rc.subr: Fix a typo

MFC after: 3 days

rc.subr: Make sure oomprotect protects existing children

The rc(8) framework support protecting services from OOM killer.
The current implementation applies the protection after the service has
already started. This works fine if only the main process is to be
protected (*_oomprotect=yes). However, the current implementation fails
to protect existing children when children are also to be protected
(*_oomprotect=all). This patch fixes that.

Note: it is not easy to apply the protectoin earlier because we want to
support both the services which use the "command" variable and those
that use the "start_cmd" variable.

PR: 256148
Approved by: adrian, osogbo
Tested by: Jamie Landeg-Jones <jamie@catflap.org>
Fixes: 3bead71e959d - Add a global option where we can protect
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D35747

Introduce startmsg and use it in rc scripts

startmsg is a new rc.subr(8) function function to be used instead of
echo(1) when for boot messages. It replaces the often forgotten

check_startmsgs && echo ...

with

startmsg ...

No functional change intended.

I adjusted the commit message and did some final clean-ups of the patch
before committing.

PR: 255207
Reported by: Jose Luis Duran <jlduran@gmail.com>
Reviewed by: imp, 0mp
Approved by: imp (src)
Differential Revision: https://reviews.freebsd.org/D34514

rc.subr: boottrace annotations

When enabled, have the framework use the boottrace(8) utility to execute
each rc script, generating trace entries for the entire suite of
scripts.

Reviewed by: 0mp (slightly earlier version)
Sponsored by: NetApp, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D31930

Allow rc.d script to provide "status" method, even if it does not
define procname or have a PID file. This might be useful for cases,
such as mounting local FS, when there is no running daemon
still some other persistent state in the system which status
can be checked.

It is still possible to have a status method before this by having
extra_commands="status", but it's not obvious and might give
an script writer some extra legwork to figure out how and why
the straight method is not working.

Reviewed by: cy
MFC after: 1 week
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D31614

rc.subr: use _pidcmd to determine pid for protect

This is a more reliable method that accounts for existing pidfiles,
procname and interpreter settings.

Current method of obtaining the pid for oomprotect="YES"|"ALL" processes
in certain cases fails to find a unique pid.

One such case are rc.d scripts defining command as:
command="daemon"

which results in all processes started via daemon being selected and
passed to protect(1) which fails and prints usage:

$ /etc/rc.d/exampled restart
Stopping exampled.
Starting exampled.
usage: protect [-i] command
protect [-cdi] -g pgrp | -p pid

Running the same with -x reveals what happens:

+ pid='3051 4268 4390 4421 4427 4470 4588 4733 4740 4870 4949 4954 4979
5835 5866 55487 55583 56525 57643 57789 57882 58072 58167 99419'
+ /usr/bin/protect -p 3051 4268 4390 4421 4427 4470 4588 4733 4740 4870
4949 4954 4979 5835 5866 55487 55583 56525 57643 57789 57882 58072 58167
99419
usage: protect [-i] command
protect [-cdi] -g pgrp | -p pid

We have a more reliable way of obtaining pid already defined in rc.subr
and available when protect(1) needs it. We can simply `eval $_pidcmd`
which also invokes `check_process` but properly accounts for existing
pidfile, procname and interpreter settings.

With the change the pidfile is properly obtained.

Submitted by: Adam Wolk <a.wolk at fudosecurity.com>
Sponsored by: Fudo Security
Differential Revision: https://reviews.freebsd.org/D30367

Remove trailing tab.

MFC after: 3 days

rc: Honor ${name}_env when a custom *_cmd is defined (e.g., start_cmd)

A user may set ${name}_env variable in rc.conf(5) in order to set additional
environment variables for a service command. Unfortunately, at the moment
this variable is only honored when the command is specified via the command
variable. Those additional environment variables coming from ${name}_env
are never set if the service is started via the ${rc_arg}_cmd variable (for
example start_cmd).

PR: 239692
Reviewed by: bcr, jilles
Approved by: src (jilles)
Differential Revision: https://reviews.freebsd.org/D21228

Revert r346017 pending compiled-in zfs fix

PR: 237172
Approved by: jilles
Differential Revision: https://reviews.freebsd.org/D18670

Remove now unnecessary kldstat check before attempting to load modules.

Since r233109, kldload has the -n option, which silently ignores options
that are already loaded.

https://lists.freebsd.org/pipermail/freebsd-rc/2018-December/003899.html

Note that this script no longer reports if the module is already loaded,
but it could be argued this wasn't particularly useful information.

PR: docs/234248
Reviewed by: bcr (docs), kib, rgrimes (visual)
Approved by: jilles
Differential Revision: https://reviews.freebsd.org/D18670

rc.subr: Fix typo

Originally intended as 'in case in needs to be re-invoked', but it was later
decided (by myself) that 're-invoke itself' makes it more clear that the
script is expected to use this in a way.

Reported by: Jose Luis Duran (jlduran @ github)
X-MFC-With: r342792

rc.subr: Provide rc_service variable for service scripts

Some rc scripts in ports (e.g. uwsgi, apache, openvpn) allow for
'application profiles' that usually require the rc script to be invoked
again for each active profile. Because there's no consistent way to
determine the path because it differs between manual/service(8) invocations
and /etc/rc invocations, this leads to patterns like these:

- www/uwsgi hardcodes the script path
- security/openvpn guesses either $_file or $0 based on $0 = /etc/rc

Instead of forcing rc scripts to guess, provide an rc_service variable to
the scripts that gets set appropriately both for direct execution or when a
script is being executed via run_rc_script (e.g. /etc/rc).

This is our analog of an OpenRC variable with the same name, different case
(RC_SERVICE).

PR: 234614
Reported by: koobs
Reviewed by: dteske, jilles
MFC after: 3 days

rc.subr: Implement list_vars without using 'read'

'read' pessimistically read(2)s one byte at a time, which can be quite
silly for large environments in slow emulators.

In my boring user environment, truss shows that the number of read()
syscalls to source rc.subr and invoke list_vars is reduced by something like
3400 to 60. ministat(1) shows a significant time difference of about -71%
for my environment.

Suggested by: jilles
Discussed with: dteske, jhb, jilles
Differential Revision: https://reviews.freebsd.org/D18481

Add new rc keywords: enable, disable, delete

This adds new keywords to rc/service to enable/disable a service's
rc.conf(5) variable and "delete" to remove the variable.

When the "service_delete_empty" variable in rc.conf(5) is set to "YES"
(default is "NO") an rc.conf.d file (in /etc/ or /usr/local/etc) is
deleted if empty after modification using "service $foo delete".

Submitted by: lme (modified)
Reviewed by: 0mp (previous version), lme, bcr
Relnotes: yes
Sponsored by: Smule, Inc.
Differential Revision: https://reviews.freebsd.org/D17113

Move the rc framework out of sbin/init into libexec/rc.

The reasons for this are forward looking to pkgbase:
* /sbin/init is a special binary; try not to replace it with
every package update because an rc script was touched.
(a follow-up commit will make init its own package)
* having rc in its own place will allow more easy replacement
of the rc framework with alternatives, such as openrc.

Discussed with: brd (during BSDCam), kmoore
Requested by: cem, bz
PR: 231522
Approved by: re (gjb)