MFC SVN r295342-295344

Differential Revision: https://reviews.freebsd.org/D12568
Submitted by: Vinicius Zava (egypcio at googlemail.com)
Reviewed by: allanjude

MFC r301683:

Fix typo with description for $ipv6_cpe_wanif (upstram -> upstream)

PR: 210146

MFC: r299242
Make "-S" a default option for mountd.

After a discussion on freebsd-fs@ there seemed to be a consensus that
the "-S" option for mountd should become the default.
Since the only known issue w.r.t. using "-S" was fixed by r299201,
this commit adds "-S" to the default mountd_flags.

MFC r295471:

Add new rc.conf parameter "jail_reverse_stop"

When a user defines "jail_list" in rc.conf the jails are started in the
order defined. Currently the jails are not are stopped in reverse order
which may break dependencies between jails/services and prevent a clean
shutdown. The new parameter "jail_reverse_stop" will shutdown jails in
"jail_list" in reverse order when set to "YES".

Please note that this does not affect manual invocation of the jail rc
script. If a user runs the command

# service jail stop jail1 jail2 jail3

the jails will be stopped in exactly the order specified regardless of
jail_reverse_stop being defined in rc.conf.

MFC r295568:

Document the new jail_reverse_stop parameter

While here clean up the documentation for jail_list

PR: 196152
Submitted by: feld

MFC r289421, r293037, r294773, and r294884.

ntp leap-leapseconds support.

r289421:

Add default leap-seconds file. This should help ntp networks get the
leap second date correct

Updates to the file can be obtained from ftp://time.nist.gov/pub/ or
ftp://tycho.usno.navy.mil/pub/ntp/.

r293037:

Update leap-seconds to latest. This will satisfy the ntpd leap-second
version check.

r294773:

Add support for automatic leap-second file updates.

The working copy of leapfile resides in /var/dbntpd.leap-seconds.list.
/etc/ntp/leap-seconds (periodically updated from ftp://time.nist.gov/pub/
or ftp://tycho.usno.navy.mil/pub/ntp/) contains the master copy should
automatic leapfile updates be disabled (default).

Automatic leapfile updates are fetched from $ntp_leapfile_sources,
defaulting to https://www.ietf.org/timezones/data/leap-seconds.list,
within $ntp_leapfile_expiry_days (default 30 days) from leap-seconds
file expiry. Automatic updates can be enabled by setting
$daily_ntpd_leapfile_enable="YES" in periodic.conf. To avoid congesting
the ntp leapfile source the automatic update randomized by default but
can be disabled through daily_ntpd_avoid_congestion="NO" in
periodic.conf.

r294884:

Allow specification of fetch options for ntp leap-seconds fetch.

Approved by: re (gjb)

MFC r292752:

Enhance rc.d/netwait script to wait for late-attaching interfaces such as
USB NICs.

USB network hardware may not be enumerated and available when the rc.d
networking scripts run. Eventually the USB attachment completes and devd
events cause the network initialization to happen, but by then other rc.d
scripts have already failed, because services which depend on NETWORKING
(such as mountcritremote) may end up running before the network is actually
ready.

There is an existing netwait script, but because it is dependent on
NETWORKING it runs too late to prevent failure of some other rc
scripts. This change flips the order so that NETWORKING depends on netwait,
and netwait now depends on devd and routing (the former is needed to make
interfaces appear, and the latter is needed to run the ping tests in
netwait).

The netwait script used to be oriented primarily towards "as soon as any
host is reachable the network is fully functional", so you gave it a list of
IPs to try and you could optionally name an interface and it would wait for
carrier on that interface. That functionality still works the same, but now
you can provide a list of interfaces to wait for and it waits until each one
of them is available. The ping logic still completes as soon as the first IP
on the list responds.

These changes were submitted by Brenden Molloy <brendan+freebsd@bbqsrc.net>
in PR 205186, and lightly modified by me to allow a list of interfaces
instead of just one.

PR: 205186
Relnotes: yes

MFC: r291536
Add support for the new "-manage-gids" option for the nfsuserd daemon
to the rc scripts. With these changes, setting nfs_server_managegids="YES"
in /etc/rc.conf will enable this capability.

Relnotes: yes

MFC r274841

Remove examples of gif_interfaces and gifconfig. These have already been
marked as deprecated in rc.conf(5) manual page but these examples
were still here.

Approved by: re (gjb)

MFH (r270391): don't set rc_debug in /etc/defaults/rc.conf

MFC r274965:

Make it possible to specify flags for autofs daemons in rc.conf.

Sponsored by: The FreeBSD Foundation

MFC r271663:
Fix a typo; master server for iprop service should be singular.

MFC r270782:

Restructure rc.d scripts for kerberos5 daemons:

- Rename $kerberos5_server_enable with $kdc_enable and rename
rc.d/kerberos with rc.d/kdc.

- Rename $kadmin5_server_enable with $kadmind_enable.

- Rename ${kerberos5,kpasswdd}_server with ${kdc,kpasswdd}_program.

- Fix rc.d/{kadmind,kerberos,kpasswdd,kfd} scripts not to change variables
after load_rc_config().

- Add rc.d/ipropd_master and rc.d/ipropd_slave scripts. These are
for iprop-master(8) and iprop-slave(8). Keytab used for iprop service is
defined in ipropd_{master,slave}_keytab (/etc/krb5.keytab by default).

- Add dependency on rc.d/kdc to SERVERS. rc.d/kdc must be invoked as early
as possible before scripts divided by rc.d/SERVERS.

Note that changes to rc.d/{kdc,kpasswdd,kadmind} are backward-compatible
with the old configuration variables:
${kerberos5,kpasswdd,kadmin5}_server{,_enable,_flags}.

MFC r270780:
Fix rc.d/gssd script to define the default values in a standard way.

MFC r270781:
- Add a warning message when an IPv6 address is specified with no prefixlen.
- Use a parameter argument in jls(8) instead of doing grep.

MFC r271424:

- Add $netif_ipexpand_max to specify the upper limit for the number of
addresses generated by an address range specification. The default value
is 2048. This can be increased by setting $netif_ipexpand_max in rc.conf.

PR: 186841

MFC r269392: Add /usr/lib32/compat to the default ld-elf32.so.1 path.

Approved by: re (gjb, delphij)

MFC r270647: Add references to vt(4) and the configuration files in /usr/sha
MFC r270653: Update man-pages to correctly refer to changed pathes and namin
MFC r270657: More man pages that need to know about vt in addition to syscon
MFC r270659: (by pluknet@) Missed comma.
MFC r270660: Back-out the references to vt(4) from this man-page. It appears
MFC r270933: Add references to vt(4) to further man-pages.
MFC r270934: Final patches to the tools used to convert syscons keymaps for
MFC r270935: Add vt(4) support to the console initialisation script, specifi

Second batch of MFCs to add support for Unicode keymaps for use with vt(4).

It contains the following changes:

- Add references to vt(4) to relevant man-pages.
- Update comment in defaults/rc.conf to mention vt
- Update rc.d/syscons to warn about syscons keymaps used under vt.
An attempt is made to identify the vt keymap to load instead.
- Minor changes to the conversion tool based on mail comments on keymaps.

Relnotes: yes

MFC r270096:

Bring in the new automounter, similar to what's provided in most other
UNIX systems, eg. MacOS X and Solaris. It uses Sun-compatible map format,
has proper kernel support, and LDAP integration.

There are still a few outstanding problems; they will be fixed shortly.

Reviewed by: allanjude@, emaste@, kib@, wblock@ (earlier versions)
Phabric: D523
Relnotes: yes
Sponsored by: The FreeBSD Foundation

Fix devfs rules not applied by default for jails.

Fix OpenSSL use-after-free vulnerability.

Fix TCP reassembly vulnerability.

Security: FreeBSD-SA-14:07.devfs
Security: CVE-2014-3001
Security: FreeBSD-SA-14:08.tcp
Security: CVE-2014-3000
Security: FreeBSD-SA-14:09.openssl
Security: CVE-2010-5298

MFC r261931,r261932:

Allow overriding rctl.conf(5) file location for /etc/rc.d/rctl

Approved by: bapt (mentor, implicit)

MFC r260486:
Remove very low default limit of 4 nfsd threads. nfsd's own default is
8 * hw.ncpu, that sounds more appropriate for these SMP/NCQ/... days.

Merge r257694 from head:

Remove remnants of BIND from /etc, since there is no BIND in base now.

Sorry, that would break users running head and BIND from ports, since
ports rely on these scripts. The ports will be fixed soon.

Approved by: re (kib)

MFC r256773:
Enable the automatic creation of a certificate (if one does not exists)
and enable the usage by sendmail if sendmail is enabled.

Reviewed by: gshapiro
Approved by: re (gjb)

MFC r256775,r256776:
Add support for "first boot" rc.d scripts.

Document this new functionality in rc.conf(5) and rc(8).

Bump __FreeBSD_version so that ports can make use of this.

Approved by: re (gjb)

Merge from project branch via main. Uninteresting commits are trimmed.

Refactor of /dev/random device. Main points include:

* Userland seeding is no longer used. This auto-seeds at boot time
on PC/Desktop setups; this may need some tweeking and intelligence
from those folks setting up embedded boxes, but the work is believed
to be minimal.

* An entropy cache is written to /entropy (even during installation)
and the kernel uses this at next boot.

* An entropy file written to /boot/entropy can be loaded by loader(8)

* Hardware sources such as rdrand are fed into Yarrow, and are no
longer available raw.

------------------------------------------------------------------------
r256240 | des | 2013-10-09 21:14:16 +0100 (Wed, 09 Oct 2013) | 4 lines

Add a RANDOM_RWFILE option and hide the entropy cache code behind it.
Rename YARROW_RNG and FORTUNA_RNG to RANDOM_YARROW and RANDOM_FORTUNA.
Add the RANDOM_* options to LINT.

------------------------------------------------------------------------
r256239 | des | 2013-10-09 21:12:59 +0100 (Wed, 09 Oct 2013) | 2 lines

Define RANDOM_PURE_RNDTEST for rndtest(4).

------------------------------------------------------------------------
r256204 | des | 2013-10-09 18:51:38 +0100 (Wed, 09 Oct 2013) | 2 lines

staticize struct random_hardware_source

------------------------------------------------------------------------
r256203 | markm | 2013-10-09 18:50:36 +0100 (Wed, 09 Oct 2013) | 2 lines

Wrap some policy-rich code in 'if NOTYET' until we can thresh out
what it really needs to do.

------------------------------------------------------------------------
r256184 | des | 2013-10-09 10:13:12 +0100 (Wed, 09 Oct 2013) | 2 lines

Re-add /dev/urandom for compatibility purposes.

------------------------------------------------------------------------
r256182 | des | 2013-10-09 10:11:14 +0100 (Wed, 09 Oct 2013) | 3 lines

Add missing include guards and move the existing ones out of the
implementation namespace.

------------------------------------------------------------------------
r256168 | markm | 2013-10-08 23:14:07 +0100 (Tue, 08 Oct 2013) | 10 lines

Fix some just-noticed problems:

o Allow this to work with "nodevice random" by fixing where the
MALLOC pool is defined.

o Fix the explicit reseed code. This was correct as submitted, but
in the project branch doesn't need to set the "seeded" bit as this
is done correctly in the "unblock" function.

o Remove some debug ifdeffing.

o Adjust comments.

------------------------------------------------------------------------
r256159 | markm | 2013-10-08 19:48:11 +0100 (Tue, 08 Oct 2013) | 6 lines

Time to eat crow for me.

I replaced the sx_* locks that Arthur used with regular mutexes;
this turned out the be the wrong thing to do as the locks need to
be sleepable. Revert this folly.

# Submitted by: Arthur Mesh <arthurmesh@gmail.com> (In original diff)

------------------------------------------------------------------------
r256138 | des | 2013-10-08 12:05:26 +0100 (Tue, 08 Oct 2013) | 10 lines

Add YARROW_RNG and FORTUNA_RNG to sys/conf/options.

Add a SYSINIT that forces a reseed during proc0 setup, which happens
fairly late in the boot process.

Add a RANDOM_DEBUG option which enables some debugging printf()s.

Add a new RANDOM_ATTACH entropy source which harvests entropy from the
get_cyclecount() delta across each call to a device attach method.

------------------------------------------------------------------------
r256135 | markm | 2013-10-08 07:54:52 +0100 (Tue, 08 Oct 2013) | 8 lines

Debugging. My attempt at EVENTHANDLER(multiuser) was a failure; use
EVENTHANDLER(mountroot) instead.

This means we can't count on /var being present, so something will
need to be done about harvesting /var/db/entropy/... .

Some policy now needs to be sorted out, and a pre-sync cache needs
to be written, but apart from that we are now ready to go.

Over to review.

------------------------------------------------------------------------
r256094 | markm | 2013-10-06 23:45:02 +0100 (Sun, 06 Oct 2013) | 8 lines

Snapshot.

Looking pretty good; this mostly works now. New code includes:

* Read cached entropy at startup, both from files and from loader(8)
preloaded entropy. Failures are soft, but announced. Untested.

* Use EVENTHANDLER to do above just before we go multiuser. Untested.

------------------------------------------------------------------------
r256088 | markm | 2013-10-06 14:01:42 +0100 (Sun, 06 Oct 2013) | 2 lines

Fix up the man page for random(4). This mainly removes no-longer-relevant
details about HW RNGs, reseeding explicitly and user-supplied
entropy.

------------------------------------------------------------------------
r256087 | markm | 2013-10-06 13:43:42 +0100 (Sun, 06 Oct 2013) | 6 lines

As userland writing to /dev/random is no more, remove the "better
than nothing" bootstrap mode.

Add SWI harvesting to the mix.

My box seeds Yarrow by itself in a few seconds! YMMV; more to follow.

------------------------------------------------------------------------
r256086 | markm | 2013-10-06 13:40:32 +0100 (Sun, 06 Oct 2013) | 11 lines

Debug run. This now works, except that the "live" sources haven't
been tested. With all sources turned on, this unlocks itself in
a couple of seconds! That is no my box, and there is no guarantee
that this will be the case everywhere.

* Cut debug prints.

* Use the same locks/mutexes all the way through.

* Be a tad more conservative about entropy estimates.

------------------------------------------------------------------------
r256084 | markm | 2013-10-06 13:35:29 +0100 (Sun, 06 Oct 2013) | 5 lines

Don't use the "real" assembler mnemonics; older compilers may not
understand them (like when building CURRENT on 9.x).

# Submitted by: Konstantin Belousov <kostikbel@gmail.com>

------------------------------------------------------------------------
r256081 | markm | 2013-10-06 10:55:28 +0100 (Sun, 06 Oct 2013) | 12 lines

SNAPSHOT.

Simplify the malloc pools; We only need one for this device.

Simplify the harvest queue.

Marginally improve the entropy pool hashing, making it a bit faster
in the process.

Connect up the hardware "live" source harvesting. This is simplistic
for now, and will need to be made rate-adaptive.

All of the above passes a compile test but needs to be debugged.

------------------------------------------------------------------------
r256042 | markm | 2013-10-04 07:55:06 +0100 (Fri, 04 Oct 2013) | 25 lines

Snapshot. This passes the build test, but has not yet been finished or debugged.

Contains:

* Refactor the hardware RNG CPU instruction sources to feed into
the software mixer. This is unfinished. The actual harvesting needs
to be sorted out. Modified by me (see below).

* Remove 'frac' parameter from random_harvest(). This was never
used and adds extra code for no good reason.

* Remove device write entropy harvesting. This provided a weak
attack vector, was not very good at bootstrapping the device. To
follow will be a replacement explicit reseed knob.

* Separate out all the RANDOM_PURE sources into separate harvest
entities. This adds some secuity in the case where more than one
is present.

* Review all the code and fix anything obviously messy or inconsistent.
Address som review concerns while I'm here, like rename the pseudo-rng
to 'dummy'.

# Submitted by: Arthur Mesh <arthurmesh@gmail.com> (the first item)

------------------------------------------------------------------------
r255319 | markm | 2013-09-06 18:51:52 +0100 (Fri, 06 Sep 2013) | 4 lines

Yarrow wants entropy estimations to be conservative; the usual idea
is that if you are certain you have N bits of entropy, you declare
N/2.

------------------------------------------------------------------------
r255075 | markm | 2013-08-30 18:47:53 +0100 (Fri, 30 Aug 2013) | 4 lines

Remove short-lived idea; thread to harvest (eg) RDRAND enropy into the
usual harvest queues. It was a nifty idea, but too heavyweight.

# Submitted by: Arthur Mesh <arthurmesh@gmail.com>

------------------------------------------------------------------------
r255071 | markm | 2013-08-30 12:42:57 +0100 (Fri, 30 Aug 2013) | 4 lines

Separate out the Software RNG entropy harvesting queue and thread
into its own files.

# Submitted by: Arthur Mesh <arthurmesh@gmail.com>

------------------------------------------------------------------------
r254934 | markm | 2013-08-26 20:07:03 +0100 (Mon, 26 Aug 2013) | 2 lines

Remove the short-lived namei experiment.

------------------------------------------------------------------------
r254928 | markm | 2013-08-26 19:35:21 +0100 (Mon, 26 Aug 2013) | 2 lines

Snapshot; Do some running repairs on entropy harvesting. More needs
to follow.

------------------------------------------------------------------------
r254927 | markm | 2013-08-26 19:29:51 +0100 (Mon, 26 Aug 2013) | 15 lines

Snapshot of current work;

1) Clean up namespace; only use "Yarrow" where it is Yarrow-specific
or close enough to the Yarrow algorithm. For the rest use a neutral
name.

2) Tidy up headers; put private stuff in private places. More could
be done here.

3) Streamline the hashing/encryption; no need for a 256-bit counter;
128 bits will last for long enough.

There are bits of debug code lying around; these will be removed
at a later stage.

------------------------------------------------------------------------
r254784 | markm | 2013-08-24 14:54:56 +0100 (Sat, 24 Aug 2013) | 39 lines

1) example (partially humorous random_adaptor, that I call "EXAMPLE")
* It's not meant to be used in a real system, it's there to show how
the basics of how to create interfaces for random_adaptors. Perhaps
it should belong in a manual page

2) Move probe.c's functionality in to random_adaptors.c
* rename random_ident_hardware() to random_adaptor_choose()

3) Introduce a new way to choose (or select) random_adaptors via tunable
"rngs_want" It's a list of comma separated names of adaptors, ordered
by preferences. I.e.:
rngs_want="yarrow,rdrand"

Such setting would cause yarrow to be preferred to rdrand. If neither of
them are available (or registered), then system will default to
something reasonable (currently yarrow). If yarrow is not present, then
we fall back to the adaptor that's first on the list of registered
adaptors.

4) Introduce a way where RNGs can play a role of entropy source. This is
mostly useful for HW rngs.

The way I envision this is that every HW RNG will use this
functionality by default. Functionality to disable this is also present.
I have an example of how to use this in random_adaptor_example.c (see
modload event, and init function)

5) fix kern.random.adaptors from
kern.random.adaptors: yarrowpanicblock
to
kern.random.adaptors: yarrow,panic,block

6) add kern.random.active_adaptor to indicate currently selected
adaptor:
root@freebsd04:~ # sysctl kern.random.active_adaptor
kern.random.active_adaptor: yarrow

# Submitted by: Arthur Mesh <arthurmesh@gmail.com>

Submitted by: Dag-Erling Smørgrav <des@FreeBSD.org>, Arthur Mesh <arthurmesh@gmail.com>
Reviewed by: des@FreeBSD.org
Approved by: re (delphij)
Approved by: secteam (des,delphij)

- Remove debugging from GENERIC* kernel configurations
- Enable MALLOC_PRODUCTION
- Default dumpdev=NO
- Remove UPDATING entry regarding debugging features
- Bump __FreeBSD_version to 1000500

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r274841

Remove examples of gif_interfaces and gifconfig. These have already been
marked as deprecated in rc.conf(5) manual page but these examples
were still here.

Approved by: re (gjb)

MFH (r270391): don't set rc_debug in /etc/defaults/rc.conf

MFC r274965:

Make it possible to specify flags for autofs daemons in rc.conf.

Sponsored by: The FreeBSD Foundation

MFC r271663:
Fix a typo; master server for iprop service should be singular.

MFC r270782:

Restructure rc.d scripts for kerberos5 daemons:

- Rename $kerberos5_server_enable with $kdc_enable and rename
rc.d/kerberos with rc.d/kdc.

- Rename $kadmin5_server_enable with $kadmind_enable.

- Rename ${kerberos5,kpasswdd}_server with ${kdc,kpasswdd}_program.

- Fix rc.d/{kadmind,kerberos,kpasswdd,kfd} scripts not to change variables
after load_rc_config().

- Add rc.d/ipropd_master and rc.d/ipropd_slave scripts. These are
for iprop-master(8) and iprop-slave(8). Keytab used for iprop service is
defined in ipropd_{master,slave}_keytab (/etc/krb5.keytab by default).

- Add dependency on rc.d/kdc to SERVERS. rc.d/kdc must be invoked as early
as possible before scripts divided by rc.d/SERVERS.

Note that changes to rc.d/{kdc,kpasswdd,kadmind} are backward-compatible
with the old configuration variables:
${kerberos5,kpasswdd,kadmin5}_server{,_enable,_flags}.

MFC r270780:
Fix rc.d/gssd script to define the default values in a standard way.

MFC r270781:
- Add a warning message when an IPv6 address is specified with no prefixlen.
- Use a parameter argument in jls(8) instead of doing grep.

MFC r271424:

- Add $netif_ipexpand_max to specify the upper limit for the number of
addresses generated by an address range specification. The default value
is 2048. This can be increased by setting $netif_ipexpand_max in rc.conf.

PR: 186841

MFC r269392: Add /usr/lib32/compat to the default ld-elf32.so.1 path.

Approved by: re (gjb, delphij)

MFC r270647: Add references to vt(4) and the configuration files in /usr/sha
MFC r270653: Update man-pages to correctly refer to changed pathes and namin
MFC r270657: More man pages that need to know about vt in addition to syscon
MFC r270659: (by pluknet@) Missed comma.
MFC r270660: Back-out the references to vt(4) from this man-page. It appears
MFC r270933: Add references to vt(4) to further man-pages.
MFC r270934: Final patches to the tools used to convert syscons keymaps for
MFC r270935: Add vt(4) support to the console initialisation script, specifi

Second batch of MFCs to add support for Unicode keymaps for use with vt(4).

It contains the following changes:

- Add references to vt(4) to relevant man-pages.
- Update comment in defaults/rc.conf to mention vt
- Update rc.d/syscons to warn about syscons keymaps used under vt.
An attempt is made to identify the vt keymap to load instead.
- Minor changes to the conversion tool based on mail comments on keymaps.

Relnotes: yes

MFC r270096:

Bring in the new automounter, similar to what's provided in most other
UNIX systems, eg. MacOS X and Solaris. It uses Sun-compatible map format,
has proper kernel support, and LDAP integration.

There are still a few outstanding problems; they will be fixed shortly.

Reviewed by: allanjude@, emaste@, kib@, wblock@ (earlier versions)
Phabric: D523
Relnotes: yes
Sponsored by: The FreeBSD Foundation

Fix devfs rules not applied by default for jails.

Fix OpenSSL use-after-free vulnerability.

Fix TCP reassembly vulnerability.

Security: FreeBSD-SA-14:07.devfs
Security: CVE-2014-3001
Security: FreeBSD-SA-14:08.tcp
Security: CVE-2014-3000
Security: FreeBSD-SA-14:09.openssl
Security: CVE-2010-5298

MFC r261931,r261932:

Allow overriding rctl.conf(5) file location for /etc/rc.d/rctl

Approved by: bapt (mentor, implicit)

MFC r260486:
Remove very low default limit of 4 nfsd threads. nfsd's own default is
8 * hw.ncpu, that sounds more appropriate for these SMP/NCQ/... days.

Merge r257694 from head:

Remove remnants of BIND from /etc, since there is no BIND in base now.

Sorry, that would break users running head and BIND from ports, since
ports rely on these scripts. The ports will be fixed soon.

Approved by: re (kib)

MFC r256773:
Enable the automatic creation of a certificate (if one does not exists)
and enable the usage by sendmail if sendmail is enabled.

Reviewed by: gshapiro
Approved by: re (gjb)

MFC r256775,r256776:
Add support for "first boot" rc.d scripts.

Document this new functionality in rc.conf(5) and rc(8).

Bump __FreeBSD_version so that ports can make use of this.

Approved by: re (gjb)

Merge from project branch via main. Uninteresting commits are trimmed.

Refactor of /dev/random device. Main points include:

* Userland seeding is no longer used. This auto-seeds at boot time
on PC/Desktop setups; this may need some tweeking and intelligence
from those folks setting up embedded boxes, but the work is believed
to be minimal.

* An entropy cache is written to /entropy (even during installation)
and the kernel uses this at next boot.

* An entropy file written to /boot/entropy can be loaded by loader(8)

* Hardware sources such as rdrand are fed into Yarrow, and are no
longer available raw.

------------------------------------------------------------------------
r256240 | des | 2013-10-09 21:14:16 +0100 (Wed, 09 Oct 2013) | 4 lines

Add a RANDOM_RWFILE option and hide the entropy cache code behind it.
Rename YARROW_RNG and FORTUNA_RNG to RANDOM_YARROW and RANDOM_FORTUNA.
Add the RANDOM_* options to LINT.

------------------------------------------------------------------------
r256239 | des | 2013-10-09 21:12:59 +0100 (Wed, 09 Oct 2013) | 2 lines

Define RANDOM_PURE_RNDTEST for rndtest(4).

------------------------------------------------------------------------
r256204 | des | 2013-10-09 18:51:38 +0100 (Wed, 09 Oct 2013) | 2 lines

staticize struct random_hardware_source

------------------------------------------------------------------------
r256203 | markm | 2013-10-09 18:50:36 +0100 (Wed, 09 Oct 2013) | 2 lines

Wrap some policy-rich code in 'if NOTYET' until we can thresh out
what it really needs to do.

------------------------------------------------------------------------
r256184 | des | 2013-10-09 10:13:12 +0100 (Wed, 09 Oct 2013) | 2 lines

Re-add /dev/urandom for compatibility purposes.

------------------------------------------------------------------------
r256182 | des | 2013-10-09 10:11:14 +0100 (Wed, 09 Oct 2013) | 3 lines

Add missing include guards and move the existing ones out of the
implementation namespace.

------------------------------------------------------------------------
r256168 | markm | 2013-10-08 23:14:07 +0100 (Tue, 08 Oct 2013) | 10 lines

Fix some just-noticed problems:

o Allow this to work with "nodevice random" by fixing where the
MALLOC pool is defined.

o Fix the explicit reseed code. This was correct as submitted, but
in the project branch doesn't need to set the "seeded" bit as this
is done correctly in the "unblock" function.

o Remove some debug ifdeffing.

o Adjust comments.

------------------------------------------------------------------------
r256159 | markm | 2013-10-08 19:48:11 +0100 (Tue, 08 Oct 2013) | 6 lines

Time to eat crow for me.

I replaced the sx_* locks that Arthur used with regular mutexes;
this turned out the be the wrong thing to do as the locks need to
be sleepable. Revert this folly.

# Submitted by: Arthur Mesh <arthurmesh@gmail.com> (In original diff)

------------------------------------------------------------------------
r256138 | des | 2013-10-08 12:05:26 +0100 (Tue, 08 Oct 2013) | 10 lines

Add YARROW_RNG and FORTUNA_RNG to sys/conf/options.

Add a SYSINIT that forces a reseed during proc0 setup, which happens
fairly late in the boot process.

Add a RANDOM_DEBUG option which enables some debugging printf()s.

Add a new RANDOM_ATTACH entropy source which harvests entropy from the
get_cyclecount() delta across each call to a device attach method.

------------------------------------------------------------------------
r256135 | markm | 2013-10-08 07:54:52 +0100 (Tue, 08 Oct 2013) | 8 lines

Debugging. My attempt at EVENTHANDLER(multiuser) was a failure; use
EVENTHANDLER(mountroot) instead.

This means we can't count on /var being present, so something will
need to be done about harvesting /var/db/entropy/... .

Some policy now needs to be sorted out, and a pre-sync cache needs
to be written, but apart from that we are now ready to go.

Over to review.

------------------------------------------------------------------------
r256094 | markm | 2013-10-06 23:45:02 +0100 (Sun, 06 Oct 2013) | 8 lines

Snapshot.

Looking pretty good; this mostly works now. New code includes:

* Read cached entropy at startup, both from files and from loader(8)
preloaded entropy. Failures are soft, but announced. Untested.

* Use EVENTHANDLER to do above just before we go multiuser. Untested.

------------------------------------------------------------------------
r256088 | markm | 2013-10-06 14:01:42 +0100 (Sun, 06 Oct 2013) | 2 lines

Fix up the man page for random(4). This mainly removes no-longer-relevant
details about HW RNGs, reseeding explicitly and user-supplied
entropy.

------------------------------------------------------------------------
r256087 | markm | 2013-10-06 13:43:42 +0100 (Sun, 06 Oct 2013) | 6 lines

As userland writing to /dev/random is no more, remove the "better
than nothing" bootstrap mode.

Add SWI harvesting to the mix.

My box seeds Yarrow by itself in a few seconds! YMMV; more to follow.

------------------------------------------------------------------------
r256086 | markm | 2013-10-06 13:40:32 +0100 (Sun, 06 Oct 2013) | 11 lines

Debug run. This now works, except that the "live" sources haven't
been tested. With all sources turned on, this unlocks itself in
a couple of seconds! That is no my box, and there is no guarantee
that this will be the case everywhere.

* Cut debug prints.

* Use the same locks/mutexes all the way through.

* Be a tad more conservative about entropy estimates.

------------------------------------------------------------------------
r256084 | markm | 2013-10-06 13:35:29 +0100 (Sun, 06 Oct 2013) | 5 lines

Don't use the "real" assembler mnemonics; older compilers may not
understand them (like when building CURRENT on 9.x).

# Submitted by: Konstantin Belousov <kostikbel@gmail.com>

------------------------------------------------------------------------
r256081 | markm | 2013-10-06 10:55:28 +0100 (Sun, 06 Oct 2013) | 12 lines

SNAPSHOT.

Simplify the malloc pools; We only need one for this device.

Simplify the harvest queue.

Marginally improve the entropy pool hashing, making it a bit faster
in the process.

Connect up the hardware "live" source harvesting. This is simplistic
for now, and will need to be made rate-adaptive.

All of the above passes a compile test but needs to be debugged.

------------------------------------------------------------------------
r256042 | markm | 2013-10-04 07:55:06 +0100 (Fri, 04 Oct 2013) | 25 lines

Snapshot. This passes the build test, but has not yet been finished or debugged.

Contains:

* Refactor the hardware RNG CPU instruction sources to feed into
the software mixer. This is unfinished. The actual harvesting needs
to be sorted out. Modified by me (see below).

* Remove 'frac' parameter from random_harvest(). This was never
used and adds extra code for no good reason.

* Remove device write entropy harvesting. This provided a weak
attack vector, was not very good at bootstrapping the device. To
follow will be a replacement explicit reseed knob.

* Separate out all the RANDOM_PURE sources into separate harvest
entities. This adds some secuity in the case where more than one
is present.

* Review all the code and fix anything obviously messy or inconsistent.
Address som review concerns while I'm here, like rename the pseudo-rng
to 'dummy'.

# Submitted by: Arthur Mesh <arthurmesh@gmail.com> (the first item)

------------------------------------------------------------------------
r255319 | markm | 2013-09-06 18:51:52 +0100 (Fri, 06 Sep 2013) | 4 lines

Yarrow wants entropy estimations to be conservative; the usual idea
is that if you are certain you have N bits of entropy, you declare
N/2.

------------------------------------------------------------------------
r255075 | markm | 2013-08-30 18:47:53 +0100 (Fri, 30 Aug 2013) | 4 lines

Remove short-lived idea; thread to harvest (eg) RDRAND enropy into the
usual harvest queues. It was a nifty idea, but too heavyweight.

# Submitted by: Arthur Mesh <arthurmesh@gmail.com>

------------------------------------------------------------------------
r255071 | markm | 2013-08-30 12:42:57 +0100 (Fri, 30 Aug 2013) | 4 lines

Separate out the Software RNG entropy harvesting queue and thread
into its own files.

# Submitted by: Arthur Mesh <arthurmesh@gmail.com>

------------------------------------------------------------------------
r254934 | markm | 2013-08-26 20:07:03 +0100 (Mon, 26 Aug 2013) | 2 lines

Remove the short-lived namei experiment.

------------------------------------------------------------------------
r254928 | markm | 2013-08-26 19:35:21 +0100 (Mon, 26 Aug 2013) | 2 lines

Snapshot; Do some running repairs on entropy harvesting. More needs
to follow.

------------------------------------------------------------------------
r254927 | markm | 2013-08-26 19:29:51 +0100 (Mon, 26 Aug 2013) | 15 lines

Snapshot of current work;

1) Clean up namespace; only use "Yarrow" where it is Yarrow-specific
or close enough to the Yarrow algorithm. For the rest use a neutral
name.

2) Tidy up headers; put private stuff in private places. More could
be done here.

3) Streamline the hashing/encryption; no need for a 256-bit counter;
128 bits will last for long enough.

There are bits of debug code lying around; these will be removed
at a later stage.

------------------------------------------------------------------------
r254784 | markm | 2013-08-24 14:54:56 +0100 (Sat, 24 Aug 2013) | 39 lines

1) example (partially humorous random_adaptor, that I call "EXAMPLE")
* It's not meant to be used in a real system, it's there to show how
the basics of how to create interfaces for random_adaptors. Perhaps
it should belong in a manual page

2) Move probe.c's functionality in to random_adaptors.c
* rename random_ident_hardware() to random_adaptor_choose()

3) Introduce a new way to choose (or select) random_adaptors via tunable
"rngs_want" It's a list of comma separated names of adaptors, ordered
by preferences. I.e.:
rngs_want="yarrow,rdrand"

Such setting would cause yarrow to be preferred to rdrand. If neither of
them are available (or registered), then system will default to
something reasonable (currently yarrow). If yarrow is not present, then
we fall back to the adaptor that's first on the list of registered
adaptors.

4) Introduce a way where RNGs can play a role of entropy source. This is
mostly useful for HW rngs.

The way I envision this is that every HW RNG will use this
functionality by default. Functionality to disable this is also present.
I have an example of how to use this in random_adaptor_example.c (see
modload event, and init function)

5) fix kern.random.adaptors from
kern.random.adaptors: yarrowpanicblock
to
kern.random.adaptors: yarrow,panic,block

6) add kern.random.active_adaptor to indicate currently selected
adaptor:
root@freebsd04:~ # sysctl kern.random.active_adaptor
kern.random.active_adaptor: yarrow

# Submitted by: Arthur Mesh <arthurmesh@gmail.com>

Submitted by: Dag-Erling Smørgrav <des@FreeBSD.org>, Arthur Mesh <arthurmesh@gmail.com>
Reviewed by: des@FreeBSD.org
Approved by: re (delphij)
Approved by: secteam (des,delphij)

- Remove debugging from GENERIC* kernel configurations
- Enable MALLOC_PRODUCTION
- Default dumpdev=NO
- Remove UPDATING entry regarding debugging features
- Bump __FreeBSD_version to 1000500

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation