Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

Reorganise the entropy device so that high-yield entropy sources
can more easily be used INSTEAD OF the hard-working Yarrow.
The only hardware source used at this point is the one inside
the VIA C3 Nehemiah (Stepping 3 and above) CPU. More sources will
be added in due course. Contributions welcome!

Staticise the random_state array.

Reviewed by: markm

Very large makeover of the /dev/random driver.

o Separate the kernel stuff from the Yarrow algorithm. Yarrow is now
well contained in one source file and one header.

o Replace the Blowfish-based crypto routines with Rijndael-based ones.
(Rijndael is the new AES algorithm). The huge improvement in
Rijndael's key-agility over Blowfish means that this is an
extremely dramatic improvement in speed, and makes a heck of
a difference in its (lack of) CPU load.

o Clean up the sysctl's. At BDE's prompting, I have gone back to
static sysctls.

o Bug fixes. The streamlining of the crypto stuff enabled me to
find and fix some bugs. DES also found a bug in the reseed routine
which is fixed.

o Change the way reseeds clear "used" entropy. Previously, only the
source(s) that caused a reseed were cleared. Now all sources in the
relevant pool(s) are cleared.

o Code tidy-up. Mostly to make it (nearly) 80-column compliant.

Make a big improvement to entropy-harvesting speed by not having any
locks (only atomic assigns) in the harvest ringbuffer.

Remove NOBLOCKRANDOM as a compile-time option. Instead, provide
exactly the same functionality via a sysctl, making this feature
a run-time option.

The default is 1(ON), which means that /dev/random device will
NOT block at startup.

setting kern.random.sys.seeded to 0(OFF) will cause /dev/random
to block until the next reseed, at which stage the sysctl
will be changed back to 1(ON).

While I'm here, clean up the sysctls, and make them dynamic.
Reviewed by: des
Tested on Alpha by: obrien

Major speedup to /dev/random and the kernel thread that reseeds it.

There is no more TAILQ fifo to harvest the entropy; instead, there
is a circular buffer of constant size (changeable by macro) that
pretty dramatically improves the speed and fixes potential slowdowns-
by-locking.

Also gone are a slew of malloc(9) and free(9) calls; all harvesting
buffers are static.

All-in-all, this is a good performance improvement.

Thanks-to: msmith for the circular buffer concept-code.

Greatly improve the boot-up unblocking time of the entropy device.

More comment changing. Keep documentation in one place.

Correct a comment. This represents a very minor policy change of my
intentions with this code.

Stop explicitly using nanotime(9) and use the new get_cyclecounter(9)
call instead.

This makes a pretty dramatic difference to the amount of work that
the harvester needs to do - it is much friendlier on the system.
(80386 and 80486 class machines will notice little, as the new
get_cyclecounter() call is a wrapper round nanotime(9) for them).

After some complaints about the dir names, the random device is
now in dirs called sys/*/random/ instead of sys/*/randomdev/*.

Introduce blocking, but only at startup; the random device will
block until the first reseed happens to prevent clients from
using untrustworthy output.

Provide a read_random() call for the rest of the kernel so that
the entropy device does not need to be present. This means that
things like IPX no longer need to have "device random" hardcoded
into thir kernel config. The downside is that read_random() will
provide very poor output until the entropy device is loaded and
reseeded. It is recommended that developers do NOT use the
read_random() call; instead, they should use arc4random() which
internally uses read_random().

Clean up the mutex and locking code a bit; this makes it possible
to unload the module again.

Move the prototypes for random_set_wakeup* from yarrow.c to yarrow.h so that
both yarrow.c and harvest.c can use them.

Approved by: markm

Large upgrade to the entropy device; mainly inspired by feedback
from many folk.

o The reseed process is now a kthread. With SMPng, kthreads are
pre-emptive, so the annoying jerkiness of the mouse is gone.

o The data structures are protected by mutexes now, not splfoo()/splx().

o The cryptographic routines are broken out into their own subroutines.
this facilitates review, and possible replacement if that is ever
found necessary.

Thanks to: kris, green, peter, jasone, grog, jhb
Forgotten to thank: You know who you are; no offense intended.

o Fix a horrible bug where small reads (< 8 bytes) would return the
wrong bytes.

o Improve the public interface; use void* instead of char* or u_int64_t
to pass arbitrary data around.
Submitted by: kris ("horrible bug")

Clean this up with some BDE-inspired fixes.

o Make the comments KNF-compliant.
o Use nanotime instead of getnanotime; the manpage lies about the
kern.timecounter.method - it has been removed.
o Fix the ENTROPYSOURCE const permanently.
o Make variable names more consistent.
o Make function prototypes more consistent.

Some more needs to be done; to follow.

Yarrow tweaks; separate the fast and slow reseed tasks so that they don't
stomp on each other; provide constant names (as enums) for the harvester
to use (makes it more self-documenting).

Add entropy gathering code. This will work whether the module is
compiled in or loaded.

New machine-independant /dev/random driver.

This is work-in-progress, and the entropy-gathering routines are not
yet present. As such, this should be viewed as a pretty reasonable
PRNG with _ABSOLUTELY_NO_ security!!

Entropy gathering will be the subject of ongoing work.

This is written as a module, and as such is unloadable, but there is
no refcounting done. I would like to use something like device_busy(9)
to achieve this (eventually).

Lots of useful ideas from: bde, phk, Jeroen van Gelderen

Reviewed by: dfr