MFC r336244, r336246-r336247: Standardize boot arg parsing

r336244:
Create helper functions for parsing boot args.

boot_parse_arg to parse a single arg
boot_parse_cmdline to parse a command line string
boot_parse_args to parse all the args in a vector
boot_howto_to_env Convert howto bits to env vars
boot_env_to_howto Return howto mask mased on what's set in the environment.

All these routines return an int that's the bitmask of the args
translated to RB_* flags. As a special case, the 'S' flag sets the
comconsole_speed env var. Any arg that looks like a=b will set the env
key 'a' to value 'b'. If =b is omitted, 'a' is set to '1'. This
should help us reduce the number of redundant copies of these routines
in the tree. It should also give a more uniform experience between
platforms.

Also, invent a new flag RB_PROBE that's set when 'P' is parsed. On
x86 + BIOS, this means 'probe for the keyboard, and if it's not there
set both RB_MULTIPLE and RB_SERIAL (which means show the output on
both video and serial consoles, but make serial primary). Others it
may be some similar concept of probing, but it's loader dependent
what, exactly, it means.

These routines are suitable for /boot/loader and/or the kernel,
though they may not be suitable for the tightly hand-rolled-for-space
environments like boot2.

r336246:
Eliminate boot loader copies of boot arg parsing.

Eliminate 4 of the copies of the arg parsing in /boot/laoder
by using boot_parse_cmdline.

r336247:
Transition to boot_env_to_howto and boot_howto_to_env in the boot
loader.

MFC r334882, r334884-r334885: loader(8) boot flag <-> environment fixes

r334882:
stand: Consolidate checking for boot flags driven by environment vars

e.g. boot_mute, boot_single, boot_verbose, and friends; we checked for these
in multiple places, consolidate into common/ and allow a setting of "NO" for
any of these to turn them off. This allows systems with multiple
loader.conf(5) or loader.conf(5) overlay systems to easily turn off
variables in later processed files by setting it to NO.

Reported by: Nick Wolff @ iXsystems
Reviewed by: imp

r334884:
stand: Fix build after r334882

Not sure how this was not caught in Universe.

r334885:
stand: One more trivial consolidation (setting environment from howto)

MFC r334432:

Fix build of stand with base gcc

* Make autoboot() a static function in stand/common/boot.c, so it does
not shadow local variables in gptboot.c and zfsboot.c.
* Remove -Winline from the Makefiles for gptboot, gptzfsboot and
zfsboot, as gcc will always fail to inline some functions, and there
is nothing we can do about it.
* For gcc <= 4.2.1, silence -Wuninitialized for isoboot, as it produces
a false positive warning.
* Remove deprecated and unnecessary -mcpu=i386 flag from stand/defs.mk,
as there is already a -march=i386 flag further in the file.

Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D15628

MFC Loader Fixes Final: r327612,r327703,r327704,r327878,r327879,r327881,
r328007,r328029,r328030,r328031,r328061,r328156,r328169,r328288,r328289,
r328290,r328291,r328292,r328411,r328536,r328603,r328614,r328642,r328769,
r328779,r328780,r328781,r328782,r328783,r328806,r328808,r328826,r328835,
r328911,r328986,r328987,r328990,r328999,r329000,r329019,r329050,r329054,
r329060

r327612: Invent new #defines for the biospci_{read,write}_config function

r327703: Define __dmadat after #include'ing ufsread.c.

r327704: Fix printf missing format variables warnings.

r327878: Add GUID for UEFI boot manager variables.

r327879: Report the boot order and where we are in that boot order.

r327881: Allow this file to be included

r328007: Fix booting on some arm64 systems after r327879 by fixing the call
to utf8_to_ucs2

r328029: When returning an error and freeing allocated memory from
ucs2_to_utf8, NULL the return pointer.

r328030: Check the return value from utf8_to_ucs2 instead of whether or not
uv is NULL.

r328031: Need to free uv after we're done using it.

r328061: utf8_to_ucs2() should check for malloc failure

r328156: stand: Move sections around to fix stand/ build with ld.lld on
armv7

r328169: Remove extra copy of bootinfo.c. It's a bit rotted copy of the one
in efi/loader.

r328288: Fix some resource leaks.

r328289: Don't leak memory when displaying help.

r328290: On malloc failure, be sure to close the include file that triggered
it.

r328291: getenv does not return tainted data in the boot loader. Attempt to
clue Coverity into that fact.

r328292: There's no tainted data here, tag it as such to avoid false
positives.

r328411: loader.efi: add missing EFI GUIDs

r328536: loader: support for mixed-endianness ELF/loader and POWER8

r328603: Add missing non-POWERPC case to give the scr value something
non-zero.

r328614: Move libstand.3 to libsa.3. Update libsa.3 to include functions

r328642: Break out the interpreters (simple and forth) w/o ifdefs.

r328769: Centralize several variables.

r328779: Retire pnp.4th and the code needed only for 4th words used here.

r328780: These 4th words were an attempt to allow integration into the boot
loader scripts. However, that path won't be taken after all it
seems.

r328781: Remove pcibios forth support.

r328782: Now that we no longer conditionally compile some files outside of
ficl

r328783: Invent new LDR_INTERP for the loader interpreter to use.

r328806: We need more heap space to properly load newer powerpc kernels.

r328808: Implement strcoll as strcmp.

r328826: Make cross-endian loader changes apply only to powerpc

r328835: Fix regression introduced in r328806, preventing boot on many
platforms.

r328911: Ignore relocation tables for non-memory-resident sections.

r328986: Fix relative location of USB sources after recent move.

r328987: A more definitions to kernel emulation shim in order to build
stand/usb.

r328990: Move the stand/usb test loader into its own directory.

r328999: Fix indentation to FreeBSD standard for interp files

r329000: Move simple interpreter 'perform' into interp.c and call it

r329019: Move to tabs for indentation and to 8-space notches, per style(9).

r329050: Fix build of userboot.so

r329054: Set script.lang in the environment to either 'forth' or 'simple' to
reflect what scripting language was compiled into the loader.

r329060: loader: fix endianness conversion

PR: 225323

MFC r325834,r325997,326502: Move sys/boot to stand/

This is effectively a direct commit to stable/11, due to differences between
stable/11 and head. Changes to DTS in sys/boot/fdt/dts were often
accompanied by kernel changes. Many of these were also risc-v updates that
likely had many more dependencies to MFC.

Because of this, sys/boot/fdt/dts remains as-is while everything else in
sys/boot relocates to stand/.

r325834: Move sys/boot to stand. Fix all references to new location

r325997: Remove empty directories.

r326502: Document the sys/boot -> stand move in hier.7 and the top-level README.

MFC r303555,r303556,r303936,r303962,r304317,r304532,r305026,r305107,r305132,
r305178,r305353,r305814,r306159,r306380,r306504: Loader fixes, 2016q3

r303555: bcache should support reads shorter than sector size

r303556: Improve boot loader quote parsing

r303936: Add kernel environment variables under smbios.system

r303962: Add the missing space between .asciz directive and opening quote
for some lines with #ifdef BTXLDR_VERBOSE/#endif

r304317: boot1.efi Free() should check for NULL to provide consistent
behavior

r304532: Replace sprintf -> snprintf for command_errbuf provisioned from
dynamic content.

r305026: Emulate efi_cons_poll when WaitForKey is not available

r305107: Create a hook 'post-initialize' for people that want to define
something to read in .conf files after all other .conf files for the purpose
of overriding.

r305132: Remove accidentally committed stray comment.

r305178: bd_int13probe() should check extended info if sector info is bad

r305353: Don't use -N to set the OMAGIC with data and text writeable and
data not page aligned.

r305814: ufsread: Do not cast struct direct from void *

r306159: Consistently declare getsecs(void) with proper return type and void
when no arguments are present.

r306380: loader command interpreter should reset command_errmsg

r306504: Fix a cluster of bugs in list EFI environment variables

PR: 204602, 211958, 211958
Relnotes: yes ("Improve boot loader quote parsing")

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

zfs boot: try to set vfs.root.mountfrom from currdev as a fallback

This way with the new zfsloader there is no need to explicitly set zfs
root filesystem either via vfs.root.mountfrom or fstab.
It should be automatically picked up from currdev which is by default
is set from bootfs.

Tested by: Florian Wagner <florian@wagner-flo.net> (x86)
MFC after: 1 month

Introduce a new option (BOOT_PROMPT_123) that lets enter the boot prompt
only when typing the sequence "123" (opposite to the standard 'push any
button' approach).
That results useful when using serial lines sending garbage and leading
to unwilling boot prompt appearence.

Obtained from: Sandvine Incorporated
Reviewed by: emaste, jhb
Sponsored by: Sandvine Incorporated
MFC: 1 week

sys/boot/common.c
=================
Extend the loader to parse the root file system mount options in /etc/fstab,
and set a new loader variable vfs.root.mountfrom.options with these options.
The root mount options must be a comma-delimited string, as specified in
/etc/fstab.
Only set the vfs.root.mountfrom.options variable if it has not been
set in the environment.

sys/kern/vfs_mount.c
====================
When mounting the root file system, pass the mount options
specified in vfs.root.mountfrom.options, but filter out "rw" and "noro",
since the initial mount of the root file system must be done as "ro".
While we are here, try to add a few hints to the mountroot prompt
to give users and idea what might of gone wrong during mounting
of the root file system.

Reviewed by: jhb (an earlier patch)

Extend `autoboot_delay' handling - is this variable is set to -1 don't allow
user to interrupt autoboot process at all. Currently, even when
`autoboot_delay' is set to 0, loader(8) still allows autoboot process to be
interrupted by pressing any key on the console when the loader reads kernel
and modules from the disk. In some cases (i.e. untrusted environment) such
behaviour is highly indesirable and user should not be allowed to interfere
with the autoboot process at all.

Sponsored by: PBXpress Inc.
MFC after: 3 days

Whitespace only: kill trailing spaces.

Use __FBSDID().
Also some minor style cleanups.

Implement the long-awaited module->file cache database. A userland
tool (kldxref(8)) keeps a cache of what modules and versions are inside
what .ko files. I have tested this on both Alpha and i386.

Submitted by: bp

The default kernel filename is "kernel" again, not "kernel.ko".

Submitted by: mckusick

Don't indirect through a null pointer if we run out of kernel filenames.

Start countdown only _after_ the kernel was loaded. Not very fair otherwise.

Fix autoboot. Now autoboot *always* show the correct kernel name. It
gets the name from the environment variable kernelname, which is set
when a kernel is loaded. For this reason, autoboot will _first_ try
to load a kernel, and only proceed with the wait prompt after that
succeeds. If it fails, it will abort immediately.

While I understand some may think this behavior undesirable, I think
it is, overall, the best thing to do, even if we do not consider the
aesthetic issue. Notice that anyone using the default loader.rc
already has the kernel loaded before autoboot.

On unload, unset kernelname.

Separate the code that tries to load a kernel from the list of options
to the function loadakernel(). It is used by both boot() and
autoboot().

With the committed changes to the loaders modules path code, the kernel
path does not need to be hard coded.

Fix the kernel default load from "/boot/modules" to "/boot/kernel".

Reported by: dcs

The kernel is now known as `kernel.ko' and it and its matching modules
live in ``/boot/kernel/''.

Cleanup warnings. Most of these are signed/unsigned warnings, as well as
some added const's.

Delay calling the device cleanup routines until the absolute last
moment. We were cleaning up after PXE too early and the module
dependancy code would not be able to load any files if it needed
too.

The loader was written so that if /kernel was missing /kernel.old would
be booted. Due to a bug, this wasn't happening.

There is still a lesser bug in that the loader decides which file to boot
after the 10sec count down. This means the bootfile listed in the count
down in is wrong in the case where the loader will boot /kernel.old.

Update loader logic to distinguish modules vs. files.
Add support for module metadata. The old way of dependancy
handling will be supported for a while.

Reviewed by: peter

Add a cleanup function. This is needed for PXE where you should
shutdown the UNDI and unload the stack.

$Id$ -> $FreeBSD$

Walk around the end of all the silly guessing of device types and unit
numbers that we have been doing in the past, and read /etc/fstab off the
proposed root filesystem to determine the actual device name and vfs
type for the root filesystem. These are then exported to the kernel
via the environment variable vfs.root.mountfrom.

The boot countdown timer says it will boot in 1 seconds,
which is grammatically incorrect.

PR: 12628
Submitted-By: R. Matthew Emerson <rme@nightfly.apk.net>

Typo: "can't boot 'boot'" should be "can't boot 'kernel'"

Always output a linefeed when we've decided whether to boot
or display a loader prompt; either we've said
"Booting [%s]..." or we've received a non-line-feed character
and need a '\n' anyway.

Standardise on ';' as a component separator; it seems to be a little more
common than ','.

* Extend the memory available for the heap from 256k to 512k.
* Embed the stack into the bss section for loader and netboot. This
is required for netboot since otherwise the stack would be inside our
heap.
* Install loader and netboot in /boot by default.
* Fix getbootfile so that it searches for a ',' instead of a ';'
when terminating the filename.

- Add a new command 'lsdev' to list devices which might be likely to host
filesystems.
- New 'help' command and data in the help.* files (not yet installed),
provides topic and subtopic help, indexes, etc.
- Don't crash if the user tries to set an invalid console. Be helpful
instead.
- Expand tabs (badly) on the i386 video console.
- Some minor cosmetic changes.

Tweak the output one more time again. The kernel or module pathname
is useful, and usually fits all on one line with the load sizes.

Cosmetic: After the autoboot timeout, print a \n.

- VERBOSE_LS is obsolete, as the heap is much better behaved now.
- Don't whine about nodes we can't stat(); these are usually
symlinks that lead out of the filesystem.
- Autoboot is now controlled by $autoboot_delay, which is a value
in seconds or NO to disable autoboot.
- Don't autoboot at the end of boot.conf if we have already tried.
- Add a 'read' command to complement 'echo'. Both are still hidden.
- Improve the 'source' command/function so that it is possible to
source scripts off removable media. The entire script is read and
saved before beginning execution. Script lines beginning with
'@' will not be echoed when being executed. Script execution will
normally terminate at the first error, however if the script line
begins with '-' this behaviour is overriden for that command.

boot.c
Increase the robustness of the "is it time to boot yet" test;
if the time skipped the "when" time, we would miss it.
Don't spin in an endless loop if we don't find the first possible
kernel suggested. When we run out, don't try to load an empty
kernel name.

load_aout.c
printf format warnings

Use the variable with the path in it for the error message.

Bootloader update.

- Implement a new copyin/readin interface for loading modules.
This allows the module loaders to become MI, reducing code duplication.
- Simplify the search for an image activator for the loaded kernel.
- Use the common module management code for all module metadata.
- Add an 'unload' command that throws everything away.
- Move the a.out module loader to MI code, add support for a.out
kld modules.

Submitted by: Alpha changes fixed by Doug Rabson <dfr@freebsd.org>

This commit was generated by cvs2svn to compensate for changes in r38465,
which included commits to RCS files with non-trunk default branches.

This is the new unified bootstrap, sometimes known previously as the
'three-stage' bootstrap.
There are a number of caveats with the code in its current state:
- The i386 bootstrap only supports booting from a floppy.
- The kernel and kld do not yet know how to deal with the extended
information and module summary passed in.
- PnP-based autodetection and demand loading of modules is not implemented.
- i386 ELF kernel loading is not ready yet.
- The i386 bootstrap is loaded via an ugly blockmap.

On the alpha, both net- and disk-booting (SRM console machines only) is
supported. No blockmaps are used by this code.

Obtained from: Parts from the NetBSD/i386 standalone bootstrap.