MFC r357845: efiloader: don't execute hooks when setting currdev/loaddev

We still need to set the hooks to prevent improper manipulations thereafter
at the loader prompt, but as it is we're actively preventing loaddev from
being set correctly in some circumstances (ZFS in particular) and doing more
work than needed with currdev -- that hook in particular validates it as a
correct device, which we can assume isn't needed in this context.

MFC r343911, r344238-r344241, r344247, r344254-r344255, r344260, r344268,
r344335, r344839, r345066, r345330

r343911:
Allow reading the UEFI variable size

When loading bigger variables form UEFI it is necessary to know their
size beforehand, so that an appropriate amount of memory can be
allocated. The easiest way to do this is to try to read the variable
with buffer size equal 0, expecting EFI_BUFFER_TOO_SMALL error to be
returned. Allow such possible approach in efi_getenv routine.

Extracted from a bigger patch as suggested by imp.

r344238:
Restore loader(8)'s ability for lsdev to show partitions within a bsd slice.

I'm pretty sure this used to work at one time, perhaps long ago. It has
been failing recently because if you call disk_open() with dev->d_partition
set to -1 when d_slice refers to a bsd slice, it assumes you want it to
open the first partition within that slice. When you then pass that open
dev instance to ptable_open(), it tries to read the start of the 'a'
partition and decides there is no recognizable partition type there.

This restores the old functionality by resetting d_offset to the start
of the raw slice after disk_open() returns. For good measure, d_partition
is also set back to -1, although that doesn't currently affect anything.

I would have preferred to make disk_open() avoid such rude assumptions and
if you ask for partition -1 you get the raw slice. But the commit history
shows that someone already did that once (r239058), and had to revert it
(r239232), so I didn't even try to go down that road.

r344239:
Use a couple local variables to avoid repetitive long expressions that
cause line-wrapping.

r344240:
Make lsdev -v output line up in neat columns by using a fixed width for
the size field and a tab between the partition type and the size.

Changes this

disk devices:
disk0 (MMC)
disk0s1: DOS/Windows 49MB
disk0s2: FreeBSD 14GB
disk0s2a: FreeBSD UFS 14GB
disk0s2b: Unknown 2048KB
disk0s2d: FreeBSD UFS 2040KB

to this

disk devices:
disk0 (MMC)
disk0s1: DOS/Windows 49MB
disk0s2: FreeBSD 14GB
disk0s2a: FreeBSD UFS 14GB
disk0s2b: Unknown 2048KB
disk0s2d: FreeBSD UFS 2040KB

r344241:
Garbage collection no-longer-used constant.

r344247:
Make uboot_devdesc properly alias disk_devdesc, so that parsing the u-boot
loaderdev variable works correctly.

The uboot_devdesc struct is variously cast back and forth between
uboot_devdesc and disk_devdesc as pointers are handed off through various
opaque interfaces. uboot_devdesc attempted to mimic the layout of
disk_devdesc by having a devdesc struct, followed by a union of some
device-specific stuff that included a struct that contains the same fields
as a disk_devdesc. However, one of those fields inside the struct is 64-bit
which causes the entire union to be 64-bit aligned -- 32 bits of padding
is added between the struct devdesc and the union, so the whole mess ends
up NOT properly mimicking a disk_devdesc after all. (In disk_devdesc there
is also 32 bits of padding, but it shows up immediately before the d_offset
field, rather than before the whole collection of d_* fields.)

This fixes the problem by using an anonymous union to overlay the devdesc
field uboot network devices need with the disk_devdesc that uboot storage
devices need. This is a different solution than the one contributed with
the PR (so if anything goes wrong, the blame goes to me), but 95% of the
credit for this fix goes to Pawel Worach and Manuel Stuhn who analyzed the
problem and proposed a fix.

r344254:
Use DEV_TYP_NONE instead of -1 to indicate no device was specified.

DEV_TYP_NONE has a value of zero, which makes more sense since the device
type is a bunch of bits describing the device, crammed into an int.

r344255:
Fix more places to use DEV_TYP_NONE instead of -1 to indicate 'no device'.

r344260:
Allow the u-boot loaderdev env var to be formatted in the "usual" loader(8)
way: device<unit>[s|p]<slice><partition>. E.g., disk0s2a or disk3p12.
The code first tries to parse the variable in this format using the
standard disk_parsedev(). If that fails, it falls back to parsing the
legacy format that has been supported by ubldr for years.

In addition to 'disk', all the valid uboot device names can also be used:
mmc, sata, usb, ide, scsi. The 'disk' device serves as an alias for all
those types and will match the Nth storage-type device found (where N is
the unit number).

r344268:
loader: ptable_close() should check its argument

If the passed in table is NULL, just return.

r344335:
Fix the handling of legacy-format devices in the u-boot loaderdev variable.
When I added support for the standard loader(8) disk0s2a: type formats,
the parsing of legacy format was broken because it also contains a colon,
but it comes before the slice and partition. That would cause disk_parsedev()
to return success with the slice and partition set to wildcard values.

This change examines the string first, and if it contains spaces, dots, or
a colon at any position other than the end, it must be a legacy-format
string and we don't even try to use disk_parsedev() on it.

r344839:
Add retry loop around GetMemoryMap call to fix fragmentation bug

The call to BS->AllocatePages can cause the memory map to become framented,
causing BS->GetMemoryMap to return EFI_BUFFER_TOO_SMALL more than once. For
example this can happen on the MinnowBoard Turbot, causing the boot to stop
with an error. Avoid this by calling GetMemoryMap in a loop.

r345066:
stand: Improve some debugging experience

Some of these files using <FOO>_DEBUG defined a DEBUG() macro to serve as a
debug-printf. -DDEBUG is useful to enable some debugging output across
multiple ELF/common parts, so switch the DEBUG-as-printf macros over to
something more like DPRINTF that is more commonly used for this kind of
thing and less likely to conflict.

userboot/elf64_freebsd debugging also assumed %llx for uint64; use PRIx64
instead.

r345330:
loader: fix loading of kernels with . in path

The loader indended to search the kernel file name (only) for . but
instead searched the entire path, so paths like
"boot/test.elfv2/kernel" would not work.

PR: 233097

MFC r341101, r341231, r341276, r341329, r341433, r341780, r342054-r342055,
r342721, r342742, r342840, r343008, r343225

r341101:
powerpcspe: Don't crash the loader on ubldr with SPE instructions.

-msoft-float seems to be insufficient for disabling the SPE on powerpcspe.
Force it off with -mno-spe as well. This prevents a crash in ubldr on
powerpcspe.

r341231:
loader: command_bcache() should print unsigned values

All bcache counters are unsigned.

r341276:
When handling CMD_CRIT error set command_errmsg to NULL after we dump it out,
so that it does not result in error message printed twice.

OK load doodoo
can't find 'doodoo'
can't find 'doodoo'
OK

r341329:
loader.efi: fix EFI getchar() for multiple consoles

This fix is ported from illumos (issue #9970), the analysis and initial
implementation was done by John Levon.

See also: https://www.illumos.org/issues/9970

Currently, efi_cons_getchar() will wait for a key. While this seems to make
sense, the implementation of getchar() in common/console.c will loop across
getchar() for all consoles without doing ischar() first.

This means that if we've configured multiple consoles, we can't input into
the serial, as getchar() will be sat waiting for input only from efi_console.c

This patch does implement a bit more generic key buffer to support
translation of input keys, and we use generic efi_readkey() to reduce
duplication from calls from getchar() and poll().

r341433:
Move inclusion of src.opts.mk later.

src.opts.mk includes bsd.own.mk. This in turn defines CTFCONVERT_CMD
depending on the MK_CTF value. We then set MK_CTF to no, which has no
real effect. The solution is to set all the MK_foo values before
including src.opts.mk.

This should stop the cdboot binary from exploding in size for releases
built WITH_CTF=yes in src.conf.

r341780:
powerpc/ubldr: Teach powerpc's ubldr to boot 64-bit kernels

This is just a copy of powerpc/ofw's ppc64_elf_freebsd.c modified to fit
ubldr's boot format.

r342054:
Print an error message in efi_main.c if we can't allocate memory for the heap

With the default Qemu parameters, only 128MB RAM gets given to a VM. This causes
the loader to be unable to allocate the 64MB it needs for the heap. This change
makes the cause of the error more obvious.

r342055:
Cast error message in efi_main.c to CHAR16* to avoid build error

r342721:
loader.efi: update memmap command to recognize new attributes

Also move memory type to string translation to libefi for later use.

r342742:
loader.efi: efi variable rework and lsefi command added

This update does add diag and debug capabilities to interpret the efi
variables, configuration and protocols (lsefi).

The side effect is that we add/update bunch of related headers.

r342840:
Create MK_LOADER_VERBOSE and connect it to ELF_VERBOSE in the loader
code.

r343008:
Add Dell Chromebook to the list of devices with E820 extmem quirk enabled

Just like for Acer C270 chromebook the E820 extmem workaround is required for
FreeBSD to boot on Dell chromebook.

r343225:
Unbreak mip64 build after r328437

Add exit and getchar functions to beri/boot2 code. They are required by
panic_action functin introduced in r328437

PR: 18498, 204916

MFC r338262, r339334, r339796, r340240, r340857, r340917, r341007

r338262:
stand: fdt: Drop some write-only assignments/variables and leaked bits

Generally straightforward enough; a copy of argv[1] was being made in
command_fdt_internal, solely used for a comparison within the
handler-search, then promptly leaked.

r339334:
loader.efi: add poweroff command

Add poweroff command to make life a bit easier.

r339796:
Simplify the EFI delay() function by calling BS->Stall()

r340240:
loader: ptable_open() check for ptable_cd9660read result is wrong

The ptable_*read() functions return NULL on read errors (and partition table
closed as an side effect). The ptable_open must check the return value and
act properly.

r340857:
Nuke out buffer overflow safety marker code, it duplicates similar code in
the malloc()/free() as well as having potential of softening the handling
in case error is detected down to a mere warning as compared to hard panic
in free().

r340917:
Update pxeboot(8) manual page to reflect the next-server change in the ISC DHCP v3 server.

r341007:
Bump the date of pxeboot(8) manual page for r340917.

PR: 123484, 232483

MFC Loader Interpreter Coexistance: r337806, r337868, r337914, r337927,
r337989, r338052, r338064, r338407, r338418, r338474

r337806:
Create a loader for each interpreter for x86 BIOS and all EFI

Create loader_{4th,lua,simp}{,.efi}. All of these are installed by
default. Create LOADER_DEFAULT_INTERP to specify the default
interpreter when no other is specified. LOADER_INTERP is the current
interpreter language building. Turn building of lua on by default to
match 4th. simploader is a simplified loader build w/o any interpreter
language (but with a simple loader). This is the historic behavir you
got with WITHOUT_FORTH. Make a hard link to the default loader. This
has to be a hard link rather than the more desirable soft link because
older zfsboot blocks don't support symlinks.

r337868:
stand: Use -Oz/-Os for all loader/stand builds.

While we're not super size constrained, the x86 BIOS /boot/loader has
to be less than about 520k-530k to be reliable. The LUA loader is at
this size today. -Oz saves 15-20% on the size, keeping us safely small
enough (comparable to where we were with the 4th loader). This will
also help with sjg's work on bringing in bearssl, though we may again
be looking for space in the LUA loader.

Size table for clang 6.0.0:
default -O1 -Os -Oz
4th 442368 417792 389120 376832
lua 524288 479232 446464 430080

r337914:
Install links for loader.efi.

r337927:
Add ashldi3 and ashrdi3 to mips.

Now that we're using -Os, mips needs these routines.

r337989:
Copy the boot loader from the new location for the co-existing
loaders.

r338052:
libsa: Add lshrdi3.c for powerpc* and mips

r338064:
Turn back the clock just a little: make userboot.so always be 4th

Turns out there was a hidden dependency we hasn't counted upon. The
host load /boot/userboot.so to boot the VMs it runs. This means that
the change to lua meant suddently that nobody could run their older
VMs because LUA wasn't in 10.0, last month's HardenedBSD, 11.2 or
whatever. Even more than for the /boot/loader* binaries, we need a
good coexistance strategy for this. While that's being designed and
implemented, drop back to always 4th for userboot.so. This will fail
safe in all but the most extreme environments (but lua-only hacks
to .lua files won't be processes in VMs until we fix it).

r338407:
lualoader: Print error messages from command failures at the prompt

Previously lualoader would remain silent, rather than printing
command_errmsg or noting that a command had failed or was not found.

r338418:
userboot: handle guest interpreter mismatches more intelligently

The switch to lualoader creates a problem with userboot: the host is
inclined to build userboot with Lua, but the host userboot's interpreter
must match what's available on the guest. For almost all FreeBSD guests in
the wild, Lua is not yet available and a Lua-based userboot will fail.

This revision updates userboot protocol to version 5, which adds a
swap_interpreter callback to request a different interpreter, and tries to
determine the proper interpreter to be used based on how the guest
/boot/loader is compiled. This is still a bit of a guess, but it's likely
the best possible guess we can make in order to get it right. The
interpreter is now embedded in the resulting executable, so we can open
/boot/loader on the guest and hunt that down to derive the interpreter it
was built with.

Using -l with bhyveload will not allow an intepreter swap, even if the
loader specified happens to be a userboot with the wrong interpreter. We'll
simply complain about the mismatch and bail out.

For legacy guests without the interpreter marker, we assume they're 4th.
For new guests with the interpreter marker, we'll read it and swap over
to the proper interpreter if it doesn't match what the userboot we're using
was compiled with.

Both flavors of userboot are installed by default, userboot_4th.so and
userboot_lua.so. This fixes the build WITHOUT_FORTH as a coincidence, which
was broken by userboot being forced to 4th.

r338474:
Be a little conservative about when to force size optimizations.

Reports have come in that there's issue with powerpc and sparc64 since
we've switched to using -Oz / -Os. We don't strictly need them for
!x86, so be conservative about when we enable them.

MFC r336837: Be more conservative about setting hw.uart.console

Note when we've found a 8250 PNP node. Only try to set hw.uart.console
if we see one (otherwise ignore serial hints). The 8250 is the only
one known to have I/O ports, so limit the guessing to when we've
positively seen one. And limit this to x86 since that's the only
platform where we have I/O ports. Otherwise, we'd set the serial port
to something crazy for the platform and fall off the cliff early in
boot.

MFC r337131, r337231: stand: efipart fixes

r337131:
loader.efi: efipart_inithandles() failure should not prevent devsw initialization

The efipart_inithandles() can return error code when there are no block devices,
however, we still should walk and initialize devsw.

r337231:
loader.efi: clean up misleading noise from missing block devices

If there are no block devices, there is no need to printout
error (ENOENT).

In case of netboot, our image path has no block device, no need to make
noise about it.

MFC r338337: Fix lualoader on arm64

Lua has a few places where it allocates a large buffer on the stack. This
is normally fine, except there are a few places where there can be multiple
frames with this buffer. This can cause a stack overflow on some arm64 SoCs.

Fix this by allocating our own stack in loader.efi large enough for these
objects. The required size has been found by tracing how the stack pointer
changes in a virtual machine and found to be no larger than 50kB. A
larger stack is allocated to reduce the likelihood of overflow from future
changes.

MFC r336535: Hoist EFI_TARGET and SOURCE_DATE_EPOCH up into efi/Makefile.inc

MFC r336463: If rootdev is set, use it as currdev.

Setting rootdev in the enviornment should specify things
completely. If it is set, then have it override everything else.

MFC r336264, r336270-r336272, r336464, r336627: UEFI Console Improvements

r336264:
Define ADR subtype of ACPI type for a device path.

r336270:
uefi stand: Guess the console better

For server machines, ComOut is set to the set of devices that the efi
console suppots. Parse it to see if we have serial, video or both.
Make that take precidence over the command line args. boot1.efi parses
them, but loader.efi doesn't. It's not clear where to read boot.conf
from, so we don't do that. The command line args can still be set via
efibootmgr, which is more inline with the UEFI boot manager to replace
that. These args are typically used only to set serial vs video and
the com speed line. We can infer that from ComOut, so do so.
Remember the com speed and hw.uart.console to match.

r336271:
Add reporting of whether or not a keyboard is detected. In addition,
note that r336270's commit message was slightly incorrect. It changed
the default setting of the console to honor the ConOut
variable. Overrides via the command line are still possible, and we
use the devices in ConOut to set the proper console. If, for example,
serial cosnole is specified, we'll set console to "efi" if ConOut has
a serial port list and to either "efi comconsole" or "comconsole efi"
if not depending on whether -D or -D -h was specified.

r336272:
Minor adjustments:

o Fix the parsing of the device path. a last minute change terminated
it too soon.
o Kill setting LINES. We don't need to do it, and even if we did hard
coding it to 24 is wrong.
o Now that the console is working again for the loader, adjust the
printfs to be more in line with other platforms.

r336464:
If the console is already set, don't override it.

If console=X is specified on the command line, it's effectively
overridden by the current code. It shouldn't do that.

r336627:
Fix the attempt to see if we're overriding the console in the command
line args. I had thought console would be NULL, but it's efi. Set it
to efi (as a clue) before we initialize the console, then test it to
see if it changed on the command line to do the automatic
override. This gets my serial console back.

RelNotes: yes

MFC GELI Loader Improvements: r336252, r336254, r336256, r336354,
r336532-r336534, r336537, r336626, r337326, r337349, r341071, r341160,
r341420, r341473, r341651, r342793

Note that this MFC contains some seemingly unrelated zfsloader bits -- this
was needed in order to pull in some later fixes for GELI hand-off w/ ZFS
bits included.

r336252:
Extend loader(8) geli support to all architectures and all disk-like devices.

This moves the bulk of the geli support from lib386/biosdisk.c into a new
geli/gelidev.c which implements a devsw-type device whose dv_strategy()
function handles geli decryption. Support for all arches comes from moving
the taste-and-attach code to the devopen() function in libsa.

After opening any DEVT_DISK device, devopen() calls the new function
geli_probe_and_attach(), which will "attach" the geli code to the open_file
struct by creating a geli_devdesc instance to replace the disk_devdesc
instance in the open_file. That routes all IO for the device through the
geli code.

A new public geli_add_key() function is added, to allow arch/vendor-specific
code to add keys obtained from custom hardware or other sources.

With these changes, geli support will be compiled into all variations of
loader(8) on all arches because the default is WITH_LOADER_GELI.

r336254:
Use if rather than case for a simple boolean. gcc thinks blks is
undefined sometimes with the case, but enc is always 0 or 1, so
and if / else is better anyway.

r336256:
Fix glitched indentation (and rewrap as needed due to deeper indent).
No functional changes.

r336354:
zfsboot: fix build with WITHOUT_LOADER_GELI

r336532:
Collapse zfsloader functionality back down into loader.

We no longer really need a separate zfsloader. It was useful when we
were first supporting ZFS and had limited ability to properly boot off
of ZFS without the special boot loader. Now that the boot loader has
matured, go the way loader.efi pioneered and just build one
binary. Change the name of the loader to load in the secondary boot
blocks to be just /boot/loader. Provide a symbolic link from zfsloader
to loader so people who have not upgraded their boot blocks are not
affected. This has the happy benefit of making coexistence easier as
well (fewer binaries in the matrix).

r336533:
Eliminate zfsloader man page.

Remove all cross references to zfsloader.8 and /boot/zfsloader.
Move ZFS specific info into loader.8.

r336534:
NM and OBJCOPY are already defined for all builds. There's no need to
conditionally define them here.

r336537:
Mention zfsloader being folded into loader in UPDATING.

r336626:
Older zfs boot blocks don't support symlinks. install the link to
zfsloader as a hard link. While newer ones do, the whole point of the
link was to transition to the new world order smoothly. A hard link is
less flexible, but it works and will result in fewer bumps. Adjust
UPDATING entry to match.

r337326:
loader: biosdisk.c has leftover geli header.

A small cleanup, remove unneeded #include.

r337349:
zfsboot: Fix startup crash

On a FreeNAS mini XL, with geli encrypted drives the loader crashed in
geli_read().

When we iterate over the list of disks and allocate the zfsdsk structures we
don’t zero out the gdev pointer. In one case that resulted in geli_read()
(called on the bogus pointer) dividing by zero.

Use calloc() to ensure the zfsdsk structure is always zeroed, so the pointer is
initialised to NULL. As a side benefit it gets rid of one #ifdef
LOADER_GELI_SUPPORT.

r341071:
Restore the ability to override the disk unit/partition at the boot: prompt
in gptboot.

When arch-independent geli support was added, a new static 'gdsk' struct
was added, but there was still a static 'dsk' struct, and when you typed
in an alternate disk/partition, the string was parsed into that struct,
which was then never used for anything. Now the string gets parsed into
gdsk.dsk, the struct that's actually used.

r341160:
Add comments describing the bootargs handoff between loader(8) and gptboot
or zfsboot, when loader(8) is the BTX loader. No functional changes.

r341420:
Eliminate duplicated code and struct member definitions in the handoff
of args data between gptboot/zfsboot and loader(8).

Despite what seems like a lot of changes here, there are no actual
changes in behavior, or in the data layout in the structures involved.
This is just eliminating identical code pasted into multiple locations.

In detail, the changes are...

- Move struct zfs_boot_args definition from libsa/zfs/libzfs.h to
i386/common/bootargs.h because it is specific to x86 booting and the
handoff between zfsboot and loader, and has no relation to the zfs
library code in general.

- The geli_boot_args and zfs_boot_args structs both contain an identical
set of member variables containing geli information. Extract this out
to a new geli_boot_data struct, and embed it in the arg-passing structs.

- Provide new routines geli_import_boot_data() and geli_export_boot_data()
that can be shared between gptboot, zfsboot, and loader instead of
pasting identical code into several different .c files.

- Remove some checks for a NULL pointer that can never be true because the
pointer being tested was set using pointer math (kargs + 1) and that can
never result in NULL in this code.

r341473:
Fix args cross-threading between gptboot(8) and loader(8) with zfs support.

When loader(8) is built with zfs support enabled, it assumes that any extarg
data present is a zfs_boot_args struct, but if the first-stage loader was
gptboot(8) the extarg data is actually a geli_boot_args struct. Luckily,
zfsboot(8) and gptzfsboot(8) have always passed KARGS_FLAGS_ZFS along with
KARGS_FLAGS_EXTARG, so we can use KARGS_FLAGS_ZFS to decide whether the
extarg data is a zfs_boot_args struct.

To avoid similar problems in the future, gptboot(8) now passes a new
KARGS_FLAGS_GELI to indicate that extarg data is geli_boot_args. In
loader(8), if the neither KARGS_FLAGS_ZFS nor KARGS_FLAGS_GELI is set but
extarg data is present (which will be the case for gptboot compiled before
this change), we now check for the known size of the geli_boot_args struct
passed by the older versions of gptboot as a way of confirming what type of
extarg data is present.

In a semi-related tidying up, since loader's main() has already decided
what type of extarg data is present and set the global 'zargs' var
accordingly, don't repeat the check in extract_currdev, just check whether
zargs is NULL or not.

r341651:
Don't reference zfs-specific variables if LOADER_ZFS_SUPPORT is undefined
because the variables will be undefined too.

r342793:
MK_ZFS -> {MK_ZFS|MK_LOADER_ZFS}, this is so we can diable userland / kernel
ZFS but keep the boot-loaders when using ZoL port.

Relnotes: yes (GELI support extended)
Relnotes: yes (zfsloader has been collapsed into loader and may be
removed after boot blocks have been updated)

MFC r336244, r336246-r336247: Standardize boot arg parsing

r336244:
Create helper functions for parsing boot args.

boot_parse_arg to parse a single arg
boot_parse_cmdline to parse a command line string
boot_parse_args to parse all the args in a vector
boot_howto_to_env Convert howto bits to env vars
boot_env_to_howto Return howto mask mased on what's set in the environment.

All these routines return an int that's the bitmask of the args
translated to RB_* flags. As a special case, the 'S' flag sets the
comconsole_speed env var. Any arg that looks like a=b will set the env
key 'a' to value 'b'. If =b is omitted, 'a' is set to '1'. This
should help us reduce the number of redundant copies of these routines
in the tree. It should also give a more uniform experience between
platforms.

Also, invent a new flag RB_PROBE that's set when 'P' is parsed. On
x86 + BIOS, this means 'probe for the keyboard, and if it's not there
set both RB_MULTIPLE and RB_SERIAL (which means show the output on
both video and serial consoles, but make serial primary). Others it
may be some similar concept of probing, but it's loader dependent
what, exactly, it means.

These routines are suitable for /boot/loader and/or the kernel,
though they may not be suitable for the tightly hand-rolled-for-space
environments like boot2.

r336246:
Eliminate boot loader copies of boot arg parsing.

Eliminate 4 of the copies of the arg parsing in /boot/laoder
by using boot_parse_cmdline.

r336247:
Transition to boot_env_to_howto and boot_howto_to_env in the boot
loader.

MFC r336084-r336085, r336118: Move zfs impl into libsa

r336084:
Move ZFS files into libsa

Move the libzfs stuff into libsa. There's no need for it to be a
separate library. The separate library adds to the issues of build
ordering that we see from time to time. Move the filesystem support
into libsa, like all the other filesystem support rather than making
zfs the odd-duck out.

Discussed with: allanjude@

r336085:
Stop using ../zfs/libzfs.h but instead use libzfs.h.

While ../zfs/libzfs.h mostly works, there are a few situations where
it does not. Eliminate the problem by using plain libzfs.h, like we do
for ufs support. This fixes the weird cases, and is easier to
understand. It also follows the general style convetion of avoiding
../ in #includes.

r336118:
Remove dirs that git svn should have removed but didn't.

MFC r335228-r335231, r335272: stand: efiloader fixes

r335228:
Migrate has_keyboard to bool.

r335229:
Provide a more direct interface to tell ZFS what the preferred handle
is. We tell the ZFS code now, and it checks rather than having a
callback to do the checks.

This will allow us to have a more graceful fallback code. In the
future, it's anticipated that we may fallback to a more global search
(or implement a command to do so) when reqeusted by the user, or we
detect a violation of the UEFI Boot Manager protocol severe enough to
warrant this backstop. For now, it just allows us to get rid of img as
a global.

r335230:
Move arg parsing into its own routine for possible later reuse.

r335231:
There's no need to walk through the tables looking for the smbios
table if we're just going to ignore it on arm, so expand, slightly,
the reach of the ifdef. Move the buffer to the inner block so we
don't have a separate #ifdef far away from these lines.

The issue on arm is that smbios_detect does unaligned accesses, which
in the u-boot implementing EFI context causes a crash.

r335272:
Many netboot scenarios don't have /boot/defaults/loader.conf. As
a fallback, also check /boot/kernel/kernel existing as well, since
that's the fallback behavior of the loader.

MFC r335117: stand bootprog_info consolidation

bootprog_info is generated in vers.c. Move it's definition to
bootstrap.h and remove all the redundant copies.

MFC r335069-r335071, r335110, r335116: stand housecleaning

r335069:
Remove fail: label. It's unused.

r335070:
Remove unused variables.

[kevans@: removed ufs.c modifications; those depend on ufs rewrite bits]

r335071:
Initialize variables we later free so they aren't used
uninitialized in the error path.
Remove unused variables.

r335110:
Remove redundant defs.mk includes. They aren't needed.

r335116:
Use bool for vargood, since it's a boolean.

MFC 338022: Fix casts between 64-bit physical addresses and pointers in EFI.

Compiling FreeBSD/i386 with modern GCC triggers warnings for various
places that convert 64-bit EFI_ADDRs to pointers and vice versa.
- Cast pointers to uintptr_t rather than to uint64_t when assigning
to a 64-bit integer.
- Cast 64-bit integers to uintptr_t before a cast to a pointer.

MFC r338219, r338250: FDT in Loader fixes

r338219:
fdt_fixups: relocate the /chosen node after applying fixups

As indicated by the comment, any fixups applied (which might include
overlays) can invalidate the previously located node by adding nodes or
setting/adding properties. The later fdt_setprop of fixup-applied property
would then fail because of the bad/wrong node offset.

This would have generally been harmless, but potentially caused multiple
applications of fixups and caused a little bit of bloat.

r338250:
efiloader: Setup FDT in autoload to fix overlays clobbering kenv

manu found in the noted PR that overlays seemed to be clobbering the kenv
and killing the boot. Further inspection revealed that one can `fdt ls` at
the loader prompt for a successful boot, but autoboot breaks it.

In the autoboot case, first setup of FDT is happening in the middle of
bi_load, which triggers loading of the DTBO from /boot.

This is bad, bad, bad. Files in the loader are loaded somewhere in the
middle of the address space one after another. bi_load starts building the
needed kernel bootinfo immediately after the highest-addr loaded file. File
loads in the middle of bi_load suddenly clobber bootinfo and everything goes
off the rails.

The solution to this is to use take advantage of arch_autoload to setup FDT
in efiloader compiled with LOADER_FDT_SUPPORT. This matches how it works in
ubldr land, and is how it should have worked when overlay support was added
to efiloader since fdt_setup_fdtp now has the potential to load files
(courtesy of fdt_platform_load_dtb).

MFC r334882, r334884-r334885: loader(8) boot flag <-> environment fixes

r334882:
stand: Consolidate checking for boot flags driven by environment vars

e.g. boot_mute, boot_single, boot_verbose, and friends; we checked for these
in multiple places, consolidate into common/ and allow a setting of "NO" for
any of these to turn them off. This allows systems with multiple
loader.conf(5) or loader.conf(5) overlay systems to easily turn off
variables in later processed files by setting it to NO.

Reported by: Nick Wolff @ iXsystems
Reviewed by: imp

r334884:
stand: Fix build after r334882

Not sure how this was not caught in Universe.

r334885:
stand: One more trivial consolidation (setting environment from howto)

MFC r332416: Refactor currdev setting

Refactor the currdev setting to find the device we booted from. Limit
searching when we don't already have a reasonable currdev from that to
the same device only. Search a little harder for ZFS volumes as that's
needed for loader.efi to live on an ESP.

MFC r330806-r330815, r330837, r330864, r330883

pc98 changes associated with the named commits are also included in this
commit, despite not having been made with the original commits due to its
removal in head.

r330806:
Minor cosmetic changes.

Make sure { on the same line as struct for all struct *devdesc. Move
some type definitions to next to the dv_type define, since that's what
sets the d_type.

r330807:
We can't use d_opendata for blkio storage.

open_disk uses d_opendata for it's own purpse. We can't store blkio
there. Fortunately, blkio is stored elsewhere and we never actually
retrieve blkio from d_opendata. Eliminate it as a source of confusion.
Eliminate all stores of d_opendata in efi since this layer doesn't own
that field.

r330808:
Make struct libi386_devdesc match the struct devdesc better

Move data to top and call it d_opendata.

r330809:
Use the actual struct devdesc at the start of all *_devdesc structs

The current system is fragile and requires very careful layout of all
*_devdesc structures. It also makes it hard to change the base
devdesc. Take a page from CAM and put the 'header' in all the derived
classes and adjust the code to match.

For OFW, move the iHandle h_handle out of a slot conflicting with
d_opendata. Due to quirks in the alignment rules, this worked.
However changing the code to use d_opendata storage now that it's a
pointer is hard, so just have a separate field for it.

All other cleanups were to make the *_devdesc structures match where
they'd taken some liberties that were none-the-less compatible enough
to work.

r330810:
Remove d_type from devdesc. It's not needed as we can fetch it from
d_dev->dv_type when we need it.

r330811:
GC unused routines.

Sponsored by: Netflix

r330812:
Use the one-line-per-file pattern here, and sort the file names.

Sponsored by: Netflix

r330813:
Move the env convenience routines out of boot1.c.

These routines are more generally useful. Even though boot1 is on its
way out, it's better to make these common during the transition than
copy them.

r330814:
Star BootCurrent entry when booting.

Sponsored by: Netflix

r330815:
Print the load and device path as well as BootCurrent and BootOrder

Sponsored by: Netflix

r330837:
biosdisk.c should not set d_opendata.

Same as 330807, d_opendata is owned by open_disk and we should not
set it.

M stand/i386/libi386/biosdisk.c

r330864:
Prefer uintXX_t to u_intXX_t

A foolish consistency is the hobgoblin of little minds, adored by
little statesmen and philosophers and divines. With consistency a
great soul has simply nothing to do. -- Ralph Waldo Emerson

r330883:
Fix typo that misteriously passes compilation.

MFC r329725, r329831

r329725:
Consolidate three copies of ZFS commands into a central location.

There's no reason to have multiple copies of lszfs and
reloadbe. Consolidate them into one location. Also ldi_get_size is the
same everywhere (except sparc64). Make it the same everywhere as the
common definition is more general and will work on spar64.

r329831:
Fix userboot w/ ZFS after r329725

r329725 cleaned up ZFS commands duplicated in multiple places, but userboot
was not setting HAVE_ZFS when MK_ZFS != "no". This resulted in a failure to
boot (as seen in PR 226118) in bhyve, with the following message:

/boot/userboot.so: Undefined symbol "ldi_get_size"

MFC r329268, r329517

r329268:
efi: Only scan the BLKIO MEDIA once

Scan only the BLOCK IO MEDIA once instead of each time for each type of
device (fd, cd and hdd).
Leave the mechanism to free and reprobe all devices if one day we want
to implement a "dev rescan" thing.

r329517:
efi: Do not pad the efi devpath structure

This solve problem when booting with efi on armv7

MFC r330868, r331241, r331361, r331365: EFIRT Fixes

r330868:
EFIRT: SetVirtualAddressMap with 1:1 mapping after exiting boot services

This fixes a problem encountered on the Lenovo Thinkpad X220/Yoga 11e where
runtime services would try to inexplicably jump to other parts of memory
where it shouldn't be when attempting to enumerate EFI vars, causing a
panic.

The virtual mapping is enabled by default and can be disabled by setting
efi_disable_vmap in loader.conf(5).

r331241:
Check if the gettime runtime service is valid.

The U-Boot efi runtime service expects us to set the address map before
calling any runtime services. It will then remap a few functions to their
runtime version. One of these is the gettime function. If we call into
this without having set a runtime map we get a page fault.

Add a check to see if this is valid in efi_init() so we don't try to use
the possibly invalid pointer.

r331361:
Enter into the EFI environment before dereferencing the runtime services
pointer. This may be within the EFI address space and not the FreeBSD
kernel address space.

r331365:
Re-work efidev ordering to fix efirt preloaded by loader on amd64

On amd64, efi_enter calls fpu_kern_enter(). This may not be called until
fpuinitstate has been invoked, resulting in a kernel panic with
efirt_load="YES" in loader.conf(5).

Move fpuinitstate a little earlier in SI_SUB_DRIVERS so that we can squeeze
efirt between it and efirtc at SI_SUB_DRIVERS, SI_ORDER_ANY. efidev must be
after efirt and doesn't really need to be at SI_SUB_DEVFS, so drop it at
SI_SUB_DRIVER, SI_ORDER_ANY.

The not immediately obvious dependency of fpuinitstate by efirt has been
noted in both places.

stand: Clean up some unintentional inconsistencies

This is a direct commit to stable/11 to address the following final
unintended inconsistencies between stable/11 and head:

- Some unused LIBSTAND= cruft left in efi/loader/Makefie
- A comment that flew in with unrelated changes
- An #include that may go away now that stand is basically self-contained

MFC Loader Fixes Final: r327612,r327703,r327704,r327878,r327879,r327881,
r328007,r328029,r328030,r328031,r328061,r328156,r328169,r328288,r328289,
r328290,r328291,r328292,r328411,r328536,r328603,r328614,r328642,r328769,
r328779,r328780,r328781,r328782,r328783,r328806,r328808,r328826,r328835,
r328911,r328986,r328987,r328990,r328999,r329000,r329019,r329050,r329054,
r329060

r327612: Invent new #defines for the biospci_{read,write}_config function

r327703: Define __dmadat after #include'ing ufsread.c.

r327704: Fix printf missing format variables warnings.

r327878: Add GUID for UEFI boot manager variables.

r327879: Report the boot order and where we are in that boot order.

r327881: Allow this file to be included

r328007: Fix booting on some arm64 systems after r327879 by fixing the call
to utf8_to_ucs2

r328029: When returning an error and freeing allocated memory from
ucs2_to_utf8, NULL the return pointer.

r328030: Check the return value from utf8_to_ucs2 instead of whether or not
uv is NULL.

r328031: Need to free uv after we're done using it.

r328061: utf8_to_ucs2() should check for malloc failure

r328156: stand: Move sections around to fix stand/ build with ld.lld on
armv7

r328169: Remove extra copy of bootinfo.c. It's a bit rotted copy of the one
in efi/loader.

r328288: Fix some resource leaks.

r328289: Don't leak memory when displaying help.

r328290: On malloc failure, be sure to close the include file that triggered
it.

r328291: getenv does not return tainted data in the boot loader. Attempt to
clue Coverity into that fact.

r328292: There's no tainted data here, tag it as such to avoid false
positives.

r328411: loader.efi: add missing EFI GUIDs

r328536: loader: support for mixed-endianness ELF/loader and POWER8

r328603: Add missing non-POWERPC case to give the scr value something
non-zero.

r328614: Move libstand.3 to libsa.3. Update libsa.3 to include functions

r328642: Break out the interpreters (simple and forth) w/o ifdefs.

r328769: Centralize several variables.

r328779: Retire pnp.4th and the code needed only for 4th words used here.

r328780: These 4th words were an attempt to allow integration into the boot
loader scripts. However, that path won't be taken after all it
seems.

r328781: Remove pcibios forth support.

r328782: Now that we no longer conditionally compile some files outside of
ficl

r328783: Invent new LDR_INTERP for the loader interpreter to use.

r328806: We need more heap space to properly load newer powerpc kernels.

r328808: Implement strcoll as strcmp.

r328826: Make cross-endian loader changes apply only to powerpc

r328835: Fix regression introduced in r328806, preventing boot on many
platforms.

r328911: Ignore relocation tables for non-memory-resident sections.

r328986: Fix relative location of USB sources after recent move.

r328987: A more definitions to kernel emulation shim in order to build
stand/usb.

r328990: Move the stand/usb test loader into its own directory.

r328999: Fix indentation to FreeBSD standard for interp files

r329000: Move simple interpreter 'perform' into interp.c and call it

r329019: Move to tabs for indentation and to 8-space notches, per style(9).

r329050: Fix build of userboot.so

r329054: Set script.lang in the environment to either 'forth' or 'simple' to
reflect what scripting language was compiled into the loader.

r329060: loader: fix endianness conversion

PR: 225323

MFC Loader Fixes 2017q4p7: r324844,r326089,r326926,r326440,r326484,r326494,
r326588,r326708,r326784,r326914,r327390,r328446,r326090,r326143,r326144,
r326182,r326384,r326421,r326440,r326441,r326442,r326443,r326444,r326445,
r326446,r326447,r326448,r326484,r326485,r326486,r326487,r326488,r326490,
r326491,r326492,r326493,r326494,r326495,r326504,r326507,r326509,r326584,
r326585,r326586,r326587,r326588,r326589,r326590,r326591,r326592,r326593,
r326594,r326600,r326616,r326671,r326707,r326708,r326709,r326710,r326711,
r326712,r326714,r326720,r326768,r326772,r326784,r326792,r326812,r326854,
r326855,r326856,r326858,r326886,r326887,r326914,r326926,r326927,r326960,
r326961,r326962,r326963,r327351,r327453,r327390,r327523,r327524,r326489,
r327880,r328437,r328438,r328439,r328441,r328446,r328448,r328449,r328612,
r328613,r328615

While here, undo our libfdt hack of not including <stdlib.h> if we're
compiling _STANDALONE.

r324844: When building standalone, don't define errno. Let the definition from
stand.h override. This is similar to what we do in the kernel.

r326089: loader.efi: efipart does not recognize partitionless disks

r326090: net_parse_rootpath() has no parameters

r326143: Fix theoretical integer overflow issues. If the product here is

r326144: Mark the func pointer as __dead2. It looks up loader_main, which

r326182: Modify all FreeBSD bootloaders on PowerPC AIM (Book-S) systems

r326384: Use const pointers to avoid casting away constness.

r326421: loader.efi: efipart should exclude iPXE stub block protocol

r326440: Remove stale dependency on ufsread.c

r326441: Minor flags cleanup

r326442: Cleanup CFALGS usage here

r326443: We don't need both _STAND and _STANDALONE, use the latter.

r326444: Move geli to common DO32 stuff

r326445: Fix random() and srandom() prototypes to match the standard.

r326446: Undefine _STANDALONE since this is test code.

r326447: Tweaks to the beri boot loader so that it builds w/o warnings.

r326448: Fix all warnings related to geli and ZFS support on x86.

r326484: Const poison the propname.

r326485: Delcare md_load in libofw.h. Make all prototypes match for ofw

r326486: Include machine/md_var to pick up __syncicache prototype.

r326487: Cast mdp (a vm_offset_t) to void * to match prototype.

r326488: e_entry can be smaller than a pointer. Cast it to an intptr_t

r326490: Declare our strange brand of main().

r326491: Disconnet ps3 from the build. There's too many warnings to fix.

r326492: Cast void * pointer to char * so the arg matches the %s format.

r326493: Provide a md_load64 prototype.

r326494: Mark two things as unused (since they are only sometimes used)

r326495: Now it's safe to bump WARNS to 1.

r326504: Switch to proper MK_LOADER_GELI tests.

r326507: increase maximum size of zfsboot

r326509: loader.efi: add note about iPXE into the efipart.c

r326584: When building standalone, include stand.h rather than the kernel

r326585: Include ficl.h before anything else

r326586: No need to include the userland md5.h, the kernel one is just fine.

r326587: Use the kernel relative paths, rather than the userland relative
paths

r326588: Need to include skein in the include path

r326589: Make sure we include the right path for skein.h

r326590: Prefer stdint.h to inttypes.h

r326591: This isn't NetBSD specific code. Include these for any kernel /

r326592: Don't inherit CFLAGS. This a specialized test program.

r326593: Stop building with the standard system headers.

r326594: Now that we offer a semi-sane standards-ish set of #include files,
stop hacking includes with sed.

r326600: Since this is contrib code, create an upstreamable version of my

r326616: dhcp_try_rfc1048() is not used any more

r326671: Avoid setting -Wno-tentative-definition-incomplete-type with gcc.

r326707: Add partial support signal.h functioanlity. Pull in
machine/signal.h

r326708: Remove _KERNEL hack now that errno.h does the right thing when
built standalone.

r326709: Provide implementations for iscntrl, ispunct and isgraph.

r326710: Put the files we're copying over into a few variables and clean
hings up.

r326711: Const poison a couple of interfaces.

r326712: Create interp class.

r326714: boot1.c needs EFI_ZFS_BOOT too, so add it globally.

r326720: This path belongs in ficl/Makefile, not the common defines for
users

r326768: Fix a comment to be more accurate

r326772: Fix regression with lua import

r326784: Revert part of 362772. It was causing problems for includes

r326792: Attempt to unbreak buildworld

r326812: Revert r326792, r326784, r326772, r326712

r326854: libefi: make efichar.h more usable in stand code

r326855: Cargo cut a fix for the regressions r326585 caused.

r326856: Fix comments after bump in size.

r326858: Revert r326855: Cargo cut a fix for the regressions r326585 caused.

r326886: Panic in sbrk if setheap hasn't been called yet. This is preferable
o a mysterious crash

r326887: Remove the 'mini libstand in libstand' that util.[ch] provided.

r326914: Move loader-only defines to loader.mk from defs.mk

r326926: Move loader help file definitions to being 100% inside of
loader.mk.

r326927: libficl is only ever used in a loader (never a boot) program. Move
it.

r326960: Simplify things a little. The RETURN macro isn't required.

r326961: Interact is always called with NULL. Simplify code a little

r326962: Hoist btx include stuff to i386/Makefile.inc

r326963: No need to use relative paths like this here.

r327351: Fix ubldr. uboot/lib uses defines for the loader.

r327453: Add a validbcd() routine that uses the bcd2bin_data[] array

r327390: Garbage-collect loader.ps3. It is currently disconnected from the
build and kboot replaces.

r327523: Don't clobber system LDFLAGS for beri boot loaders.

r327524: Use 'extern uint8_t' instead of 'extern void' for external symbols.

r326489: Allow this file to be used in libsa without warning...

r327880: Move getsecs() prototype to stand.h from net.h so it can be used

r328437: Split panic routine

r328438: Implement abort() as a call to panic.

r328439: Provide abs form stdlib.h.

r328441: abort() should be marked __dead2 since it won't return.

r328446: Now that exit is __dead2, we need to tag ub_exit() as __dead2.

r328448: Make exit() never return until host_exit can be written.

r328449: Tag unreachable places as such. I left the while (1); in place

r328612: Move strtold wrapper from strtol.c to its own strtold.c.

r328613: Kill copies of strtol and strtoul.

r328615: Update stand.h for changes for strto*l

PR: 223969

MFC r325834,r325997,326502: Move sys/boot to stand/

This is effectively a direct commit to stable/11, due to differences between
stable/11 and head. Changes to DTS in sys/boot/fdt/dts were often
accompanied by kernel changes. Many of these were also risc-v updates that
likely had many more dependencies to MFC.

Because of this, sys/boot/fdt/dts remains as-is while everything else in
sys/boot relocates to stand/.

r325834: Move sys/boot to stand. Fix all references to new location

r325997: Remove empty directories.

r326502: Document the sys/boot -> stand move in hier.7 and the top-level README.

MFC Loader Fixes 2017q4p6: r324649,r324650,r324651,r324652,r324653,r324654,
r324700,r324702,r324709,r324717,r324719,r324841,r324842,r324843,r324845,
r324850,r324876,r324877,r324878,r324879,r324880,r324881,r324883,r324980,
r324981,r324982,r324995,r325014,r325093,r325094,r325114,r325170,r325171,
r325172,r325173,r325174,r325175,r325176,r325248,r325286,r325310,r325332,
r325338,r325339,r325376,r325377,r325379,r325380,r325382,r325478,r325479,
r325480,r325482,r325483,r325484,r325485,r325556,r325641,r325681,r325685,
r325686,r325687,r325688,r325689,r325690,r325691,r325692,r325693,r325694,
r325743,r325744,r325748,r325775,r325779,r325780

r324649: Move common/Makefile.inc to sys/boot/loader.mk.

r324650: tweak style

r324651: create defs.mk for common definitions

r324652: Move all the ficl common code into ficl.mk

r324653: LOADER_foo_SUPPORTED

r324654: Explicitly inlcude SYSDIR in the include path

r324700: loader: initialize dv_cleanup in md.c to eliminate clang warning

r324702: We need to include disk.o in libuboot.a when we're building with
support for disk access.

r324709: Revert "Unify boot1 with loader" change r324646

r324717: libsa/ip.c: misplaced comment, ip_v is half char, not ip_p

r324719: libsa/ip: stop read loop on bad fragments

r324841: Use BOOTDIR more consistently in defs.mk rather than repeat
sys/boot.

r324842: Introduce BOOTOBJ: The top level object directory for the boot tree

r324843: Stopgap fix to the mistmatch between LOADER_GELI_SUPPORT and
LOADER_NO_GELI_SUPPORT.

r324845: Use BOOTOBJ and BOOTDIR to find geli includes and libraries.

r324850: Define LIBSA32 to LIBSA on i386 to fix build.

r324876: Move fdt and uboot defines into common uboot.mk.

r324877: End source directories with SRC rather than a hodgepodge of names

r324878: Make at91 boot loader compile again.

r324879: Prefer SRCTOP paths for bits we're grabbing from libc.

r324880: Use BOOTSRC here.

r324881: Use SYSDIR instead of ${.CURDIR}/../..<etc>/sys.

r324883: Use preferred defined paths, rather than relative paths in fdt.

r324980: Use BOOTDIR consistently.

r324981: Move BINDIR definition to defs.mk, and override where it isn't
/boot

r324982: Remove sys/boot/arm/at91 and ixp425

r324995: loader.efi: Make framebuffer commands available for arm64

r325014: Add a 'place holder' arm struct efi_fb until a real one comes

r325093: Define new EFI variables

r325094: Cleanup non-arch Makefiles

r325114: Use defs.mk defins in most MD code

r325170: Use defs.mk values for userboot

r325171: Use defs.mk name and prefer bsd.init.mk

r325172: Remove the -nostdlib stuff I added. Instead, fix LDFLAGS to be
honored correctly with the new Makefile.inc include order.

r325173: We don't need to build a special ficl for userboot.

r325174: Minor cleanup

r325175: For amd64, compile both zfs and zfs32 libraries.

r325176: Actually add zfs32/Makefile

r325248: loader ptblread() is broken with >512B sectors

r325286: efipart_strategy is using wrong offset with >512B sectors

r325310: zfs.c:vdev_read() needs to be careful about large sectors

r325332: loader: re-enable gzip support for x86

r325338: loader: fix BOOTSRC -> BOOTOBJ in a library path

r325339: This used to have bzip2 support too.

r325376: WIP: centralize machine links

r325377: mostly libsa

r325379: Revert "mostly libsa"

r325380: Revert "WIP: centralize machine links"

r325382: Cleanup stray libstand names to be libsa names.

r325478: Powerpc is a 32-bit boot loader.

r325479: Define LIBFICL32 to be libficl.a on i386 and libficl32.a on amd64.

r325480: Use DO32 for all the places that we need to flag we're building a
32-bit version of a library.

r325482: Move machine and other link creation to defs.mk

r325483: MACHINE can never be powerpc64, so cleanup code that thinks it can.

r325484: Prefer bsd.init.mk to src.opts.mk

r325485: Centralize all 32-bit builds on 64-bit platform stuff.

r325556: loader: set options before including bsd.init.mk

r325641: loader.efi: efi_devpath_is_prefix should return bool

r325681: boot1: avoid using NULL device path

r325685: libsa32 isn't needed for i386. It's already a 32-bit platform.

r325686: Simplify this if to a direct assignment.

r325687: Remove all the empty help files from the powerpc build.

r325688: FDT support doesn't make sense for ps3, remove it.

r325689: Remove LOADER_FDT_SUPPORT as a Makefile variable.

r325690: Remove LOADER_ZFS_SUPPORT as a Makefile variable

r325691: Remove useless PNP define here.

r325692: Replace LOADER_FIREWIRE_SUPPORT variable

r325693: Move LOADER_{NO,}_GELI_SUPPORT to MK_LOADER_GELI

r325694: Install the 4th files in sys/boot/forth instead of each loader

r325743: Make sure the proper loader.rc gets installed.

r325744: boot1: also check for NULL device

r325748: Use proper include file.

r325775: Add loader.conf to the list of files that are MD.

r325779: Add /boot/dts to the list of default modules.

r325780: Don't add /boot/dt*s* but /boot/dt*b*. Stupid think-o.

MFC Loader Fixes 2017q4p5: r324557,r324558,r324559,r324646,r324647

r324557: Rename exit to efi_exit to avoid clashing with libsa exit
definition

r324558: Define prototype for exit and ensure references

r324559: Move panic back into libsa. It's documented in libstand(3) to
belong there.

r324646: Unify boot1 with loader.

r324647: Move orphaned man pages into new man directory from common.

MFC Loader Fixes 2017q4p3: r324460,r324471

r324460: Rather than laying whack-a-mole with including the path to stand.h,
always include it.

r324471: Fix typos: `stand alon{e,g}` should be spelled like `standalone`

MFC Loader Fixes 2017q4p1: r324321,r324359,r324360,r324388,r324449,r324450,
r324451,r324452

r324321: This README file was quite relevant for FreeBSD 3 or 4. No more.

r324359: Add efi_devpath_is_prefix

r324360: Encapsulate ZFS preferences into efi_zfs_is_preferred

r324388: Unbreak building efiboot with MK_ZFS == no

r324449: Prefer ${LIBSTAND} to -lstand

r324450: Define SASRC and use it

r324451: Add missing -I${SASRC} in a couple of places so that stand.h
doesn't accidentally come in via host pollution.

r324452: Define LIBSA* and use them instead of overloaded LIBSTAND

MFC Loader Fixes 2017q3: r320547,r320553,r321621,r321844,r321969,r321991,
r322037,r322038,r322039,r322040,r322056,r322074,r322542,r322592,r322593,
r322896,r322923,r323671,r322930,r322931,r322932,r322933,r322934,r322935,
r322936,r322937,r322938,r322939,r322941,r323062,r323063,r323064,r323065,
r323100,r323131,r323174,r323258,r323261,r323272,r323367,r323379,r323389,
r323407,r323428,r323436,r323494,r323496,r323497,r323541,r323554,r323589,
r323707,r323867,r323885,r323886,r323895,r323896,r323897,r323905,r323906,
r323907,r323908,r323909,r323952,r323991,r324099,r324558,r326445,r326609,
r326610

This batch includes a special kludge to fix powerpc loader build; <stdlib.h>
was included after <stand.h> there, causing problems with DEBUG_MALLOC bits.
Include <stdlib.h> a little bit earlier to fix the build with the intention
of removing this when eventually libsa silently replaces stdlib.h with
stand.h.

r320547: Link EFI/uboot loaders with -znotext

r320553: Integer underflow in efipart_realstrategy when I/O starts after end
of disk

r321621: Always set the receive mask in loader.efi.

r321844: Clean up style in print_state(..) and pager_printf(..)

r321969: Fix the return types for printf and putchar to match their libc

r321991: Revert r321969

r322037: Add stpcpy and stpncpy to libstand

r322038: Add definitions and utilities for EFI drivers

r322039: Move EFI ZFS functions to libefi

r322040: Add EFI utility functions to libefi

r322056: Move EFI fmtdev functionality to libefi

r322074: libefi/time.c cstyle cleanup

r322542: loader.efi: repace XXX with real comments in trap.c

r322592: Remove unused defines.

r322593: Define proposed GUID for FreeBSD boot loader variables.

r322896: Make spinconsole platform independent and hook it up into EFI
loader

r322923: Hide length of geli passphrase during boot.

r323671: Fix language used in the r322923.

r322930: Move efi_main into efi/loader

r322931: Cleanup efi_main return type

r322932: Use the loader.efi conventions for the various EFI tables.

r322933: No need for MK_ZFS around these: they are by their nature only
active when MK_ZFS is true.

r322934: _STAND is sometimes defined on the command line. Make the define
here match.

r322935: Fix warnings due to type mismatch.

r322936: Remove useless 'static' for an enum definition.

r322937: Forward declare struct dsk to avoid warnings when building libi386.

r322938: Link in libefi for boot1

r322939: Use efi_devpath_str for debug path info.

r322941: Eliminate redunant device path matching.

r323062: Make efichar.c routines available to libefi.

r323063: boot1.efi: print more info about where boot1.efi is loaded from

r323064: Exit rather than panic for most errors.

r323065: Save where we're booted from

r323100: libstand: nfs_readlink() should return proper return code

r323131: Revert r322941: Eliminate redundant device matching functions

r323174: Fix loader bug causing too many pages allocation when bootloader
is U-Boot

r323258: ucs2len

r323261: Fix armv6 build

r323272: Be consistent and do return (1);

r323367: Mark init_chroot and init_script variables as deprecated.

r323379: It's been pointed out that init_script at least is useful w/o

r323389: loader.efi: chain loader should provide proper device handle

r323407: boot1 generate-fat: generate all templates at once

r323428: r323389 breaks the kernel build when WITHOUT_ZFS is defined in
src.conf

r323436: boot1: remove BOOT1_MAXSIZE default value

r323494: loader should support large_dnode

r323496: libstand: tftp_open() can leak pkt on error

r323497: libefi: efipart_open should check the status from disk_open

r323541: libefi: efipart_realstrategy rsize pointer may be NULL

r323554: Increase EFI boot file size frok 128k to 384k

r323589: loader: biosmem.c cstyle cleanup

r323707: loader: biosmem allocate heap just below 4GB

r323867: libefi: devicename.c cleanups

r323885: libefi: efi_devpath_match() should return bool

r323886: libefi: efipart.c should use calloc()

r323895: libefi: efi_devpath_match local len should be unsigned

r323896: r323885 did miss efilib.h update

r323897: efilib.h: typo in structure member description

r323905: libefi: pdinfo_t pd_unit and pd_open should be unsigned

r323906: libefi: efipart_strategy() should return ENXIO when there is no
media

r323907: libefi: efipart.c cstyle fix for efipart_print_common()

r323908: libefi: efipart_hdinfo_add_filepath should check strtol result

r323909: libefi: define EISA PNP constants

r323952: After the r317886 support for TFTP and NFS can be enable
simultaneously.

r323991: libefi: efipart_floppy() will should not pass acpi pointer if the
HID test fails

r324099: Compile loader as Little-Endian on PPC64/POWER8

r324558: Define prototype for exit and ensure references

r326445: Fix random() and srandom() prototypes to match the standard.

r326609: Make putenv and getenv match the userland definition

r326610: Fix random() prototype to match the system.

PR: 219000 221001 222215
Relnotes: yes ("The length of the geli passphrase is hidden during boot")

MFC Loader Fixes 2017q2: r316437,r316577,r316578,r316585,r316590,r316612,
r316625,r316628,r316654,r316682,r316704,r316771,r317092,r317097,r317099,
r317652,r317785,r317886,r317887,r318142,r318320,r318356,r318678,r318754,
r318982,r318986,r318987,r318988,r318989,r318990,r318991,r318992,r318993,
r318994,r318999,r319083,r319084,r319085,r320011,r320234,r320288,r320304,
r320467,r320482

r316437: Small cleanup to make i386/loader match efi/loader boot environment
code

r316577: loader: part.c cstyle cleanup

r316578: loader: want mechanism to avoid RA with bcache

r316585: loader: zfs reader should check all labels

r316590: libstand/dosfs: cache FAT32 in 128 Kb blocks to save loader memory

r316612: In r298230 the value of HEAP_MIN was changed from 3MB to 64MB.
Correct a comment.

r316625: Do not use -msoft-float with intention of disabling FP on aarch64

r316628: Silence GCC warning by initializing the local variable.

r316654: loader: r316585 did miss userboot update

r316682: loader: r316585 did miss sparc/ofw

r316704: loader.efi: only fetch zfs pool guid for the actual boot device

r316771: loader: Avoid possible overflow via environment variable

r317092: loader: zfs reader vdev_probe should check for minimum device size

r317097: loader: F_READ/F_WRITE should be checked against masked flag

r317099: loader: uboot disk ioctl should call disk_ioctl

r317652: loader.efi: ResetSystem does not use data with EFI_SUCCESS

r317785: zfsboot: drvsize() may be unusable on some systems

r317886: distinguish NFS versus TFTP boot by rootpath

r317887: loader: network read rework

r318142: libstand: NULL pointer dereference in rarp

r318320: loader: add ip layer code into libstand

r318356: libstand: increase nfs max read size to 16k

r318678: Replacing iterating over rootpath by strsep(3).

r318754: Pass -N directly to ld via -Wl rather than passing it to the
compiler driver.

r318982: Pass a "FREEBSD" user-class in PXE dhcp request

r318986: add a comment on vendor index 19 and 20 to avoid confusion

r318987: Support URI scheme for root-path in netbooting

r318988: Always build tftpfs support along with nfs for pxeboot

r318989: Always issue the pxe request

r318990: Partially revert r314948

r318991: Document recent changes on pxeboot

r318992: Capitalize DHCP

r318993: Use the usual FreeBSD spelling for the DHCP user class

r318994: Catch with the change in the user class

r318999: Update the comments concerning net_parse_rootpath to reflect what
it is now

r319083: Followup on the user-class changes

r319084: Small cleanup in dev_net.c

r319085: use the same option list for dhcp discovery and request

r320011: Add chain loader support for loader

r320234: Make structure padding explicit in EFI_MEMORY_DESCRIPTOR

r320288: Allow Clang's integrated assembler to assemble boot0

r320304: loader.efi: Disable smbios for arm

r320467: loader: chain load relocate data declaration is bad

r320482: As with arm64 mark the EFI PE header as allocated on arm.

PR: 218473

MFC Loader Fixes 2017q1: r311458,r312237,r312314,r312374,r312947,r313042,
r313047,r313166,r313328,r313332,r313333,r313337,r313348,r313349,r313389,
r313442,r313451,r313575,r313645,r313710,r314114,r314213,r314275,r314945,
r314948,r315008,r315408,r315427,r315645,r315646,r315648,r315653,r315850,
r316064,r316078,r316079,r316100,r316104,r316111,r316112,r316171,r316279,
r316280,r316287,r316311,r316343,r316424,r316436

r311458: Use compiler driver to link BERI boot loaders

r312237: loader.efi: find_currdev() can leak memory

r312314: loader: move device path definitions to include/efidevp.h

r312374: loader: efi devpath api usage should be more aware of NULL pointers

r312947: Remove "-Xassembler -G0" from CFLAGS.

r313042: loader.efi environment related cleanups

r313047: loader: disk/part api needs to use uint64_t offsets

r313166: loader: libefi/env.c warnings in arm build

r313328: loader: Implement disk_ioctl() to support DIOCGSECTORSIZE and
DIOCGMEDIASIZE.

r313332: loader: bcache read ahead block count should take account the large
sectors

r313333: loader: Replace EFI part devices.

r313337: loader: 313329 missed ZFS guard in loader/main.c

r313348: loader: biosdisk fix for 2+TB disks

r313349: loader: disk io should not use alloca()

r313389: efipart is also using the '%S' printf format, add -Wno-format for
it.

r313442: loader: possible NULL pointer dereference in efipart.c

r313451: loader: possible NULL pointer dereference in bcache.c

r313575: makefs: make the buffer functions look exactly like the kernel ones

r313645: loader: implement MEDIA_FILEPATH_DP support in efipart

r313710: loader: cstyle fixes and DIOCGMEDIASIZE should use uint64_t

r314114: Use LDFLAGS with CC instead of _LDFLAGS.

r314213: Remove control+r handling from geliboot's pwgets()

r314275: Remove unused macro from common/drv.c.

r314945: Some style(9) fixes. No functional changes.

r314948: Try to extract the RFC1048 data from PXE.

r315008: r314948 seems to be missing a variable or two that will break

r315408: loader: remove open_disk cache

r315427: loader: biosdisk should report IO error from INT13

r315645: loader: disk_cleanup was left in userboot_disk.c

r315646: loader: pxe.h constants have wrong values

r315648: libstand: verify value provided by nfs.read_size

r315653: loader: verify the value from dhcp.interface-mtu and use snprintf
o set mtu

r315850: The original author abused Nd (one-line description, used by
makewhatis)

r316064: Fix build with path names with 'align' or 'nop' in them.

r316078: gpt*boot: Save a bit more memory when LOADER_NO_GELI_SUPPORT is
specified

r316079: Simply retire the sedification of the boot2.s file.

r316100: Remove -fno-guess-branch-probability and -fno-unit-at-a-time.

r316104: Use `NO_WCAST_ALIGN` instead of spelling it out as -Wno-cast-align
in CFLAGS

r316111: loader: move bios getsecs into time.c

r316112: loader: ls command should display file types properly

r316171: xfsread inlined uses more space, so remove the inline tag.

r316279: loader: efipart should check disk size from partition table

r316280: loader: simplify efi_zfs_probe and avoid double probing for zfs.

r316287: Remove OLD_NFSV2 from loader and libstand

r316311: Add explicit_bzero() to libstand, and switch GELIBoot to using it

r316343: Implement boot-time encryption key passing (keybuf)

r316424: Fix sparc64 build broken by r316343 and r316076

r316436: Restore EFI boot environment functionality broken in r313333

PR: 216940 217298 217935

MFC r307322,r307323,r307324,r307326,r307327,r307338,r307879,r307908,r307911,
r307942,r307950,r307951,r307954,r307955,r308125,r308195,r308476,r308534,
r308535,r308776,r308843,r310236,r310726: Loader fixes, 2016q4

r307322: Remove /boot/boot.conf, deprecated for 16 years

r307323: Remove fetching of pInterp.

r307324: Create a new linker set, Xficl_compile_set which contains a list of
functions to call to register new forth words.

r307326: In UEFI mode expose the SMBIOS anchor base address via kenv

r307327: Update i386 build of loader.efi (but leave it disabled) so that we
at least build it now.

r307338: Create a pcibios-version environment FORTH word.

r307879: Preliminary support for EFI in boot loader.

r307908: Fix the build on both arm64 and when WITHOUT_FORTH is defined.

r307911: Add better comment...

r307942: Really make WITHOUT_FORTH (MK_FORTH==no) work.

r307950: Add it (Makefile.ficl) to the right place

r307951: Fix two backwards tests.

r307954: Back out the move to the loader script from -N.

r307955: LIBSTAND goes last, so put it last here too.

r308125: In loader.efi, instead of exiting directly, try to fallback.

r308195: efinet_dev_print should honor verbose option.

r308476: boot/forth spelling issue in forth word

r308534: The file_loadraw function grew an argument, update install function

r308535: Add support for LOADER_RC setting in the pkgfs manifes

r308776: loader: zfs toplevel vdev must have spa set.

r308843: loader: smbios version check is not correct

r310236: Renumber license clauses to avoid skipping #3

r310726: cdboot: add explict suffix to ambiguous or instruction

PR: 214375

MFC r303555,r303556,r303936,r303962,r304317,r304532,r305026,r305107,r305132,
r305178,r305353,r305814,r306159,r306380,r306504: Loader fixes, 2016q3

r303555: bcache should support reads shorter than sector size

r303556: Improve boot loader quote parsing

r303936: Add kernel environment variables under smbios.system

r303962: Add the missing space between .asciz directive and opening quote
for some lines with #ifdef BTXLDR_VERBOSE/#endif

r304317: boot1.efi Free() should check for NULL to provide consistent
behavior

r304532: Replace sprintf -> snprintf for command_errbuf provisioned from
dynamic content.

r305026: Emulate efi_cons_poll when WaitForKey is not available

r305107: Create a hook 'post-initialize' for people that want to define
something to read in .conf files after all other .conf files for the purpose
of overriding.

r305132: Remove accidentally committed stray comment.

r305178: bd_int13probe() should check extended info if sector info is bad

r305353: Don't use -N to set the OMAGIC with data and text writeable and
data not page aligned.

r305814: ufsread: Do not cast struct direct from void *

r306159: Consistently declare getsecs(void) with proper return type and void
when no arguments are present.

r306380: loader command interpreter should reset command_errmsg

r306504: Fix a cluster of bugs in list EFI environment variables

PR: 204602, 211958, 211958
Relnotes: yes ("Improve boot loader quote parsing")

MFC r308434, 308827

MFC r308434: Loader paged/pageable data is not always paged.
MFC r308827: lsdev device name section headers should be printed by dv_print
callback.

MFC r304321,304753,304754,306751,316077,316110:
SHA512, skein, large block support for loader zfs

MFC r304321: Add SHA512, skein, large blocks support for loader zfs.
MFC r304753: loader: zio_checksum_verify() must test spa for NULL pointer
MFC r304754: r304321 broken bhyve zvol VM bhyveload hang 100% WCPU
MFC r306751: Disable loop unrolling in skein for sys/boot
MFC r316077: Unbreak compilation with gcc 4.2.1
MFC r316110: Use `-Wno-missing-declarations` with CWARNFLAGS for skein.c

MFC r316119 (by ngie):

self_reloc.c: Pass -Wno-error=maybe-uninitialized to gcc versions
greater than 4.2.1

MFC r318193:
Set the right variable when overriding the default console speed.

Fix the arm64 userland building with lld:

MFC 308124:
On arm64 build the efi loader with -fPIC. Without this clang 3.9 will
generate relocation in the self relocation code.

MFC 316608:
Add -fPIC to the standalone build flags on arm64. This is needed as
loader.efi is position independend, however we were not building it as
such causing a build failure when building with lld.

MFC 315452:
Mark the EFI PE header as allocated. While ld.bfd doesn't seem to care
about not having this flag ld.lld fails to link without it.

MFC: 314547, 314770, 314828, 314891, 314956, 314962, 315235

r314547
loader.efi: reduce the size of the staging area if necessary

The loader assumes physical memory in [2MB, 2MB + EFI_STAGING_SIZE)
is Conventional Memory, but actually it may not, e.g. in the case
of Hyper-V Generation-2 VM (i.e. UEFI VM) running on Windows
Server 2012 R2 host, there is a BootServiceData memory block at
the address 47.449MB and the memory is not writable.

Without the patch, the loader will crash in efi_copy_finish():
see PR 211746.

The patch verifies the end of the staging area, and reduces its
size if necessary. This way, the loader will not try to write into
the BootServiceData memory any longer.

Thank Marcel Moolenaar for helping me on this issue!

The patch also allocates the staging area in the first 1GB memory.
See the comment in the patch for this.

PR: 211746
Reviewed by: marcel, kib, sephe
Approved by: sephe (mentor)
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D9686

r314770
loader.efi: fix recent UEFI-boot regression on physical machines

This patch fixes my recent patch
"loader.efi: reduce the size of the staging area if necessary", which
causes EFI-boot failure on physical machines since Mar 2:
on the host there is a 1MB LoaderData memory range, which splits
the big Conventional Memory range into a small one (15MB) and a
big one: the small one is too small to hold the staging area.

We can actually use the LoaderData range safely, because when
amd64_tramp -> efi_copy_finish() starts to run, we're almost at
the very end of the efi loader code and we're going to "return"
to the kernel entry, so we're pretty sure we won't access any loader
data any more.

For people who are interested in the details: please see
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=211746#c22

PS, some people also reported the regression happened to FreeBSD VM
running on Bhyve in EFI mode. This patch should resolve it too,
though I don't have such a setup to test.

Reviewed by: sephe
Approved by: sephe (mentor)
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D9904

r314828
loader.efi: fix an off-by-one bug in efi_verify_staging_size()

Also remove the warning message: it may not be unusual to see
the memory range containing 2MB is not of EfiConventionalMemory.

Sponsored by: Microsoft

r314891
loader.efi: finally fix the off-by-one bug in efi_verify_staging_size()

r314828(loader.efi: fix an off-by-one bug in efi_verify_staging_size())
doesn't really fix the bug and this patch adds the missing part.

It's a shame that I didn't make everything correct at the very beginning...

Sponsored by: Microsoft

r314956
loader.efi: only reduce the size of the staging area on Hyper-V

Doing this on physical hosts turns out to be problematic, e.g. see comment
24 and 28 in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=211746.

To fix the real underlying issue correctly & thoroughly, IMO we need
a relocatable kernel, but that would require a lot of complicated long
term work: https://reviews.freebsd.org/D9686?id=25414#inline-56969

For now, let's only apply efi_verify_staging_size() to VMs running on
Hyper-V, and restore the old behavior on physical machines since that
has been working for people for a long period of time, though that's
potentially unsafe...

Sponsored by: Microsoft

r314962
loader.efi: only include the machine/ header files on x86

The 2 files may not exist on other archs like aarch64 and hence we
can have a build failure there.

Reported by: lwhsu
Sponsored by: Microsoft

r315235
loader.efi: use stricter check for Hyper-V

Some other hypervisors like Xen can pretend to be Hyper-V but obviously
they can't implement all Hyper-V features. Let's make sure we're genuine
Hyper-V here.

Also fix some minor coding style issues.

PR: 211746
Sponsored by: Microsoft

PR: 211746

MFC r310267: Deduplicate loader vers.c Makefile rules

The Makefile rule to create vers.c for loader version info was
previously duplicated in each of the various loader Makefiles.
Instead, share a common rule in Makefile.inc.

Sponsored by: The FreeBSD Foundation

MFC r311888, r311890-r311891

r311888:
[efi] Fix off-by-one error in ARM .bss zeroing code in loader's _start

__bss_end should not be included in .bss zeroing code. Otherwise first 4
bytes of the section that follows .bss (in loader's case it's .sdata) are
overwritten by zero.

Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D9108

r311890:
[efi] Fix .rel.data.* being erroneously merged into .data on ARM

Fix section pattern code to exclude .rel.data.* sections from being
merged into .data. Otherwise relocations in those sections are lost
in final binary

Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D9108

r311891:
[efi] Build EFI bits with -fPIC on ARM

clang 3.9.0 without -fPIC generates absolute jump table for
switch/case statement which trips boot1.efi and loader.efi
on ARM platform.

Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D9108

MFC r310225: Reduce boot loader version string duplication

Instead of repeating "%s, Revision %s" "(%s %s)" in each loader, just
create the full version string in vers.c

MFC r306264: Use 32-bit value for .text padding, for linker portability

GNU ld interprets the padding value as a variable-length byte string,
while GNU gold and LLVM lld interpret it as a 32-bit value.

MFC r305854: Use arch-specific .text padding fill value in EFI loaders

The fill pattern was previously an ia64 instruction sequence. Presumably
ia64's linker script was copied as a starting point.

MFC: 304677, 304680, 305125, 305770, 305769

r304677:
Add tftpfs support for the EFI loader

Allow netbooting on efi without having to setup any NFS server by rebuilding the
loader with LOADER_TFTP_SUPPORT like for the i386 pxeloader

r305125:
Netboot: allow both tftpfs and nfs in both pxeboot and loader.efi

Add a new 'netproto' variable which can be set for now to
NET_TFTP or NET_NFS (default to NET_NONE)

From the dhcp options if one sets the root-path option to:
"ip:path", the loader will consider it is booting over NFS
(meaning same behaviour as the default current behaviour)

if the dhcp option "tftp server address" is set (option 150)
the loader will consider it is booting over tftpfs, it will then
consider the root-path options with 2 possible case
1. "path" then the IP of the tftp server will be the one passed by
the option 150, and the files will be retrieved under "path" on the tftp
server
2. "ip:path" then the IP of the tftp server will be the one passed in
the option "overwritting the IP from the option 150.

We could not "abuse" the rootpath option in the form or tftp://ip:path because
this is already used for other purpose by iPXE preventing any chainload from
iPXE to the FreeBSD loader.

Given at each open(), the loader loops over all available filesystems and keep
the "best" error, we needed to prevent tftpfs to fallback on nfs and vice versa.
the tftpfs and nfs implementation in libstand now return EINVAL early if
'netproto' for that purpose.

Relnotes: yes
Sponsored by: Gandi.net

MFC r306090:
Simple post-mortem reporter for amd64 loader.efi.

MFC r305943:
Utilize pmap.h names.

MFC r305160: Set UEFI boot loader PE/COFF timestamps to known value for reproducible builds

Sponsored by: The FreeBSD Foundation

MFC r305484:

loader.efi: Bump the staging size to 64M.

This is required on my system, which loads nvidia, vmm, and zfs, and 48M is
no longer enough for that. nvidia-driver's recent update increased its size
by several megabytes.

MFC 304801:
Don't set *dev in the zfs root case, it may be NULL and will correctly be
set later in the function. This fixes a potential NULL pointer dereference
found on arm64.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation

MFC 304018: Add defines needed to export SMBIOS serial numbers

Some defines needed for exporting serial numbers from the SMBIOS were
missed during integration of SMBIOS support in the EFI boot loader (r281138).
This is needed for getting the hostid set from the system hardware UUID.

PR: 206031

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Use the UEFI event timer to update the time on arm and arm64. The current
code uses the GetTime function from the Runtime Service, however this has
been shown to not return a useable time on many arm64 UEFI implementations.

Reviewed by: jhb, smh
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D6709

Fix unit number of EFI net interfaces and ignore psuedo network interfaces.

In r277943, the efinet_match() routine was changed to use an off by one
when matching network interfaces. The effect was that using "net1"
actually used the device attached to "net0".

Digging into the hardware that needed this workaround more, I found that
UEFI was creating two simple network protocol devices for each physical
NIC. The first device was a "raw" Ethernet device and the second device
was a "IP" device that used the IP protocol on top of the underlying
"raw" device. The PXE code in the firmware used the "IP" device to pull
across the loader.efi, so currdev was set to "net1" when booting from the
physical interface "net0". (The loaded image's device handle referenced
the "IP" device that "net1" claimed.)

However, the IP device isn't suitable for doing raw packet I/O (and the
current code to open devices exclusively actually turns the "IP" devices
off on these systems).

To fix, change the efinet driver to only attach to "raw" devices. This
is determined by fetching the DEVICE_PATH for each handle which supports
the simple network protocol and examining the last node in the path. If
the last node in the path is a MAC address, the device is assumed to be
a "raw" device and is added as a 'netX' device. If the last node is not
a MAC address, the device is ignored.

However, this causes a new problem as the device handle associated with
the loaded image no longer matches any of the handles enumerated by
efinet for systems that load the image via the "IP" device. To handle
this case, expand the logic that resolves currdev from the loaded image
in main(). First, the existing logic of looking for a handle that
matches the loaded image's handle is tried. If that fails, the device
path of the handle that loaded the loaded image is fetched via
efi_lookup_image_devpath(). This device path is then walked from the
end up to the beginning using efi_handle_lookup() to fetch the handle
associated with a path. If the handle is found and is a known handle,
then that is used as currdev. The effect for machines that load the
image via the "IP" device is that the first lookup fails (the handle
for the "IP" device isn't claimed by efinet), but walking up the
image's device path finds the handle of the raw MAC device which is used
as currdev.

With these fixes in place, the hack to subtract 1 from the unit can now
be removed, so that setting currdev to 'net0' actually uses 'net0'.

PR: 202097
Tested by: ambrisko
Sponsored by: Cisco Systems

Remove second copy of the -Wno-format band-aid for printf %S.

efi loader: Match format string to EFI_ERROR_CODE()

Silence a format specifier warning.

Sponsored by: EMC / Isilon Storage Division

Large improvements to efi-show (though some weird problems
linger). We now print only printable characters for the
values and we print ascii strings as strings.

Implement efi-set and efi-unset

Cleanup to use %S.

Fix logic error so UEFI variables are reported correctly
without error at the end.

Minor spelling fixes.

Fix several instances where the boot loader ignored pager_output
return value when it could return 1 (indicating we should stop).
Fix a few instances of pager_open() / pager_close() not being called.
Actually use these routines for the environment variable printing code
I just committed.

Fix build breakage on arm64 by papering over the problem. We implement
a slightly non-standard %S that's more useful in the UEFI environment,
so ignore printf errors. There's no good cast to use. We'll need to
revisit this in the future.

Implement UEFI set environment variable, as well as exporting the EFI
version. This is also scriptable, though additional scripting will be
needed.

Differential Review: https://reviews.freebsd.org/D4494
MFC After: 3 days

sys/boot: spelling fixes in comments.

No functional change.

sys/boot: use our nitems() macro when it is available through param.h.

No functional change, only trivial cases are done in this sweep,

Discussed in: freebsd-current

A new implementation of the loader block cache

The block cache implementation in loader has proven to be almost useless, and in worst case even slowing down the disk reads due to insufficient cache size and extra memory copy.
Also the current cache implementation does not cache reads from CDs, or work with zfs built on top of multiple disks.
Instead of an LRU, this code uses a simple hash (O(1) read from cache), and instead of a single global cache, a separate cache per block device.
The cache also implements limited read-ahead to increase performance.
To simplify read ahead management, the read ahead will not wrap over bcache end, so in worst case, single block physical read will be performed to fill the last block in bcache.

Booting from a virtual CD over IPMI:
0ms latency, before: 27 second, after: 7 seconds
60ms latency, before: over 12 minutes, after: under 5 minutes.

Submitted by: Toomas Soome <tsoome@me.com>
Reviewed by: delphij (previous version), emaste (previous version)
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D4713

Deprecate using hints.acpi.0.rsdp to communicate the RSDP to the
system. This uses the hints mechnanism. This mostly works today
because when there's no static hints (the default), this value can be
fetched from the hint. When there is a static hints file, the hint
passed from the boot loader to the kernel is ignored, but for the BIOS
case we're able to find it anyway. However, with UEFI, the fallback
doesn't work, so we get a panic instead.

Switch to acpi.rsdp and use TUNABLE_ULONG_FETCH instead. Continue to
generate the old values to allow for transitions. In addition, fall
back to the old method if the new method isn't present.

Add comments about all this.

Differential Revision: https://reviews.freebsd.org/D5866

boot/efi: Prefer nm to objdump

Both objdump and nm are equally capable of reporting undefined symbols.

This gets us a step closer to building without binutils as we have an nm
implementation from ELF Tool Chain.

Reviewed by: bdrewery
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D5613

Add more .NOMETA missed in r291320

Sponsored by: EMC / Isilon Storage Division

DIRDEPS_BUILD: Regenerate without local dependencies.

These are no longer needed after the recent 'beforebuild: depend' changes
and hooking DIRDEPS_BUILD into a subset of FAST_DEPEND which supports
skipping 'make depend'.

Sponsored by: EMC / Isilon Storage Division

Add a missing call to dev_cleanup from the arm64 loader.efi.

Sponsored by: ABT Systems Ltd

Ensure the bss is aligned to a 4-byte address as we use 4-byte aligned
stores to clear it.

While here reduce the alignment of the data from 4k to 16 byte aligned.
This should be more than enough, without wasting too much space.

Sponsored by: ABT Systems Ltd

Implement -P for boot loader. It's a bit easier to implement here than
in boot1, like is normally done. When a keyboard appears in the UEFI
device tree, assume -D -h, just like on a BIOS boot.

# It is unclear if an ACPI keyboard appearing in the tree means there's
# a real keyboard or not. A USB keyboard doesn't seem to appear unless
# it is really there.

Differential Revision: https://reviews.freebsd.org/D5223

Parse the command line arguments, and do it before we initialize the
console so it can be changed by the command line arguments.

Differential Revision: https://reviews.freebsd.org/D5038

Connect the ZFS boot environment menu to the UEFI loader

MFC after: 3 days
Sponsored by: ScaleEngine Inc.

Add EFI ZFS boot support

This builds on the modular EFI loader support added r294060 adding a
module to provide ZFS boot support on EFI systems.

It should be noted that EFI uses a fixed size memory block for all
allocations performed by the loader so it may be necessary to tune this
size.

For example when building an image which uses mfs_root e.g. mfsbsd, adding
the following to /etc/make.conf would be needed to prevent EFI from running
out of memory when loading the mfs_root image.
EFI_STAGING_SIZE=128

Submitted by: Eric McCorkle
MFC after: 2 weeks
X-MFC-With: r293268
Sponsored by: Multiplay

Modularise EFI boot loader

Make EFI boot loader modular in preparation for adding ZFS support.

This is a partial commit of the D4515.

Submitted by: Eric McCorkle
Reviewed by: emaste (in part)
MFC after: 2 weeks
X-MFC-With: r293268
Sponsored by: Multiplay
Differential Revision: https://reviews.freebsd.org/D4515

Only build EFI components on supported compilers

As the in-tree GCC does not support __attribute__((ms_abi)) EFI can only
be built with Clang.

The EFI loader and boot1 validated this, but unused libefi was still built
causing issues under GCC after warnings where enabled by r293724.

Disable building all of EFI when the selected compiler is GCC.

MFC after: 2 weeks
X-MFC-With: r293268
Sponsored by: Multiplay

Enable warnings in EFI boot code

Set WARNS if not set for EFI boot code and fix the issues highlighted by
setting it.

Most components are set to WARNS level 6 with few being left at lower
levels due to the amount of changes needed to fix at higher levels.

Error types fixed:
* Missing / invalid casts
* Missing inner structs
* Unused vars
* Missing static for internal only funcs
* Missing prototypes
* Alignment changes
* Use of uninitialised vars
* Unknown pragma (intrinsic)
* Missing types etc due to missing includes
* printf formatting types

Reviewed by: emaste (in part)
MFC after: 2 weeks
X-MFC-With: r293268
Sponsored by: Multiplay
Differential Revision: https://reviews.freebsd.org/D4839

style(9) fixes for EFI boot

Fix some style(9) nits for EFI boot code, no functional changes.

MFC after: 2 weeks
X-MFC-With: r293268
Sponsored by: Multiplay

loader.efi style(9) cleanup

Submitted by: smh

Introduce and use new EFI_ERROR_CODE macro for EFI errors

Submitted by: smh
MFC after: 1 week

loader.efi: add terminal emulation support

This is based on the vidconsole implementation.

Submitted by: Toomas Soome <tsoome@me.com>
Reviewed by: adrian
MFC after: 2 weeks
Relnotes: Yes
Differential Revision: https://reviews.freebsd.org/D4797

loader.efi: support non-contiguous console modes

Submitted by: Toomas Soome <tsoome@me.com>
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D4760

Set env vars from values on the efi loader command line.

Examine each cmdline arg and if it contains an '=' convert it to ascii and
pass it to putenv(). This allows var=value settings to come in on the
command line.

This will allow overriding dhcp server-provided data in loader(8), as
discussed in PR 202098

PR: 202098
Differential Revision: https://reviews.freebsd.org/D4561

loader.efi: strip trailing whitespace

Sponsored by: The FreeBSD Foundation

loader.efi: refresh size in GetMemoryMap retry loop

If ExitBootServices fails due to a changed efi_mapkey then GetMemoryMap
must be called again. In this case it is also possible for the memory
map to grow, so repeat the initial GetMemoryMap call to fetch the new
size.

Also roll bi_add_efi_data_and_exit into bi_load_efi_data as there's no
need for it to be a separate function.

PR: 202455
Reported by: Berislav Purgar <bpurgar@gmail.com>
Tested by: Berislav Purgar <bpurgar@gmail.com>
Reviewed by: kib
MFC after: 1 week
MFC with: r292338
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D4621

Pass the UEFI system table into the kernel from the boot loader via
the FW_HANDLE metadata item. The kernel can get the rest of what it
needs from this.

loader.efi: show EFI error number, not full status value

EFI return values set the high bit to indicate an error. The log
messages changed here are printed only in the case of an error,
so including the error bit is redundant. Also switch to decimal to
match the error definitions (in sys/boot/efi/include/efierr.h).

MFC after: 1 week
Sponsored by: The FreeBSD Foundation

UEFI: combine GetMemoryMap and ExitBootServices and retry on error

The EFI memory map may change before or during the first
ExitBootServices call. In that case ExitBootServices returns an error,
and GetMemoryMap and ExitBootServices must be retried.

Glue together calls to GetMemoryMap(), ExitBootServices() and storage of
(now up-to-date) MODINFOMD_EFI_MAP metadata within a single function.

That new function - bi_add_efi_data_and_exit() - uses space previously
allocated in bi_load_efi_data() to store the memory map (it will fail if
that space is too short). It handles re-calling GetMemoryMap() once to
update the map key if necessary. Finally, if ExitBootServices() is
successful, it stores the memory map and its header as MODINFOMD_EFI_MAP
metadata.

ExitBootServices() calls are now done earlier, from within arch-
independent bi_load() code.

PR: 202455
Submitted by: Ganael LAPLANCHE
Reviewed by: kib
MFC after: 2 weeks
Relnotes: Yes
Differential Revision: https://reviews.freebsd.org/D4296

META MODE: Update dependencies with 'the-lot' and add missing directories.

This is not properly respecting WITHOUT or ARCH dependencies in target/.
Doing so requires a massive effort to rework targets/ to do so. A
better approach will be to either include the SUBDIR Makefiles directly
and map to DIRDEPS or just dynamically lookup the SUBDIR. These lose
the benefit of having a userland/lib, userland/libexec, etc, though and
results in a massive package. The current implementation of targets/ is
very unmaintainable.

Currently rescue/rescue and sys/modules are still not connected.

Sponsored by: EMC / Isilon Storage Division

Use EFI page size constants instead of hardcoding 4096.

Reviewed by: emaste
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D3692

The EFI boot loader allocates a single chunk of contiguous memory to
hold the kernel, modules, and any other loaded data. This memory block
is relocated to the kernel's expected location during the transfer of
control from the loader to the kernel.

The GENERIC kernel on amd64 has recently grown such that a kernel + zfs.ko
no longer fits in the default staging size. Bump the default size from
32MB to 48MB to provide more breathing room.

PR: 201679
Reviewed by: imp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D3666

Various small cleanups to EFI loader Makefiles.

Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D3641

As expected, things aren't as simple as hoped. Consequently, we have
no option but to use the smbios information to fill in the blanks.
It's a good thing UGA is a protocol of the past and GOP has all the
info we need.

Anyway, the logic has been tweaked a little to get the easier bits
of information up front. This includes the resolution and the frame
buffer address. Then we look at the smbios information and define
expected values as well as the missing bits (frame buffer offset and
stride). If the values obtained match the expect values, we fill in
the blanks and return. Otherwise we use the existing detection logic
to figure it out.

Rename the environment variables from uga_framebuffer abd uga_stride
to hw.efifb.address and hw.efifb.stride. The latter names are more
in line with other variable names.

We currently have hardcoded settings for:
1. Mid-2007 iMac (iMac7,1)
2. Late-2007 MacBook (MacBook3,1)

Auto-detect the UGA frame buffer and stride on a MacBook. We're
striking a delicate balance between exhaustive searching and
banking on assumptions. The environment variables can be used
as a fall-back anyway. With this change, all known and tested
Macs with only UGA should have a working console out of the
box... for now...

My MacBook has UGA only, but we fail to detect any changes
in the frame buffer when we flip pixels. Allow the detection
to be bypassed by setting the uga_framebuffer and uga_stride
variables. The kernel console works fine even when we can't
detect pixel changes in the frame buffer, which indicates
that the problem could be with reading from the frame buffer
and not writing to it.

For UGA, the frame buffer address obtained by scanning the
PCI BARs does not necessarily correspond to the upper-left
most pixel. Scan the frame buffer for which byte changed
when changing the pixel at (0,0).

Use the same technique to determine the stride. Except for
changing the pixel at (0,0), we change the pixel at (0,1).

PR: 202730
Tested by: hartzell (at) alerce.com

Install the forth bits on arm64. For now limit it to just arm64 as on x86
these should have been installed as part of the regular loader.

Add support for the UGA draw protocol. This includes adding a
command called 'uga' to show whether UGA is implemented by the
firmware and what the settings are. It also includes filling
the efi_fb structure from the UGA information when GOP isn't
implemented by the firmware.

Since UGA does not provide information about the stride, we
set the stride to the horizontal resolution. This is likely
not correct and we should determine the stride by trial and
error. For now, this should show something on the console
rather than nothing.

Refactor this file to maximize code reuse.

PR: 202730

Add a gop command to help diagnose VT efifb problems. The gop
command has the following sub-commands:
list - list all possible modes (paged)
get - return the current mode
set <mode> - set the current mode to <mode>

Pass the ACPI table pointer to the arm64 kernel from loader.efi.

Sponsored by: The FreeBSD Foundation

Remove OUTPUT_FORMAT from theARM EFI linker script, it breaks building
for big-endian arm.

Don't check the return value from self_reloc(), it can't fail and doesn't
return a value.

Despite what I said in my prior commit, it turns out this one platform
was checking the return value from the old self-reloc code (which returned
a hard-coded 0).

The self-relocation code is not efi-specific, move it to boot/common.

The function was defined as taking 4 parameters and returning EFI_STATUS,
but all existing callers (in asm code) passed only two parameters and don't
use the return value. The function signature now matches that usage, and
doesn't refer to efi-specific types.

Parameters and variables now use the cannonical typenames set up by elf.h
(Elf_Word, Elf_Addr, etc) instead of raw C types. Hopefully this will
prevent suprises as new platforms come along and use this code.

The function was renamed from _reloc() to self_reloc() to emphasize its
difference from the other elf relocation code found in boot/common.

Differential Revision: https://reviews.freebsd.org/D2490

Add FDT to the list of known GUIDs.

Disable the use of floating-point and vector registers in the loader. They
need the vfp unit to be enabled which may not be the case.

When cross-building ${LIBSTAND} may be set to the host copy. Point to the
version built with the toolchain.

Differential Revision: https://reviews.freebsd.org/D2312
Submitted by: jhb

* Remove the wfi when the efi loader returns, it's unneeded and is not
available on older designs.
* Enable the efi loader on arm

Add support for arm64 to loader.efi and boot1.efi

Reviewed by: emaste
Sponsored by: The FreeBSD Foundation

Use MACHINE in the efi loader when it is what we mean, it may not be the
same as MACHINE_CPUARCH, it just happened to be the case the architectures
this code currently supports.

Sponsored by: The FreeBSD Foundation

Create the correct symlinks for the machine directory, and only create the
x86 symlink on i386 and amd64. Before this incorrect symlinks were being
created on armi and i386.

Differential Revision: https://reviews.freebsd.org/D2283
Reviewed by: emaste, imp
Sponsored by: The FreeBSD Foundation

Add fdt support to the arm loader.efi

Only add -fPIC to CFLAGS and -Wl,-znocombreloc to LDFLAGS on x86, they
shouldn't be used on arm.

Move reloc.c to the top level Makefile as it has become generic.

Update the arm efi ldscript to generate a valid efi binary

Add build support for i386 loader.efi

Kernel changes are required for this to be useful but this will allow
others to experiment.

Differential Revision: https://reviews.freebsd.org/D2276

Limit EFI framebuffer metadata to amd64 for now

The i386 loader.efi does not yet exist. Avoid dealing with vt and
framebuffer issues there until the initial bootstrapping is done.

Port the EFI reloc codeto work on arm64. This used the rela relocation
table so wiill need the addend included in the relocation calculation.

Sponsored by: The FreeBSD Foundation

Remove EOL whitespace from i386 EFI loader source

EFI: use common reloc.c for all architectures

Much of this file is common to the architectures we support, so share
an implementation by adding a little #ifdef-ery.

Differential Revision: https://reviews.freebsd.org/D2241
Reviewed by: imp
Sponsored by: The FreeBSD Foundation

Add support to the efi boot1 and loader for 32-bit ARM. This will be used
by the future qemu virt support.

Differential Revision: https://reviews.freebsd.org/D2238
Reviewed by: emaste

SMBIOS support for EFI.

MFC after: 1 week

Add FDT support to loader.efi. This will be used on arm and arm64.

Differential Revision: https://reviews.freebsd.org/D2219

loader/EFI: improve the help of the 'mode' command.

Only enable the efi framebuffer on x86 for now

Only include machine/specialreg.h on x86

Only enable comconsole and nullconsole on x86.

Move i386/efi files to new home in efi/loader/arch/i386

This was not (and still is not) connected to the build, but the EFI
loader is in the process of being built for other than amd64 so these
files ought to live in their eventual MD location.

Move the efi loaders to be under sys/boot/efi. This will help us add
support for booting arm and arm64 from UEFI.

Differential Revision: https://reviews.freebsd.org/D2164
Reviewed by: emaste, imp (previous version)
Sponsored by: The FreeBSD Foundation