loader/lua: Remove compat shim for loader.lua_path

loader.lua_path was committed before stable/13 was branched, and merged
in to for 12.2. Remove workaround for it not being present.

Sponsored by: Netflix
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D44143

loader: lua: remove the default kernel if it doesn't exist

The `kernel` env var provides the default kernel, usually "kernel". It
may be the case that the user doesn't have a "kernel" kernel, just
"kernel.*" kernels, but have left `kernel` to the default because we
autodetect entries by default anyways.

If we're doing autodetection, take note of whether the default kernel
exists or not and remove it from the list if it doesn't and we had found
any other kernels. We avoid it in the #kernels == 1 case because
something fishy has likely happened and we should just trust the
configuration.

Reviewed by: imp, manu
Differential Revision: https://reviews.freebsd.org/D42967

loader: lua: assume late ACPI detection if the feature isn't enabled

While we're here, enable the feature in the places we detect ACPI. This
lets us side-step the existing issues and provide a path forward for
folks upgrading from previous releases that haven't updated their ESP
yet.

Let's also fix core.setACPI: the hint already indicates that the
user's disabled it more consistently than loader.acpi_disabled_by_user.
Even more, the latter is wrong because we set it by default if we did
not detect ACPI. The ACPI hint remains even when we're setting defaults
because ACPI loaded into the kernel will make some noise if it's not
hinted off, even when we didn't detect it.

imp notes that this will result in some relatively harmless noise on
platforms that don't support ACPI but aren't using the UEFI loader, as
we would enable the ACPI module for loading on them and then loader
would not be able to find it. These are non-fatal, but should probably
be fixed by just declaring support for EARLY_ACPI in those loaders since
we know they won't have ACPI early on -- punting on this for the time
being, though, in favor of providing a safer upgrade path sooner.

Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D42727

loader: provide a features table for binary compatibility advertisement

liblua now provides a loader.has_feature() function to probe the loader
binary for features advertised. name => desc mappings are provided in
loader.features to get a list of all of the features loader *can*
support. core.hasFeature is provided as a shim to loader.has_feature
so that individual consumers don't need to think about the logic of the
loader module not providing has_feature; we know that means the feature
isn't enabled.

The first consumer of this will be EARLY_ACPI to advertise that the
loader binary probes for ACPI presence before the interpreter has
started, so that we know whether we can trust the presence of acpi.rsdp
as relatively authoritative. In general, it's intended to be used to
avoid breaking new scripts on older loaders within reason.

This will be used in lua as `core.hasFeature("EARLY_ACPI")`, while the
C bits of loader will `feature_enable(FEATURE_EARLY_ACPI)`.

Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D42695

stand: Retire setting hw.ata.wc: it doesn't exist.

hw.ata.wc was disconnected as part ot the 2013 cam-ification of ata. No
need to continue setting it. It's been unused in FreeBSD 10.x and newer.

Sponsored by: Netflix

stand: Retire setting hw.eisa_slots.

When the eisa code was removed in 2017, prior to the stable/12 branch,
setting hw.eisa_slots became a nop. The oldest supported branch doesn't
have eisa at all. The need to set it manually on boot disappeared
largely by 2000...

Sponsored by: Netflix

stand: bandaide for acpi

Old binaries do not set acpi.rsdp early enough. So when we boot with an
older loader.efi from an ESP that's not been updated, we assume there's
no ACPI on this system. This is unwise. Put a band-aide on this until we
can implement a proper 'feature' variable that the binary reports so we
can do conditionals for things like this in the future.

This is at best a rapid-response stop-gap.

Glanced at by: kevans
Sponsored by: Netflix

loader: improve lua ACPI detection and handling

This is a follow-up patch to https://reviews.freebsd.org/D42459
that modifies the loader lua to use the correct loader variables
for determining ACPI availability.

This also fixes a bug where ACPI can be inadvertently disabled when
setting System Defaults at the loader menu.

Reviewed by: imp, kevans
Approved by: kp
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D42483

Remove $FreeBSD$: two-line lua tag

Remove /^--\n--\s*\$FreeBSD\$.*$\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

Add support for getting early entropy from UEFI

UEFI provides a protocol for accessing randomness. This is a good way
to gather early entropy, especially when there's no driver for the RNG
on the platform (as is the case on the Marvell Armada8k (MACCHIATObin)
for now).

If the entropy_efi_seed option is enabled in loader.conf (default: YES)
obtain 2048 bytes of entropy from UEFI and pass is to the kernel as a
"module" of name "efi_rng_seed" and type "boot_entropy_platform"; if
present, ingest it into the kernel RNG.

Submitted by: Greg V
Reviewed by: markm, kevans
Approved by: csprng (markm)
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D20780

loader: lua: test that /boot exists first

Otherwise on fs like tftp where no directory listing is possible we fail
on the .dir method.

Reviewed by: imp, kevans
MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D33414

loader_lua: consider userboot console as serial

We use ascii box chars with serial console because we do not know
if terminal can draw unixode box chars. Same problem is about userboot
console.

MFC after: 5 days

stand: lua: enhance lfs.dir() to speed up kernels_autodetect

This eliminates a lot of stat() calls that happen when lualoader renders the
menu with the default settings, and greatly speeds up rendering on my
laptop.

ftype is nil if loader/loader.efi hasn't been updated yet, falling back to
lfs.attributes() to test.

This is technically incompatible with lfs, but not in a particularly
terrible way.

Reviewed-by: cem
MFC-after: 4 days
Differential Revision: https://reviews.freebsd.org/D27542

loader: implement framebuffer console

Draw console on efi.
Add vbe framebuffer for BIOS loader (vbe off, vbe on, vbe list,
vbe set xxx).
autoload font (/boot/fonts) based on resolution and font size.
Add command loadfont (set font by file) and
variable screen.font (set font by size). Pass loaded font to kernel.

Export variables:
screen.height
screen.width
screen.depth

Add gfx primitives to draw the screen and put png image on the screen.
Rework menu draw to iterate list of consoles to enamble device specific
output.

Probably something else I forgot...

Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D27420

lualoader: clear up some luacheck warnings

- One (1) unused argument
- One (1) trailing whitespace
- Two (2) "non-standard global" (curenv, rewind)

tools/boot/lua-lint.sh is once again happy.

Report the kernel console on the boot screen

Report what console the boot loader is telling the kernel to use:
o Dual (Serial Primary)
o Dual (Video Primary)
o Serial
o Video

This allows the user to interrupt the boot and tweak the cosnole, if
needed, in a trivial way. Useful for installs where the default
selected may not be quite what you want, or when you are running a
dual setup and need to toggle over to the other console being primary.
The 'c'/'C' keys will do the cycling through the consoles. Note:
you'll still have to drop into the loader to set details about serial
consoles. And this doesn't change the console the loader is using.

Reviewed by: kevans@
MFC After: 3 days
Differential Revision: https://reviews.freebsd.org/D26573

zfs: add an option to the bootloader to rewind the ZFS checkpoint

The checkpoints are another way of keeping the state of ZFS.
During the rewind, the pool has to be exported.
This makes checkpoints unusable when using ZFS as root.
Add the option to rewind the ZFS checkpoint at the boot time.
If checkpoint exists, a new option for rewinding a checkpoint will appear in
the bootloader menu.
We fully support boot environments.
If the rewind option is selected, the boot loader will show a list of
boot environments that existed before the checkpoint.

Reviewed by: tsoome, allanjude, kevans (ok with high-level overview)
Differential Revision: https://reviews.freebsd.org/D24920

loader: Fully reset terminal settings, not just colors

Reviewed by: kevans
Reviewed by: tsoome
Approved by: mav (mentor)
MFC after: 1 week
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D21733

lualoader: correct a typo from r354247

r354247 converted try_include to lfs + dofile with the loader.lua_path added
just before. Fortunately, there was a hardcoded /boot/lua fallback in case
loader.lua_path wasn't being set yet- I typo'd it as loader.lua_paths.

Fix the typo.

X-MFC-With: r354247
MFC after: 3 days

lualoader: rewrite try_include using lfs + dofile

Actual modules get require()'d in, rather than try_include(). All instances
of try_include should be provided with proper hooks/API in the rest of
loader to do the work they need to do, since we can't rely on them to exist.
Convert this now to lfs + dofile since we won't really be treating them as
modules.

lfs is required because dofile will properly throw an error if the file
doesn't exist, which is not in the spirit of 'optionally included'.

Getting out of the pcall game allows us to provide a loader.exit() style
call that backs out to the common bits of loader (autoboot sequence unless
disabled with a loader.setenv("autoboot_delay", "NO")). The most ideal way
identified so far to implement loader.exit() is to throw a special
abort-style error that indicates to the caller in interp_lua that we've not
actually errored out, just continue execution. Otherwise, we have to hack in
logic to bubble up and return from loader.lua without continuing further,
which gets kind of ugly depending on the context in which we're aborting.

A compat shim is provided temporarily in case the executing loader doesn't
yet have loader.lua_path, which was just added in r354246.

lualoader: Revert to ASCII menu frame for serial console

The box drawing characters we use aren't necessarily safe with a serial
console; for instance, in the report by npn@, these were causing his xterm
to send back a sequence that lua picked up as input and halted the boot.
This is less than ideal.

Fallback to ASCII frames for console with 'comconsole' in it. This is a
partial revert r338108 by imp@ -- instead of removing the menu entirely and
disabling color/cursor sequences, just reverting the default frame to ASCII
is enough to not break in this setup.

Reported by: npn
Triaged and recommended by: tsoome

lualoader: Fix try_include error handling

The previous iteration of try_include attempted to be 'friendly' and error()
out if we hit an error that wasn't ENOENT. This was semi-OK, but fragile as
it relied on pattern matching the error message.

Move the responsibility for handling failure to the caller. Following
a common lua pattern, we'll return the return value of the underlying
require() on success, or false and an error message.

Reported by: bcran
MFC after: 3 days

lualoader: Honor boot_* variables at lua init

For non-UEFI systems, boot.config(5) may have -s or -v specified for
single-user and verbose boot respectively. These were not being properly
taken into account and reflected in the "Boot Options" submenu. When we
initialize core.lua, we'll record boot_single and boot_verbose as we do ACPI
and consider these the system defaults.

Reported by: David Wolfskill <david@catwhisker.org>
Approved by: re (kib)

lualoader: Handle comma-separated kernels as well

The format for kernels is documented as being space-delimited, but
forthloader was more lenient on this and so people began to depend on it.

A later pass will be made to document all of the fun features that forthloader
allowed that may not be immediately obvious.

Reported by: mmacy
Approved by: re (kib)

lualoader: Just compare expression directly

Serial console menus for lua.

Remove a bunch of special cases for UEFI and serial consoles. We do
want to do curses and menu things here. This makes us match what we do
in FORTH, with the possible exception of boxes around menus.

Differential Revision: https://reviews.freebsd.org/D16816

lualoader: Actually re-raise error in try_include

It was previously only printed, but we do actually want to raise it as a
full blown error so that things don't look OK when they've actually gone
wrong.

The second parameter to error, level, is set to 2 here so that the error
message reflects the position of the try_include caller, rather than the
try_include itself. Example:

LUA ERROR: /boot/lua/loader.lua:46: /boot/lua/local.lua:1: attempt to call a
nil value (global 'cxcint').

lualoader: Implement try_include and use it for including the local module

This provides a way to optionally include a module without having to wrap it
in filesystem checks. try_include is a little more robust, using the lua
search path instead of forcing us to explicitly consider all of the places
we could want to include a module. Errors are still generally raised from
trying to load the module, but ENOENT will not get raised unless we're doing
a verbose load.

This will also be used to split out logo/brand graphics into their own files
so that we can safely scale up the number of graphics included without
worrying about the extra memory consumption- opting to lazily load graphics
instead.

Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D14658

UEFI: Ditch console mode setting, choose optimal GOP mode later in boot

boot1 is too early to be deciding a good resolution. Console modes don't map
cleanly/predictably to actual screen resolutions, and GOP does not reflect
the actual screen resolution after a console mode change. Rip it out.

Add an efi-autoresizecons command to loader to choose an optimal screen
resolution based on the current environment. We'll explicitly execute this
later, preferably before we draw anything of value but after we load config
and pick up any tunables we may need to decide where we're going.

This method also allows us to actually pass the correct framebuffer
information on to the kernel.

UGA autoresizing is not implemented because it doesn't have the kind of mode
enumeration that GOP does. If an interested person with relevant hardware
could get in contact, we can take a look at implementing UGA autoresize.

This effectively "fixes" the breakage caused by r327058, but doesn't
actually set the resolution correctly until the interpreter calls
efi-autoresizcons. The lualoader version of this has been included for
reference; the forth equivalent will follow.

Reviewed by: imp (with some hestitation), manu
Differential Revision: https://reviews.freebsd.org/D14788

lualoader: Add primitive hook module, use it to untangle bogus reference

See: comments in the hook module about intended usage, as well as the
introduced use for config.reloaded.

Use the newly introduced hook module to define a "config.reloaded" hook.
This is currently used to register core's clearKernelCache as a reload hook
to avoid a circular dependency and fix this functionality- it didn't
actually work out, and it isn't immediately obvious how it slipped into src.

Other hook types will be introduced into the core lualoader as useful hook
points are identified.

lualoader: Cache kernel list

With autodetection turned on, hitting the filesystem everytime we need to
calculate choices for the kernel carousel is kind of slow. Cache once on the
first listing and reload it anytime the config is reloaded in case any of
the loader.conf(5) changes that affect this (kernel, kernels,
kernels_autodetect) have changed. This also picks up the case where we've
changed currdev and the autodetected kernels could change.

lualoader: Only loadelf before boot/autoboot if no kernel loaded

Back when I "fixed" the loading of kernel/modules to be deferred until
booting, I inadvertently broke the ability to manually load a set of kernels
and modules in case of something bad having happened. lualoader would
instead happily load whatever is specified in loader.conf(5) and go about
the boot, leading to a panic loop as you try to rediscover a way to stop the
panicky efirt module from loading and fail miserably.

Reported by: me, sadly

lualoader: Add note that \027 is a decimal representation

We've included an extra '0' in there (which might get removed later, but
it's maintained for the moment for legacy purposes) which oftentimes
indicate that the following number should be treated as octal. This is not
the case, so note that to prevent future confusion (of myself and others).

lualoader: Convert instances of KEYSTR_ESCAPE .. "[" -> KEYSTR_CSI

lualoader: Re-work menu skipping bits

This is motivated by a want to reduce heap usage if the menu is being
skipped. Currently, the menu module must be loaded regardless of whether
it's being skipped or not, which adds a cool ~50-100KB worth of memory
usage.

Move the menu skip logic out to core (and remove a debug print), then check
in loader.lua if we should be skipping the menu and avoid loading the menu
module entirely if so. This keeps our memory usage below ~115KB for a boot
with the menu stripped.

Also worth noting: with this change, we no longer explicitly invoke autoboot
if we're skipping the menu. Instead, we let the standard loader behavior
apply: try to autoboot if we need to, then drop to a loader prompt if not or
if the autoboot sequence is interrupted. The only thing we still handle
before dropping to the loader autoboot sequence is loadelf(), so that we can
still apply any of our kernel loading behavior.

lualoader: More argument name expansion, part 2

screen also has some instances, but it also has other cleanup to go with it.
Because of this, I will be committing the screen changes separately.

lualoader: Clean up naming conventions a little bit

We mostly use camel case for function names, but some local functions got
mixed in using internal underscores. Doubles down on camel case.

lualoader: Use "local function x()" instead of "local x = function()"

The latter is good, but the former is more elegant and clear about what 'x'
is. Adopt it, preferably only using the latter kind of notation where needed
as values for tables.

lualoader: shallowCopyTable => deepCopyTable

I called it a shallow copy, but it wasn't really a shallow copy at all.

Add copyright notice to core.lua

I've also made some not-insignificant changes/additions to this file, to
include the added constants, ACPI changes, boot environment listing, and
some utility functions.

Add SPDX tags to lua files

lualoader: Address some 'luacheck' concerns

luacheck pointed out an assortment of issues, ranging from non-standard
globals being created as well as unused parameters, variables, and redundant
assignments.

Using '_' as a placeholder for values unused (whether it be parameters
unused or return values unused, assuming multiple return values) feels clean
and gets the point across, so I've adopted it. It also helps flag candidates
for cleanup later in some of the lambdas I've created, giving me an easy way
to re-evaluate later if we're still not using some of these features.

lualoader: Consistently use double quotes

lualoader: Split cli bits out into a cli module

This module will, in the not-so-distant future, grow functionality for
reducing boilerplate in functions that implement cli commands. It will
likely also house most in-tree cli commands.

lualoader: Make kernel autodetection contingent on loader.conf(5) var

Instead of based it off of whether 'kernels' was specified, base it off of a
new variable: kernels_autodetect. If set to yes, we'll run the autodetection
bits and add any detected kernels to the already existing list *after* both
'kernel' and 'kernels'.

lualoader: Add boot environment support

This looks a little bit differently than the forth version for the time
being, just to get off the ground- rather than a paging system, it's
implemented as a simple carousel like the kernel selector.

Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D14436

lualoader: Don't autodetect kernels if 'kernels' is explicitly set

lualoader: Return only argstr if with_kernel not requested

lualoader: Drop explicit boolean tests; b or not b

lualoader: Drop excessive parenthesizing

This was also a convenience convention (for me) that is not very lua-tic.
Drop it.

I've maintained some parentheses where I'd prefer them, for example,
'if x or y or (z and w) then', but these situations are far and few between.

lualoader: Drop terminating semicolons

This was previously chosen out of convenience, as we had a mixed style and
needed to be consistent. I started learning Lua on Friday, so I switched
everything over. It is not a very lua-nic convention, though, so drop it.

Excessive parenthesizing around conditionals is next on the chopping block.

lualoader: Intercept the 'autoboot' cli command

lualoader: Intercept boot cli command

This should be functional and roughly equivalent to the Forth version.

Stop doing a loadelf() on menu exit now that we can DTRT with boot
invocations. autoboot interception will follow not long after.

lualoader: Prepare for interception of "boot" CLI cmd

core.boot and core.autoboot may both take arguments; add a helper to cleanly
append an argstring to the given loader command.

Also provide a popFrontTable() that we'll use pop the command name off of an
argv table. We don't have the table library included, and including it is
non-trivial, so we'll implement this one function that we need in lua for
the time being.

lualoader: Ignore ACPI bits on !i386

stand/lua: Consistently organize modules

We follow pretty closely the following structure of a module:

1. Copyright notice
2. Module requires
3. Module local declarations
4. Module local definitions
5. Module exports
6. return

Re-organize the one-offs (config/drawer) and denote the start of module
exports with a comment.

stand/lua: Don't set ACPI off just because we can't detect it.

This should unbreak EFI/!i386 cases.

Reported by: Peter Lei <peter.lei@ieee.org>

stand/lua: Change boot menu items' names when swapped

[Enter] should be moved to the single user menu item when we swap them.

Define a non-standard menu entry function "alternate_name" to use for this
purpose for ultimate flexibility if we change our minds later. When we're
booting single user, make a shallow copy of the menu that we'd normally
display and swap the items and their name functions to use alternate_name
instead. Toggling single user in the options menu and going back to the main
menu will now correctly reflect the current boot setting with the first two
menu options and "[Enter]" will always be on the right one.

This shallow copy technique has the chance of being quite slow since it's
done on every redraw, but in my testing it does not seem to make any obvious
difference.

shallowCopyTable could likely belong better in a general-purpose utility
module, but this (and the key constnats) are the only candidates we have at
the moment so we'll drop it into our core stuff for the moment and consider
re-organization at a later date.

stand/lua: Add core.isSingleUserBoot

stand/lua: Rename bootserial for clarity

stand/lua: Defer kernel/module loading until boot or menu escape

Loading the kernel and modules can be really slow. Loading before the menu
draws and every time one changes kernel/boot environment is even more
painful.

Defer loading until we either boot, auto-boot, or escape to loader prompt.
We still need to deal with configuration changes as the boot environment
changes, but this is generally much quicker.

This commit strips all ELF loading out of config.load/config.reload so that
these are purely for configuration. config.loadelf has been created to deal
with kernel/module loads. Unloading logic has been ripped out, as we won't
need to deal with it in the menu anymore.

Discussed in part with: allanjude

lua loader: Auto detect eligible list of kernels to boot

Reviewed by: imp, kevans
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D14419

stand/lua: Style pass

These are the style points that I'd like to try and maintain in our lua
scripts:
- Parentheses around conditionals
- Trailing semicolons, except on block terminators
- s:method(...) instead of string.method(s, ...) where applicable

There's likely more, but that'll get hammered out as we continue.

stand/lua: Correct usage and acceptance of BACKSPACE/DELETE keys

stand/lua: Remove sneaky kernel assignment

stand/lua: Create/use some MENU_ constants where applicable

stand/lua: Set ACPI's default the proper way (setACPI)

stand/lua: Remove a magic number/string (not a trivial literal)

We'll arbitrarily use KEYSTR_ for string representations of non-trivial
characters.

stand/lua: Consistently use semicolons for line endings

stand/lua: Set reasonable ACPI default based on presence

Set it based on hint.acpi.0.rsdp. Initially, hint.acpi.0.disabled will be
respected. "Using System Defaults" will override whether it's explicitly
disabled by hint and re-load it based on whether it's present on the system.

Unlike the 4th version, this is not restricted to x86. I have no strong
reasoning for this, so this is definitely open to change.

stand/lua: Reduce magic numbers

Enter/backspace values are hardcoded in both the menu and password scripts.
Separate these out to core for reuse between the two.

Add the lua scripts from the lua-bootloader SoC

These are the .lua files from from Pedro Souza's 2014 Summer of Code
project. Rui Paulo, Pedro Arthur and Wojciech A. Koszek also
contributed.

Obtained from: https://wiki.freebsd.org/SummerOfCode2014/LuaLoader
Sponsored by: Google Summer of Code

Improve the SoC lua menu code to bring it in line with forth
menu functionality

Submitted by: Zakary Nafziger
Sponsored by: FreeBSD Foundation

Use loader.setenv and loader.unsetenv instead of loader.perform

Convert from include("/boot/foo.lua") to foo = require("foo");
to bring in line with latest lua module conventions.

Enforce a uniform style for the new .lua files:
o hard tab indenation for 8 spaces
o don't have if foo then bar; else bas; end on one line

MFC After: 1 month
Relnotes: yes
Differential Review: https://reviews.freebsd.org/D14295