MFC r354673:

Logging improvements to loader::nfs

Include the server IP address when logging nfs_open(), add a few missing
"\n"s, and correct a typo.

MFC various libsa fixes: r337037-r337039, r337065, r337412-r337413, r337874,
r338535, r338540, r339651, r339992-r339993, r340026

r337037:
libsa: pointer differs in signedness

A small cleanup, fix the argument type and while there, replace (char *)0 with
NULL.

r337038:
libsa: bootp is using pointers with different sign

Just change bp_file to char and same for variable s.

r337039:
libsa: assignment to char * from u_char *

Cast to char * instead of u_char *

r337065:
libsa: dereferencing type-punned pointer in cd9660

The warning is given by gcc build, but it is good to fix anyhow.
use bcopy instead of direct assignment.

r337412:
libsa: dos_checksum() should take unsigned chars

Fix pointers to integers with different sign issue.

r337413:
libsa: gzipfs.c converts pointers to integer types with different sign

Signed versus unsigned char.

r337874:
libsa: zfs_probe() needs to set spa to NULL

Silence the warning about possibly uninitialized use of spa.

r338535:
libsa: memory leak in tftp_open()

tftpfile is allocated just above and needs to be freed.

r338540:
libsa: validate tftp_makereq() after we did reset the read

The name check referred in the comment is not the only possible error source,
we need to validate the result.

r339651:
libsa: re-send ACK for older data packets in tftp

In current tftp code we drop out-of-order packets; however, we should play
nice and re-send ACK for older data packets we are receiving. This will
hopefully stop server repeating those packets we already have received.
Note we do not answer duplicates from "previous" session (that is, session
with different port number), those will eventually time out.

r339992:
libsa: tftp should not read past file end

When we have the file size via tsize option, use it to make sure we
will not attempt to read past file end.

r339993:
libsa: tftp should use calloc

instead of malloc() memset(), use calloc().

r340026:
libsa: cstyle cleanup tftp.c

No functinal changes intended.

MFC r335069-r335071, r335110, r335116: stand housecleaning

r335069:
Remove fail: label. It's unused.

r335070:
Remove unused variables.

[kevans@: removed ufs.c modifications; those depend on ufs rewrite bits]

r335071:
Initialize variables we later free so they aren't used
uninitialized in the error path.
Remove unused variables.

r335110:
Remove redundant defs.mk includes. They aren't needed.

r335116:
Use bool for vargood, since it's a boolean.

MFC r329879, r329892

r329879:
libsa: Const-ify buffer argument of write(2) analog

r329892:
libsa: Change write(2)-alike prototype to match definition

Broken in r329879.

Apparently old GCC detects this, but modern GCC didn't. Mea culpa.

MFC r325834,r325997,326502: Move sys/boot to stand/

This is effectively a direct commit to stable/11, due to differences between
stable/11 and head. Changes to DTS in sys/boot/fdt/dts were often
accompanied by kernel changes. Many of these were also risc-v updates that
likely had many more dependencies to MFC.

Because of this, sys/boot/fdt/dts remains as-is while everything else in
sys/boot relocates to stand/.

r325834: Move sys/boot to stand. Fix all references to new location

r325997: Remove empty directories.

r326502: Document the sys/boot -> stand move in hier.7 and the top-level README.

MFC r324551: Move lib/libstand to sys/boot/libsa

MFC Loader Fixes 2017q3: r320547,r320553,r321621,r321844,r321969,r321991,
r322037,r322038,r322039,r322040,r322056,r322074,r322542,r322592,r322593,
r322896,r322923,r323671,r322930,r322931,r322932,r322933,r322934,r322935,
r322936,r322937,r322938,r322939,r322941,r323062,r323063,r323064,r323065,
r323100,r323131,r323174,r323258,r323261,r323272,r323367,r323379,r323389,
r323407,r323428,r323436,r323494,r323496,r323497,r323541,r323554,r323589,
r323707,r323867,r323885,r323886,r323895,r323896,r323897,r323905,r323906,
r323907,r323908,r323909,r323952,r323991,r324099,r324558,r326445,r326609,
r326610

This batch includes a special kludge to fix powerpc loader build; <stdlib.h>
was included after <stand.h> there, causing problems with DEBUG_MALLOC bits.
Include <stdlib.h> a little bit earlier to fix the build with the intention
of removing this when eventually libsa silently replaces stdlib.h with
stand.h.

r320547: Link EFI/uboot loaders with -znotext

r320553: Integer underflow in efipart_realstrategy when I/O starts after end
of disk

r321621: Always set the receive mask in loader.efi.

r321844: Clean up style in print_state(..) and pager_printf(..)

r321969: Fix the return types for printf and putchar to match their libc

r321991: Revert r321969

r322037: Add stpcpy and stpncpy to libstand

r322038: Add definitions and utilities for EFI drivers

r322039: Move EFI ZFS functions to libefi

r322040: Add EFI utility functions to libefi

r322056: Move EFI fmtdev functionality to libefi

r322074: libefi/time.c cstyle cleanup

r322542: loader.efi: repace XXX with real comments in trap.c

r322592: Remove unused defines.

r322593: Define proposed GUID for FreeBSD boot loader variables.

r322896: Make spinconsole platform independent and hook it up into EFI
loader

r322923: Hide length of geli passphrase during boot.

r323671: Fix language used in the r322923.

r322930: Move efi_main into efi/loader

r322931: Cleanup efi_main return type

r322932: Use the loader.efi conventions for the various EFI tables.

r322933: No need for MK_ZFS around these: they are by their nature only
active when MK_ZFS is true.

r322934: _STAND is sometimes defined on the command line. Make the define
here match.

r322935: Fix warnings due to type mismatch.

r322936: Remove useless 'static' for an enum definition.

r322937: Forward declare struct dsk to avoid warnings when building libi386.

r322938: Link in libefi for boot1

r322939: Use efi_devpath_str for debug path info.

r322941: Eliminate redunant device path matching.

r323062: Make efichar.c routines available to libefi.

r323063: boot1.efi: print more info about where boot1.efi is loaded from

r323064: Exit rather than panic for most errors.

r323065: Save where we're booted from

r323100: libstand: nfs_readlink() should return proper return code

r323131: Revert r322941: Eliminate redundant device matching functions

r323174: Fix loader bug causing too many pages allocation when bootloader
is U-Boot

r323258: ucs2len

r323261: Fix armv6 build

r323272: Be consistent and do return (1);

r323367: Mark init_chroot and init_script variables as deprecated.

r323379: It's been pointed out that init_script at least is useful w/o

r323389: loader.efi: chain loader should provide proper device handle

r323407: boot1 generate-fat: generate all templates at once

r323428: r323389 breaks the kernel build when WITHOUT_ZFS is defined in
src.conf

r323436: boot1: remove BOOT1_MAXSIZE default value

r323494: loader should support large_dnode

r323496: libstand: tftp_open() can leak pkt on error

r323497: libefi: efipart_open should check the status from disk_open

r323541: libefi: efipart_realstrategy rsize pointer may be NULL

r323554: Increase EFI boot file size frok 128k to 384k

r323589: loader: biosmem.c cstyle cleanup

r323707: loader: biosmem allocate heap just below 4GB

r323867: libefi: devicename.c cleanups

r323885: libefi: efi_devpath_match() should return bool

r323886: libefi: efipart.c should use calloc()

r323895: libefi: efi_devpath_match local len should be unsigned

r323896: r323885 did miss efilib.h update

r323897: efilib.h: typo in structure member description

r323905: libefi: pdinfo_t pd_unit and pd_open should be unsigned

r323906: libefi: efipart_strategy() should return ENXIO when there is no
media

r323907: libefi: efipart.c cstyle fix for efipart_print_common()

r323908: libefi: efipart_hdinfo_add_filepath should check strtol result

r323909: libefi: define EISA PNP constants

r323952: After the r317886 support for TFTP and NFS can be enable
simultaneously.

r323991: libefi: efipart_floppy() will should not pass acpi pointer if the
HID test fails

r324099: Compile loader as Little-Endian on PPC64/POWER8

r324558: Define prototype for exit and ensure references

r326445: Fix random() and srandom() prototypes to match the standard.

r326609: Make putenv and getenv match the userland definition

r326610: Fix random() prototype to match the system.

PR: 219000 221001 222215
Relnotes: yes ("The length of the geli passphrase is hidden during boot")

MFC Loader Fixes 2017q2: r316437,r316577,r316578,r316585,r316590,r316612,
r316625,r316628,r316654,r316682,r316704,r316771,r317092,r317097,r317099,
r317652,r317785,r317886,r317887,r318142,r318320,r318356,r318678,r318754,
r318982,r318986,r318987,r318988,r318989,r318990,r318991,r318992,r318993,
r318994,r318999,r319083,r319084,r319085,r320011,r320234,r320288,r320304,
r320467,r320482

r316437: Small cleanup to make i386/loader match efi/loader boot environment
code

r316577: loader: part.c cstyle cleanup

r316578: loader: want mechanism to avoid RA with bcache

r316585: loader: zfs reader should check all labels

r316590: libstand/dosfs: cache FAT32 in 128 Kb blocks to save loader memory

r316612: In r298230 the value of HEAP_MIN was changed from 3MB to 64MB.
Correct a comment.

r316625: Do not use -msoft-float with intention of disabling FP on aarch64

r316628: Silence GCC warning by initializing the local variable.

r316654: loader: r316585 did miss userboot update

r316682: loader: r316585 did miss sparc/ofw

r316704: loader.efi: only fetch zfs pool guid for the actual boot device

r316771: loader: Avoid possible overflow via environment variable

r317092: loader: zfs reader vdev_probe should check for minimum device size

r317097: loader: F_READ/F_WRITE should be checked against masked flag

r317099: loader: uboot disk ioctl should call disk_ioctl

r317652: loader.efi: ResetSystem does not use data with EFI_SUCCESS

r317785: zfsboot: drvsize() may be unusable on some systems

r317886: distinguish NFS versus TFTP boot by rootpath

r317887: loader: network read rework

r318142: libstand: NULL pointer dereference in rarp

r318320: loader: add ip layer code into libstand

r318356: libstand: increase nfs max read size to 16k

r318678: Replacing iterating over rootpath by strsep(3).

r318754: Pass -N directly to ld via -Wl rather than passing it to the
compiler driver.

r318982: Pass a "FREEBSD" user-class in PXE dhcp request

r318986: add a comment on vendor index 19 and 20 to avoid confusion

r318987: Support URI scheme for root-path in netbooting

r318988: Always build tftpfs support along with nfs for pxeboot

r318989: Always issue the pxe request

r318990: Partially revert r314948

r318991: Document recent changes on pxeboot

r318992: Capitalize DHCP

r318993: Use the usual FreeBSD spelling for the DHCP user class

r318994: Catch with the change in the user class

r318999: Update the comments concerning net_parse_rootpath to reflect what
it is now

r319083: Followup on the user-class changes

r319084: Small cleanup in dev_net.c

r319085: use the same option list for dhcp discovery and request

r320011: Add chain loader support for loader

r320234: Make structure padding explicit in EFI_MEMORY_DESCRIPTOR

r320288: Allow Clang's integrated assembler to assemble boot0

r320304: loader.efi: Disable smbios for arm

r320467: loader: chain load relocate data declaration is bad

r320482: As with arm64 mark the EFI PE header as allocated on arm.

PR: 218473

MFC Loader Fixes 2017q1: r311458,r312237,r312314,r312374,r312947,r313042,
r313047,r313166,r313328,r313332,r313333,r313337,r313348,r313349,r313389,
r313442,r313451,r313575,r313645,r313710,r314114,r314213,r314275,r314945,
r314948,r315008,r315408,r315427,r315645,r315646,r315648,r315653,r315850,
r316064,r316078,r316079,r316100,r316104,r316111,r316112,r316171,r316279,
r316280,r316287,r316311,r316343,r316424,r316436

r311458: Use compiler driver to link BERI boot loaders

r312237: loader.efi: find_currdev() can leak memory

r312314: loader: move device path definitions to include/efidevp.h

r312374: loader: efi devpath api usage should be more aware of NULL pointers

r312947: Remove "-Xassembler -G0" from CFLAGS.

r313042: loader.efi environment related cleanups

r313047: loader: disk/part api needs to use uint64_t offsets

r313166: loader: libefi/env.c warnings in arm build

r313328: loader: Implement disk_ioctl() to support DIOCGSECTORSIZE and
DIOCGMEDIASIZE.

r313332: loader: bcache read ahead block count should take account the large
sectors

r313333: loader: Replace EFI part devices.

r313337: loader: 313329 missed ZFS guard in loader/main.c

r313348: loader: biosdisk fix for 2+TB disks

r313349: loader: disk io should not use alloca()

r313389: efipart is also using the '%S' printf format, add -Wno-format for
it.

r313442: loader: possible NULL pointer dereference in efipart.c

r313451: loader: possible NULL pointer dereference in bcache.c

r313575: makefs: make the buffer functions look exactly like the kernel ones

r313645: loader: implement MEDIA_FILEPATH_DP support in efipart

r313710: loader: cstyle fixes and DIOCGMEDIASIZE should use uint64_t

r314114: Use LDFLAGS with CC instead of _LDFLAGS.

r314213: Remove control+r handling from geliboot's pwgets()

r314275: Remove unused macro from common/drv.c.

r314945: Some style(9) fixes. No functional changes.

r314948: Try to extract the RFC1048 data from PXE.

r315008: r314948 seems to be missing a variable or two that will break

r315408: loader: remove open_disk cache

r315427: loader: biosdisk should report IO error from INT13

r315645: loader: disk_cleanup was left in userboot_disk.c

r315646: loader: pxe.h constants have wrong values

r315648: libstand: verify value provided by nfs.read_size

r315653: loader: verify the value from dhcp.interface-mtu and use snprintf
o set mtu

r315850: The original author abused Nd (one-line description, used by
makewhatis)

r316064: Fix build with path names with 'align' or 'nop' in them.

r316078: gpt*boot: Save a bit more memory when LOADER_NO_GELI_SUPPORT is
specified

r316079: Simply retire the sedification of the boot2.s file.

r316100: Remove -fno-guess-branch-probability and -fno-unit-at-a-time.

r316104: Use `NO_WCAST_ALIGN` instead of spelling it out as -Wno-cast-align
in CFLAGS

r316111: loader: move bios getsecs into time.c

r316112: loader: ls command should display file types properly

r316171: xfsread inlined uses more space, so remove the inline tag.

r316279: loader: efipart should check disk size from partition table

r316280: loader: simplify efi_zfs_probe and avoid double probing for zfs.

r316287: Remove OLD_NFSV2 from loader and libstand

r316311: Add explicit_bzero() to libstand, and switch GELIBoot to using it

r316343: Implement boot-time encryption key passing (keybuf)

r316424: Fix sparc64 build broken by r316343 and r316076

r316436: Restore EFI boot environment functionality broken in r313333

PR: 216940 217298 217935

MFC: 304677, 304680, 305125, 305770, 305769

r304677:
Add tftpfs support for the EFI loader

Allow netbooting on efi without having to setup any NFS server by rebuilding the
loader with LOADER_TFTP_SUPPORT like for the i386 pxeloader

r305125:
Netboot: allow both tftpfs and nfs in both pxeboot and loader.efi

Add a new 'netproto' variable which can be set for now to
NET_TFTP or NET_NFS (default to NET_NONE)

From the dhcp options if one sets the root-path option to:
"ip:path", the loader will consider it is booting over NFS
(meaning same behaviour as the default current behaviour)

if the dhcp option "tftp server address" is set (option 150)
the loader will consider it is booting over tftpfs, it will then
consider the root-path options with 2 possible case
1. "path" then the IP of the tftp server will be the one passed by
the option 150, and the files will be retrieved under "path" on the tftp
server
2. "ip:path" then the IP of the tftp server will be the one passed in
the option "overwritting the IP from the option 150.

We could not "abuse" the rootpath option in the form or tftp://ip:path because
this is already used for other purpose by iPXE preventing any chainload from
iPXE to the FreeBSD loader.

Given at each open(), the loader loops over all available filesystems and keep
the "best" error, we needed to prevent tftpfs to fallback on nfs and vice versa.
the tftpfs and nfs implementation in libstand now return EINVAL early if
'netproto' for that purpose.

Relnotes: yes
Sponsored by: Gandi.net

MFC 305588

pxeboot: Add nfs.read_size tunable.

Increase this tunable improves kernel loading speed.

Submitted by: Jun Su <junsu microsoft com>
Reviewed by: rpokala, wblock (previous version)
Obtained from: DragonFlyBSD
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7756

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

libstand: use NULL instead of zero for pointers.

Add a divisor parameter to twiddle() so that callers can request that output
only happen on every Nth call. Update the existing twiddle() calls done in
various IO loops to roughly reflect the relative IO sizes. That is, tftp
and nfs call twiddle() on every 1K block, ufs on every filesystem block,
so the network calls now use a much larger divisor than disk IO calls.

Also add a new twiddle_divisor() function that allows an application to set
a global divisor that is applied on top of the per-call divisors. Nothing
calls this yet, but loader(8) will be using it to further throttle the
cursor for slow serial consoles.

Move pos++ out of the complicated equation, introduced at r240780.
There is an oppinion that result of that equation is compiler-specific.

Submitted by: dt71@gmx.com, kientzle
Reviewed by: rmacklem
MFC after: 3 days

Revert r240850 and remove redundant NULL check before free(3).
free(3) handles NULL parameter fine.

Reviewed by: kib, Garrett Cooper

Avoid NULL dereference

Make nfs_readdir() more careful about using response data, cached in global
buffer. For now it fixes bug when following `ls` command will return data
from previous one aborted by pager. Also it should allow to read several
directories same time, for example, for recursive tracerse.

Don't use global nfs_root_node variable as per-file storage. There are
fields that should be file-specific.

Modify lib/libstand/nfs.c to use NFSv3 instead of NFSv2.
This allows the nfs_getrootfh() function to return the
correct file handle size to pxe.c for pxeboot. It also
results in NFSv2 no longer being used by default anywhere
in FreeBSD. If built with OLD_NFSV2 defined, the old
code that predated this patch will be built and NFSv2
will be used.

Tested by: danny at cs.huji.ac.il
MFC after: 2 weeks

ANSIfy, whitespace and comment adjustments, to simplify comparing these
files with NetBSD / OpenBSD versions.

use the correct variable in a debug message.

On i386, don't try to do network-type stuff if the device name is'nt pxeN.

Clean up error handling in libstand filesystem code to be more consistent:
- bzipfs and gzipfs now properly return errno values directly from their
read routines rather than returning -1.
- missing errno values on error returns for the seek routines on almost
all filesystems were added.
- fstat() now returns -1 if an error occurs rather than ignoring it.
- nfs's readdir() routine now reports valid errno values if an error or
EOF occurs rather than EPERM (It was just returning 0 for success and
1 for failure).
- nullfs used the wrong semantics for every function besides close() and
seek(). Getting it right for close() appears to be an accident at that.
- read() for buffered files no longer returns 0 (EOF) if an error occurs,
but returns -1 instead.

Fix a machine check abort caused by the EFI loader trying to open a
file in the NFS file system when the underlying device is not a
network device. A Sparc64 specific hack for this exact problem was
already present (nfs.c:1.9, tftp.c:1.10), but the problem is not
specific to Sparc64. The hack has been promoted to a non-i386 test
because on non-i386 architectures it's either impossible to have
non-network devices coexist in the same loader with the NFS FS, or
network and non-network device coexist and NFS filesystems can only
be used on top of network devices. I believe i386 pxeboot is where
this does not hold.

The root cause of this problem is in open.c where each file system
is tried until no more file systems exist or a file system returns
success. There's no notion of a list of valid file systems given
the underlying device and the non-existence of a file can cause
the invalid combination to be tried.

Stash various networking paramters in the environment for the kernel
to pick up, ala pxe.

Add a hack (kludge?) to avoid trying to access files backed by disk
devices as though they were backed by network devices.

Remove 'register' keyword.

Add __FBSDID()s to libstand

Expose the NFS root node, so that an evil consumer can use it to get the
NFS filehandle for the root mount.

Add readdir support to the NFS filesystem in libstand.

Do not attempt to free a nfs node if it is the root node. The root
node is statically allocated and is not guarded, so free will panic
in nfs_close.

Add a readdir function to the loader fsops vector, and implement the
functionality for some of the filesystesms.

Path arguments to *_open functions should be const, but we were mangling
them.

Submitted by: write-protected text segment in BTX

This commit was generated by cvs2svn to compensate for changes in r38451,
which included commits to RCS files with non-trunk default branches.

This is libstand; a support library for standalone executables (eg. bootstrap
modules).
Obtained from: NetBSD, with some architectural changes and many additions.