Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r235329,235343,235361,235364: zfsboot/zfsloader: support accessing
filesystems within a pool

MFC: r236579

The workaround added in r151650 for handling firmwares that don't allow
a single device to be opened multiple times concurrently unfortunately
isn't sufficient with ZFS. This is due to the fact, that ZFS may open
different partitions of a single device simultaneously. So the best we
can do in this case is to cache the lastly used device path and close
and open devices in ofwd_strategy() as needed.

PR: 165025
Submitted by: Gavin Mu

MFC: r234898, r235207

Add initial support for booting from ZFS on sparc64. At least on Sun Fire
V100, the firmware is known to be broken and not allowing to simultaneously
open disk devices, causing attempts to boot from a mirror or RAIDZ to cause
a crash. This will be worked around later. The firmwares of newer sun4u models
don't seem to exhibit this problem though.

PR: 165025
Submitted by: Gavin Mu

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

Check the OF_getprop() return value before proceeding. Allocate only as
much space as needed for the mode buffer. Use strcmp, relying on OF giving
back NULL terminated strings.

Submitted by: marius
Approved by: nwhitehorn (mentor)

Check the real-mode? OF property to find out whether we operate in real or
virtual mode. In virtual mode we have to do memory mapping. On PowerMacs it is
usually false while on pSeries we have found that it is true. The real-mode?
property is not available on sparc64.

Approved by: nwhitehorn (mentor)

Move the declaration of the eh struct (used only when debugging is enabled)
from ofwn_put into the debug section.

Approved by: nwhitehorn (mentor)

Fix the printf() in init_heap so that it can run before the console is up.

Pointed out by: marius

Fix netboot on some Apple machines on which calling dma-free on the
network device can hang the machine. This causes the loss of 64 KB of
accessible memory on netbooted machines.

Fix some memory management issues discovered when trying to boot the PPC
OF loader on systems where address cells and size cells are both 2 (the
Mambo simulator) and fix an error where cons_probe() was called before
init_heap() but used malloc() to set environment variables.

MFC after: 1 month

In the case of non-sequential mappings, ofw_mapmem() could ask Open
Firmware to map a memory region with negative length, causing crashes
and Undefined Behavior. Add the appropriate check to make the behavior
defined.

MF tbemd: Minor tweaks, prefer MACHINE_CPUARCH generally to MACHINE_ARCH (which simplifies some powerpc/powerpc64 ifs)

Provide support in loader for booting 64-bit PowerPC kernels. Like amd64,
64-bit PowerPC kernels are loaded by a 32-bit loader, since nearly all
powerpc64 firmwares execute in 32-bit mode.

(S)LIST_HEAD_INITIALIZER takes a (S)LIST_HEAD as an argument.
Fix some wrong usages.
Note: this does not affect generated binaries as this argument is not used.

PR: 137213
Submitted by: Eygene Ryabinkin (initial version)
MFC after: 1 month

Don't close OF disk devices on PowerPC. This fixes loader when booting from
disk on my Blue & White G3 system.

Backout rev 183181. It appears that I should have been using boot-device
of "cd:,\\:tbxi" with properly configured boot.tbxi, instead of booting
\boot\loader directly. Rev 183168 could probably stay, since it can be
viewed as an anti-foot-shooting measure and has no impact on normal
operation. I can revert it as well, if anybody objects.

On PowerPC send output to both "/chosen/stdout" and "screen" nodes, unless
they point to the very same device. This should make loader usable on
some (all?) PowerMacs, where "/chosen/stdout" is disconneted from the
"screen" by the OF init process by default, except when user actually
has requested interaction with OF by holding ALT-CMD-O-F. Along with
rev 183168 this should provide a way to build bootable FreeBSD/ppc
installation or live CD that works OOB. Also, it should bring PowerMac
experience closer to that on other arches.

MFC after: 1 week
(assiming re@ blessing)

Enable GCC stack protection (aka Propolice) for userland:
- It is opt-out for now so as to give it maximum testing, but it may be
turned opt-in for stable branches depending on the consensus. You
can turn it off with WITHOUT_SSP.
- WITHOUT_SSP was previously used to disable the build of GNU libssp.
It is harmless to steal the knob as SSP symbols have been provided
by libc for a long time, GNU libssp should not have been much used.
- SSP is disabled in a few corners such as system bootstrap programs
(sys/boot), process bootstrap code (rtld, csu) and SSP symbols themselves.
- It should be safe to use -fstack-protector-all to build world, however
libc will be automatically downgraded to -fstack-protector because it
breaks rtld otherwise.
- This option is unavailable on ia64.

Enable GCC stack protection (aka Propolice) for kernel:
- It is opt-out for now so as to give it maximum testing.
- Do not compile your kernel with -fstack-protector-all, it won't work.

Submitted by: Jeremie Le Hen <jeremie@le-hen.org>

Make the openfirmware getchar entry point non-blocking. This catches up
with jhb's 2005/05/27 loader multiple-console change.

Tested by: marius/sparc64, grehan/ofwppc

- Make better use of the global chosen, memory and mmu handles instead
of obtaining them over and over again and pretending we could do
anything useful without them (for chosen this includes adding a
declaration and initializing it in OF_init()).
- In OF_init() if obtaining the memory or mmu handle fails just call
OF_exit() instead of panic() as the loader hasn't initialized the
console at these early stages yet and trying to print out something
causes a hang. With OF_exit() one at least has a change to get back
to the OFW boot monitor and debug the problem.
- Fix OF_call_method() on 64-bit machines (this is a merge of
sys/dev/ofw/openfirm.c rev 1.6).
- Replace OF_alloc_phys(), OF_claim_virt(), OF_map_phys() and
OF_release_phys() in the MI part of the loader with wrappers around
OF_call_method() in the sparc64. Beside the fact that they duplicate
OF_call_method() the formers should never have been in the MI part
of the loader as contrary to the OFW spec they use two-cell physical
addresses.
- Remove unused functions which are also MD dupes of OF_call_method().
- In sys/boot/sparc64/loader/main.c add __func__ to panic strings as
different functions use otherwise identical panic strings and make
some of the panic strings a tad more user-friendly instead of just
mentioning the name of the function that returned an unexpected
result.

Sync the styles of sys/boot/ofw/libofw/openfirm.c and
sys/dev/ofw/openfirm.c with themselves, with each-other
and with style(9).

Extend struct devdesc with a unit field, called d_unit. Promote the
device (kind) specific unit field to the common field. This change
allows a future version of libefi to work without requiring anything
more than what is defined in struct devdesc and as such makes it
possible to compile said version of libefi for different platforms
without requiring that those platforms have identical derivatives
of struct devdesc.

Don't unconditionally compile-in the bcache code. It's only used on
i386/amd64 and pc98. Remove useless calls to bcache_init() from the
ia64 and sparc64 loaders, as well as from the OFW common code.

more sun4v fallout - missed add
Approved by: rwatson (mentor)

add sun4v support to the sparc64 boot loader

Approved by: rwatson (mentor)
Reviewed by: jmg
Tested by: kris, dwhite, and jmg

- Add a workaround for the fact that OFW doesn't guarantee that
devices can be opened multiple times simultaneously but we're
expected to be able to do so by the rest of the loader.
This fixes booting from disks attached to the on-board SCSI
controller of Sun Ultra 1 (previously this triggered a trap)
and probably also of AX1115 boards.
- While here, remove unused variables and add empty lines where
style(9) requires such.

Tested on: powerpc (grehan), sparc64
MFC after: 1 month

In ofw_parsedev() check the return value of malloc() and protect
against a NULL pointer dereference when ofw_parsedev() is called
with a NULL path argument.

Tested on: powerpc (grehan), sparc64

Use an ihandle_t to store the instance handle of an opened device
instead of a phandle_t (package handle). Since both are typedefed
to unsigned int, this is more or less cosmetic.

Remove unused variables.

Make code match comment: make the smallest unit of page allocation
from OpenFirmware be 16 pages to avoid fragmentation in the list
of mappings returned when the kernel requests it in pmap_bootstrap.

This allows a static buffer to be used when obtaining the existing
mappings - very useful on the G5 when random physical pages can't
be grabbed because they can't be BAT-mapped.

MFC after: 3 days

Since the BAT mapping was removed a long time back, it can't
be assumed that modules are contiguous in memory (they're not)
so don't blindly __syncicache start/end. In fact, don't bother
syncing the icache for modules since the kernel will do it after
fixing up relocations.

This fixes the trap when loading modules at boot time.

Reported by: orlando at break dot net

Change ofw_readin/ofw_copyin to map the entire region before
copying, rather than a page at a time. This was creating far
too many single-page mappings, and eventually OFW overflowed
some internal data structure and refused to map any more.
The new algorithm creates far less mappings and fixed a bug
where multiple mappings for the same page would be created.

'Twas known this was a problem, but only became urgent when the
install CD's mfs_root grew large enough to cause the overflow.

Remove unused variables.

Start each of the license/copyright comments with /*-

For variables that are only checked with defined(), don't provide
any fake value.

Instead of "OpenFirmware", "openfirmware", etc. use the official spelling
"Open Firmware" from IEEE 1275 and OpenFirmware.org (no pun intended).

Ok'ed by: tmm

G5 support: handle the case where the OpenFirmware memory array uses
64 bits for the phys address, but only 32 for the virtual address.

All hell breaks loose if the loader image is released before
jumping to the kernel. Another bug exposed by removing the
1:1 BAT mapping. Sparc64 doesn't do this either.

Compile tested on: panther (sparc64). Code built, but not used, on sparc64.

Correctly calculate size of memory to be mapped when copying. Removal
of the 256Mb 1:1 BAT mapping exposed this as copying into memory that
hadn't been claimed from OpenFirmware.

compiled-tested on: panther (sparc64). Code built, but not used, on sparc64

Extract entry point from elf header before releasing the heap.
This caused a data-miss trap when the BAT init code was removed
on PPC.

Compile tested on: panther (sparc64). Code built, but not used, on sparc64.

Convert to __FBSDID.

- Add ':' as a separator between the OpenFirmware device space and
the file path. Commonly used on Macs e.g. "hd:9".
- Update the ofw_setcurrdev routine to match libstand setenv prototype

Not objected to by: sparc64

Make __elfN(ofw_loadfile) match parameter declaration for file_format
in boot/common/bootstrap.h. Having a 32-bit size when a 64-bit param
is declared wreaks havoc on PPC.

Not objected to by: sparc64

Only print out an error if returned data size is < 0. A value of 0
happens almost every time at the end of a file when using NFS.

No objection by: sparc64

Use daddr_t instead of u_long for byte offset in strategy
routine to avoid >4G truncation on 32-bit systems.

no objection by: sparc64

- use correct pointer arithmetic in heapsize calculation
- handle multiple Ofw memory regions when determining mem size
- allow currdev to be set as a loader command-line option.
parse() is used to allow future options to be processed.

Enable the i386 loader to load and run an amd64 kernel. If this puts
things over floppy size limits, I can exclude it for release builds or
something like that. Most of the changes are to get the load_elf.c file
into a seperate elf32_ or elf64_ namespace so that you can have two
ELF loaders present at once. Note that for 64 bit kernels, it actually
starts up the kernel already in 64 bit mode with paging enabled. This
is really easy because we have a known minimum feature set.

Of note is that for amd64, we have to pass in the bios int 15 0xe821
memory map because once in long mode, you absolutely cannot make VM86
calls. amd64 does not use 'struct bootinfo' at all. It is a pure loader
metadata startup, just like sparc64 and powerpc. Much of the
infrastructure to support this was adapted from sparc64.

Cut&Paste considered far too easy:

Don't include <sys/disklabel.h>

Fix a dumb bug that broke net booting on sparc64. The wrong length was
passed to strncmp.

Noticed by: tmm
Approved by: re
Pointy hat to: jake

Change the device path representation in libofw to use the full firmware
path, instead of an internal i386 specific one. Don't try to interpret
a disklabel in ofw_disk.c, open the partition's device node directly and
let the firmware do it. This fixes booting from a partition other than 'a'
on sparc64, which is needed to support more installation methods.

No objection: ppc

Major rework of how we copy data into kernel space.

We now talk to the memory and mmu instances directly rather than using the
OpenFirmware "claim" method.

Add support for passing metadata.

Fix up the DMA buffer allocation call.

Removed now unused INTERNALSTATICLIB.
INTERNALLIB now implies NOPIC and NOPROFILE.
Removed gratuitous NOMAN.

Back out last commit. I expect our bsd.*.mk gods to remove the need for
defining so many extra things in addition to INTERNALLIB. We don't like
repetitive C code and we shouldn't for make code either.

Change the disk probing so that it will actually find disks other
than the first one on a controller, and work for secondary
controllers.
Due to the prom not having nodes for each disk, but a catch-all one,
we have to iterate over each device, trying to open it to determine
whether it is actually present.
Since probing this way takese some time (and spews some spurious
warnings), it should maybe be short-circuited if we use the
device we were booted from.
Implement lazy device probing, and correct slice/partiniton
handling in the ofwd_open() code. With this, I can now actually boot
a kernel from disk, and the loader does not create unnecessary
delays.

Submitted by: tmm

NOPIC, NOPROFILE, NOMAN, and INTERNALSTATICLIB are redundant when using
INTERNALLIB now.

Add -ffreestanding to CFLAGS.

Fix test for != 0 that should be > 0.

Check the return values of index() and don't dereference it if its null;
the path may not have a ':' in it. The mac address property is called
mac-address for sparc64. Don't allocate a dma buffer for sparc64 either.

#if 0 more bootinfo stuff.

Add OF_claim_virt() and OF_release_phys().

Submitted by: tmm

- Remove unneeded variable.
- Remove change for my local configuration that slipped in with
the last commit; I am having problems booting when multiple SCSI
disks are attached, so I will change this part as soon as I find
a solution, anyway.

This file is now unneeded.

- Do not include a NetBSD header as it is not needed anymore.
- Remove two constants that were needed in conjuction with the
NetBSD disklabel header. Use the FreeBSD equivalents.
To boot from NetBSD/sparc64 partitions, define LABELOFFSET to
be 128.
- Do not use the complete open firmware path to filter out cdrom drives.
No path containing "cdrom" is detected as a disk now.
- Simplify some code.

Make the openfirmware entry point function pointer non-static so that it
can be passed to the kernel.

Add a prototype for OF_alloc_phys.

Make the ofw_reg structure and ofw_alloc_heap 64-bit save.

Do not include openfirm.h; it is now included by libofw.h.

- Use the cell_t type definition for Open Firmware arguments in
combination with requisite casts as this avoids fatal side
effects on 64-bit architectures.
- Add the OF_alloc_phys function.

- Fill dummy functions with code to read from disk using Open
Firmware.
- Add a temporary disklabel header to boot off a NetBSD/sparc64
partition. This file can be deleted when we have got a FCode
bootblock.

The disklabel header was obtained from NetBSD.

- Include openfirm.h for phandle_t.
- Add some necessary members to the ofwdisk structure.
- Add a prototype for ofw_parseofwdev.

Add a function for parsing an Open Firmware boot path into the
ofw_devdesc structure.

- Add an ifdef guard.
- Use unsigned types for the (32-bit) Open Firmware device handles
to avoid sign extension on 64-bit architectures.
- Add a standard type definition for Open Firmware arguments.

Whitespace fixes.

Add new files needed by previous commit.

Mega-patch for OpenFirmware loader support.

- Flesh out ofw_readin routine.
- Add OpenFirmware load and exec routines.
- Make sure memory allocation for the kernel is done correctly.
- Change the way the heap is allocated so as to make it easier to deallocate
when we hand over.
- Add a command to print memory maps similar to the one for ia64.

With this patch, I can now load and hand over to a kernel on my iMac. There
are some problems with OpenFirmware routines failing after the hand over that
still need to be addressed.

Pass NULL instead of MAXPHYS to the DMA allocation method. Be a bit more
verbose if we fail to allocate the DMA buffer.

OpenFirmware/PowerPC loader, part 2.

As of this patchset, the loader builds (under NetBSD/macppc), boots, interacts
and talks to BOOTP/NFS servers.

(main.c was moved from boot/ofw/libofw to boot/ofw/common but has no revision
history)

Reviewed by: obrien

Convert from the Alpha compontents to PowerPC ones.

"Ok, my loader's now up to putting up a prompt. It probes disks partially
but can't boot from them yet."

Thanks to Stephane Potvin for the some of the code in this set.

Submitted by: Benno Rice <benno@jeamland.net>

Fix the more obvious warnings to deal with my earlier warning cleanups.

$Id$ -> $FreeBSD$

Bring the 'new-bus' to the i386. This extensively changes the way the
i386 platform boots, it is no longer ISA-centric, and is fully dynamic.
Most old drivers compile and run without modification via 'compatability
shims' to enable a smoother transition. eisa, isapnp and pccard* are
not yet using the new resource manager. Once fully converted, all drivers
will be loadable, including PCI and ISA.

(Some other changes appear to have snuck in, including a port of Soren's
ATA driver to the Alpha. Soren, back this out if you need to.)

This is a checkpoint of work-in-progress, but is quite functional.

The bulk of the work was done over the last few years by Doug Rabson and
Garrett Wollman.

Approved by: core

* Add old UFS compatibility code to alpha/boot1.
* Fix a raft of warnings, printf and otherwise.
* Allocate the correct amount in mod_searchmodule to prevent an overflow.
* Fix the makefiles so they work outside my home directory (oops).

Make the alpha bootstrap build again, fix some warning and change sdboot to daboot.

Bootstrap updates.

- Move some startup code from MD to MI sections
- Add a 'copyout' and some copyout-related functions. These will be
obsoleted when BTX is available for the 386 and the kernel load
area becomes directly addressable.
- Add the ability load an arbitrary file as a module, associating
and arbitrary type string with it. This can be used eg. for loading
splash-screen images etc.
- Add KLD module dependancy infrastructure. We know how to look for
dependancies inside KLD modules, how to resolve these dependancies
and what to do if things go wrong. Only works for a.out at the
moment, due to lack of an MI ELF loader. Attach KLD module information
to loaded modules as metadata, but don't pass it to the kernel (it
can find it itself).
- Load a.out KLD modules on a page boundary. Only pad the a.out BSS
for the kernel, as it may want to throw symbols away. (We might want
to do this for KLD modules too.)
- Allow commands to be hidden from the '?' display, to avoid cluttering
it with things like 'echo'. Add 'echo'.
- Bring the 'prompt' command into line with the parser syntax.
- Fix the verbose 'ls'; it was using an uninitialised stack variable.
- Add a '-v' flag to 'lsmod' to have it display module metadata as well
(not terribly useful for the average user)
- Support a 'module searchpath' for required modules.
- The bootstrap file on i386 is now called 'loader' to permit the
/boot directory to use that name.
- Discard the old i386 pread() function, as it's replaced by
arch_readin()

Bootloader update.

- Implement a new copyin/readin interface for loading modules.
This allows the module loaders to become MI, reducing code duplication.
- Simplify the search for an image activator for the loaded kernel.
- Use the common module management code for all module metadata.
- Add an 'unload' command that throws everything away.
- Move the a.out module loader to MI code, add support for a.out
kld modules.

Submitted by: Alpha changes fixed by Doug Rabson <dfr@freebsd.org>

Minor tweaks to track a couple of i386 changes and to make it compile.

This is the new unified bootstrap, sometimes known previously as the
'three-stage' bootstrap.
There are a number of caveats with the code in its current state:
- The i386 bootstrap only supports booting from a floppy.
- The kernel and kld do not yet know how to deal with the extended
information and module summary passed in.
- PnP-based autodetection and demand loading of modules is not implemented.
- i386 ELF kernel loading is not ready yet.
- The i386 bootstrap is loaded via an ugly blockmap.

On the alpha, both net- and disk-booting (SRM console machines only) is
supported. No blockmaps are used by this code.

Obtained from: Parts from the NetBSD/i386 standalone bootstrap.