MFS10 SVN r259621:

MFC r259276,259468-259470,259472,259474,259476-259478,259480-259481,259570,
259572, and 259597-259598...

r259276: Fix bug in `services' script in adding dumpdev comment to rc.conf
r259468: Ignore spurious escape generated by VMware's Ctrl-Cmd combination
r259469: Mask errors in `config' script from newaliases(1) about non-FQHN
r259470: Set atime=on for /var/mail zfsboot dataset to support mail server
r259472: Accept NULL input for zfsboot SWAP to indicate SWAP of zero bytes
r259474: Multiple changes, including bug-fixes and debugging improvements
r259476: Change default ZFS disk layout, making it easier to resize
r259477: fletcher4 is now the default (zfsboot related)
r259478: De-uglify the geli(8)-setup infobox (zfsboot related)
r259480: Fix ghosted zroot issue by always performing labelclear on swap
r259481: Auto-enable 4k sector alignmet when geli(8) is enabled (zfsboot)
r259570: Fix numerical comparison error (zfsboot)
r259572: Mask spurious rm error in bsdinstall_log from `auto' script
r259597: Fix zfsboot regression when installing to 3+ disks
r259598: Set cachefile property of bootpool so it imports to new system

Approved by: re (gjb)

MFC r259113, r259115, r259144, r259148:

r259113 (dteske):
Fix failed attempt to send pkg(8) stderr to /dev/null

r259115 (dteske):
Prevent truncating /tmp/bsdinstall_log each time we
exec a module.

r259144 (dteske):
Fix a regression after successfully installing to encrypted
ZFS root, the passphrase is not accepted and a message about
"incorrect key" is displayed.

r259148 (dteske):
Fix a regression resulting in mountroot prompt after attempting
to install to encrypted ZFS root (caused by a typo in a
variable name -- ZFSBOOT_BOOT_FSNAME -> ZFSBOOT_BOOTFS_NAME).

Approved by: re (glebius)
Sponsored by: The FreeBSD Foundation

Remove svn:mergeinfo from the releng/10.0 branch.

After branch creation from stable/10, the stable/10 branch mergeinfo
was moved to the root of the branch.

Since there have not been any merges from stable/10 to releng/10.0
yet, we do not need to track any of the existing mergeinfo here.

Merges to releng/10.0 should now be done to the root of the branch.

For future branches during the release cycle, unless otherwise noted,
this change will be done as part of the stable/ and releng/ branch
creation.

Discussed with: peter
Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

- Copy stable/10 (r259064) to releng/10.0 as part of the
10.0-RELEASE cycle.
- Update __FreeBSD_version [1]
- Set branch name to -RC1

[1] 10.0-CURRENT __FreeBSD_version value ended at '55', so
start releng/10.0 at '100' so the branch is started with
a value ending in zero.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r258927: Fix a regression introduced by SVN r257842 that prevents
Encryption from being enabled.

Approved by: re (gjb)

MFC Revisions:
256549: man-page updates
256551: .Dd bump in manpage
256553: bootenv -> ROOT fix
257842: monster list of fixes
257843: minor touchup
257844: copy install log
257845: fixup to last
257872: mdoc fixup
257939: debug check
258021: MBR bootcode fixup

Reviewed by: many
Discussed on: -current
Approved by: re (hrs)

MFC r257749:
Switch to using pkg(8) for the doc install in bsdinstall(8).
This also bootstraps the system with pkg(8) for future use.

Approved by: re (delphij)
PR: 183488
Sponsored by: The FreeBSD Foundation

MFC r256540, r256544:

r256540 (dteske):
Add loader.conf(5) entries to import bootpool after boot when
using full-disk encryption and ZFS.

r256544 (dteske):
Fix loader.conf(5) entries from r256540.

Approved by: re (hrs)
Sponsored by: The FreeBSD Foundation

MFC r256489:
Add executable bit to docsinstall [old] and entropy [new] scripts.

MFC r256541:
Document BSDINSTALL_TMPBOOT environment variable introduced by SVN r256343.

Approved by: re (gjb)

MFC revisions 256321-256323,256331,256333,256335,256343:
Bring in a new zfsboot auto script for performing automatic setup of a boot
pool (optionally encrypted) with many other options, validations, features.
Originally submitted by Allan Jude; modified in collaboration.

MFC revisions 256325,256330,256345:
Rewrite the keymap selection menu to display keymaps and provide a test
mechanism. Test mechanism originally submitted by Warren Block; modified.

MFC r256347:
Prominently display "Wireless" for each wireless network interface. Part
of PR bin/161547; submitted by Warren Block; slightly modified.

MFC r256348:
Remove the dumpdev configuration dialog, merge it into the regular
services configuration and enable it by default. Originally submitted by
Allan Jude; slightly modified.

PR: bin/161547
Submitted by: Allan Jude, Warren Block <wblock@wonkity.com>
In collaboration with: Allan Jude <freebsd@allanjude.com>
Approved by: re (glebius)

MFH (r256338): store some entropy after installation

Approved by: re (gjb)

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Remove ftp5.se.f.o, as per request to -hubs@

Approved by: re (glebius)
MFC after: 3 days

- Remove the is (Iceland) mirror per mail from the admins.

Approved by: re
With hat: clusteradm@

Add installer support for CHRP/PAPR PowerPC systems that use MBR+BSD
formatting, like x86, but with an additional MBR slice containing a raw
boot partition.

Approved by: re (gjb)

Make sure bootonly.iso for -BETAs and -RCs use the releases/
directory on the FTP mirrors to fetch distributions, since
these are always pushed to releases/ during the release cycle.

MFC after: 3 days
X-MFC-To: stable/9, releng/9.2

Remove ftp2.tr.freebsd.org, it is out of date.

MFC after: 3 days

Update mirror list to more closely match
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/mirrors-ftp.html

MFC after: 1 week

Fix bindings of keys when in the partition editor. By adding the usual input binding to the "partlist" sub window.
This is a workaround, as for unknown yet reason the keys binded on the Partition Edition window are the one from partlist instead of the one from standard "formfield"

Reported by: alfred, nwhitehorn

Add some missing DPADD.

Replace deprecated (or remove obsolete) libarchive 2.8 functions
with libarchive 3.0 counterparts

If no resolv.conf has been made for the new system already and the install
media has one already, copy it in lieu of leaving things blank. This
reduces the foot-shooting potential for PXE installs that immediately
add packages.

Properly quote argument.

MFC after: 3 days

Improve error handling and remove an unnecessary check on geom provider
type. GEOM provider names can't duplicate (or shouldn't -- devfs will either
break or only use the first one if they do) so using the first provider
by that name is a sufficient check. This also lets the scripted partitioner
install onto gmirror and geli and such things.

Improve documentation of BSDINSTALL_DISTSITE and mdoc.

Clarify that the setup script is optional.

Update change date.

Document bsdinstall scripting as it currently operates. It may change in
the future depending on feedback from people using it to do unattended
system setup.

Remove some debugging code that snuck in.

Add a scripting frontend. Documentation and release(7) support coming later.

Scripts take the form:

------------------------------------
PARTITIONS=ada0
DISTRIBUTIONS="kernel.txz base.txz ports.txz"

#!/bin/sh
echo "sshd_enable=YES" >> /etc/rc.conf
pkg add puppet
echo "System setup complete"
------------------------------------

The second part of the script (beginning with #!) is run in the newly
installed system after the installation onto ada0 (with default partitioning,
see scriptedpart for more complicated scenarios) is complete.

If no partition configuration is specified in {}, use the automatic setup
the interactive installer uses.

Example: bsdinstall scriptedpart ada0
Result: Use entire disk for a FreeBSD system with default partition layout

Add a simple scripted partitioner. Documentation and more scripting support
will come soon. This lets the install process have a line like:

bsdinstall scriptedpart 'ada0 GPT {1.5G freebsd-ufs /, 10G freebsd-swap,
auto freebsd-ufs /usr}'

to set up a system with a 1.5GB /, some swap space, and a /usr using the
rest of ada0.

MFC after: 1 month

Make "Finish" the default choice in the partition editor. This lets you
successfully complete an installation with all defaults by pressing
Enter repeatedly until your machine reboots.

MFC after: 3 weeks

Note that cpufreq(4) is not available on all hardware.

PR: bin/175139
MFC after: 2 weeks

Remove EOL whitespace.

Make sade actually be a symlink to partedit, as per the commit message in
r244859, rather than a hardlink. This fixes installation if /usr/sbin
is on a different filesystem than /usr/libexec.

Submitted by: se

Add a man page for the new sade(8), which is mostly the same uninformative
manpage as the old sade(8).

Replace sade the extracted piece of sysinstall with sade the extracted
piece of bsdinstall (although this time with a symlink instead of duplicated
source code).

Discussed on: freebsd-geom
MFC after: 3 months

If invoked as sade, don't complain about having a filesystem on / already
-- it's something you expect (and indeed hope for).

MFC after: 1 week

Add missed BETA* to list.

Approved by: simon (implicit)
MFC after: 3 days
X-MFC-With: r244058

Get 'uname -r' earlier, so it can be used to determine what branch is
being run to set BSDINSTALL_DISTSITE accordingly. This change allows
non-RELEASE branches to use the FTP snapshots directory for bootonly.iso
installations.

Approved by: simon

Remove snapshots.jp.freebsd.org. It stopped working years ago.

Discussed with: hrs

- Remove snapshots.se.freebsd.org [1]
- Add ftp6.se.freebsd.org

Discussed with: brd [1]

Fix bug introduced by r241902 (MANIFEST uses TAB delimiter).

PR: bin/173140
Approved by: adrian (co-mentor)

Optimize syntax to use builtins and reduce unnecessary forking where possible.

Reviewed by: nwhitehorn
Approved by: adrian (co-mentor)

Fix "unexpected operator" error when passed multi-word first-argument
containing whitespace. Also make other changes to support multi-word
arguments.

PR: bin/170759
Submitted by: dteske
Reviewed by: emaste (mentor)
Approved by: emaste (mentor)
MFC after: 3 days

Remove variables which are initialized but never used thereafter reported by gcc46 warning

Approved by: cperciva
MFC After: 3 days

Add the IPv6 tag to ftp4.fr.freebsd.org.

Fix typo.

Submitted by: pawel dot worach at gmail dot com
MFC after: 3 days

Remove trailing whitespace per mdoc lint warning

Disussed with: gavin
No objection from: doc
Approved by: joel
MFC after: 3 days

mdoc: document title should be all caps.

Make the chroot shell more functional by providing devfs.

Reported by: Robert Simmons
MFC after: 4 days

Fix segfault if distfetch and distextract binaries are run standalone
without the DISTRIBUTIONS environment variable set.

PR: bin/165492
Submitted by: Fernando Apesteguia
MFC after: 4 days

Fix a whitespace nit

Reviewed by: bz

Experimental support for booting CHRP-type PowerPC systems from hard disks.

Warn if trying to install over an existing partition, which usually fails
anyway due to libarchive not being able to overwrite schg flags.

PR: bin/164278
MFC after: 4 days

Prevent user astonishment by providing the shell option at the end, after
any installer-provided configuration files have been copied. This allows
users to edit their fstab, if desired, and to see what the installer has
placed in rc.conf.

Requested by: phk
MFC after: 1 day

Add a screen that asks if the user would like to enable crash dumps,
giving them a very brief description of the trade-offs. Whether the
user opts in or out add an entry to what will become /etc/rc.conf
explaining what dumpdev is and how to turn on/off crash dumps. The folks
who handle interacting with users submitting PRs have asked for this.

Reviewed by: nwhitehorn
MFC after: 1 day

Plug memory leaks and fix open(2) error check.

Reviewed by: nwhitehorn
MFC after: 3 days

Provide an error message and error handling if there are no network
interfaces in the system. This is a non-fatal error except when doing a
network installation.

PR: bin/161950
MFC after: 3 days

Provide an error message instead of silent failure if no disks are present
in the system.

PR: bin/161950
MFC after: 3 days

Avoid some "Invalid argument" errors in the installer.

PR: bin/161926
MFC after: 5 days

Escape the newline so we get a proper line continuation. Without this
the text of the menu selections doesn't get displayed properly and it
makes the installer appear to lock up for no obvious reason.

Reviewed by: nwhitehorn

Avoid magicking into existence sub-partitions due to leftover blocks when
creating new ones by destroying any geom that may have come into
existence immediately after adding a partition. The EBR partition scheme
is particularly enthusiastic about false positives in this case.

MFC after: 3 days

Usability enhancements: do not allow setting a mountpoint on bsdlabel
container partitions, which didn't do anything anyway, and check for
an existing freebsd-boot partition before bothering the user to make one.

PR: bin/160931
MFC after: 3 days

Work around some behavior of gpart that I absolutely do not understand in
order to make every operation of the partition editor fully revertable.
Under *no circumstances* will it any longer touch the disks until the user
presses Finish and confirms it.

MFC after: 3 days

Move "Exit" to the first entry in the list, so that it is the default
choice.

PR: bin/160913
MFC after: 3 days

Ask if you want to set the keymap before invoking kdbmap to prevent
confusion.

PR: bin/160913
MFC after: 3 days

Inquire about terminal type if starting from a serial console and do
not configure the host's networking if netbooted [1]. Also fix FTP
installations behind some firewalls [2].

PR: bin/159583 [2]
Reported by: stas [1]
Approved by: re (kib)

Fix a typo introduced in r219892 that prevented file system options from
being set correctly.

Approved by: re (kib)

Fix WPA setup broken by changes to shell argument passing inside the
installer network configuration.

PR: bin/159558
Approved by: re (kib)

Allow WPA PSKs up to 63 characters, the max length according to
wpa_supplicant.conf(5). Up to now, the PSK was truncated
to 15 characters (the length of the input field).

PR: 159559
Submitted by: Gavin Chappell <gavin.chappell@gmail.com>
Approved by: re (kib)

Add ftp4.se.freebsd.org to the IPv4 and IPv6 mirror lists.

PR: 157210, 157211
Submitted by: Niclas Zeising <zeising@lysator.liu.se>
Approved by: re (kib)

Correct a typo.

Submitted by: gcooper
Reviewed by: nwhitehorn
Approved by: re (kib)

Fix two problems:
1) if no default gateway is given the route command will fail
and the script will abort with an error leading to an endless
loop unless configuration is cancelled or changed. [1]
2) In the IPv6 case add -inet6 to not alter the legacy IP default
route.

Reported by: Oleg Ginzburg (olevole olevole.ru) [1]
Tested by: Oleg Ginzburg (olevole olevole.ru) [1]
Approved by: re (kib)

Swap FTP paths to be MACHINE/MACHINE_ARCH instead of MACHINE_ARCH/MACHINE.
This follows what the output of make targets looks like, and reduces the
number of directories needed on the FTP server, since we now have only one
architecture with multiple machine types (i386, which has i386 and pc98),
but 3 machines with 9 architectures between them (powerpc, mips, and arm).

Requested by: kensmith, bz
Approved by: re(kensmith)

Implement support for GRAID volumes in the installer partition editor,
rename a few options, clarify some help text, and add help text for the
buttons on the main partition editor screen.

Approved by: re (kib)

Make messages about typos appear on the screen instead of in the log file.

Reported by: lev
Approved by: re (kib)

Fix a bug that prevented docsinstall from being able to use DNS in most
cases and provide a better error handling mechanism during package
installation.

Approved by: re (kib)

Add back ftp4, a new machine has taken its place. ftp2 and ftp8 are both IPv6 capable
mirrors. cvsup4/ftp4 have already been added to the handbook.

Reviewed by: nathanw
Approved by: re (kib)

To inet or not two inet. Unbreak static IPv4 configuration. If you tried
you were stuck in an endless loop.

Reported by: kensmith
Approved by: re (kib) for BETA1
Pointy hat to: bz - wonder why I had not tested legacy IP thoroughly..

Add PS3 partitioning (and framework for future pseries support) to the
installer.

Run newaliases at the end of the install process.

Submitted by: kevlo

Per request of the docs team, install docs as packages, instead of via
the normal distfile mechanism. Thanks to Marc Fonvieille for the patch and
for putting up with me taking entirely too long to commit this!

Submitted by: blackend

Add an extra tab between fs_file and fs_vfstype

Reviewed by: nwhitehorn

I forgot what year it is, apparently.

Submitted by: brucec

Long-overdue man page for bsdinstall.

Remove TODO which is not longer needed and the default.

Fix resolv.conf search list creation:
1) do not print out an empty "search ", things do not like it.
2) the search list is not comma separated.

Sponsored by: The FreeBSD Foundation
Sponsored by: iXsystems

Write the multi step netconfig to a temporary file and only move that
to the final name if netconfig was completely finished. This fixes
reentrance problems even better than r222611.

Suggested by: nwhitehorn
Reviewed by: nwhitehorn
Sponsored by: The FreeBSD Foundation
Sponsored by: iXsystems

Empty the network configuration only after the user decided to pick an
interface. Otherwise an accidental start of the netowrk configuration
and immediate cancel after the install has finished removes the previously
configured settings.

Discussed with: nwhitehorn
Sponsored by: The FreeBSD Foundation
Sponsored by: iXsystems

Fix opening a shell on the new system (prevent the shell's stderr from
ending up in the install log).

Contrary to the rc.conf framework, when manualy enabling IPv6 we have
to -ifdiabled ourselves.

Sponsored by: The FreeBSD Foundation
Sponsored by: iXsystems

While doing it right for current configuration, fix the entry for rc.conf
adding the missing mandatory "inet6" keyword.

Sponsored by: The FreeBSD Foundation
Sponsored by: iXsystems

Split netconfig into three parts:
- netconfig - what auto will call which in turn will check for
IPv4 and IPv6 to be available and ask the user to configure it
by calling
- netconfig_ipv4 doing DHCP and static IPv4 addresses, and
- netconfig_ipv6 doing rtsol and static IPv6 addresses,
and then checking, querying and updating resolv.conf upon return.
Both DHCP and rtsol (in the future) might update resolv.conf already so
we seed ourselves from that file if available.

Reviewed by: nwhitehorn
Sponsored by: The FreeBSD Foundation
Sponsored by: iXsystems

Fix shell-based partitioning.

Disconnect sun4v architecture from the three.

Some files keep the SUN4V tags as a code reference, for the future,
if any rewamped sun4v support wants to be added again.

Reviewed by: marius
Tested by: sbruno
Approved by: re

Fix mirror selection, which was broken by stderr redirection.

Reported by: Garrett Cooper

Update to reflect net install changes.

Make the keymap script continue to work now that stderr is being
redirected to the log file.

Rename the "Reboot" option to "Exit", which reflects what it actually
does. The rc.local wrapper script will also soon grow an option to use the
live environment after installation, so an immediate reboot isn't even
necessarily implied.

Check correctly for whether there are any available wireless networks. This
was tested on a train in the middle of an upstate New York swamp.

Improve logging by always sending stderr to the installation log file.
Reduce warnings by making sure the temporary etc directory exists.

Committing while tired is never a good idea. Remove an unconditional error
exit left over from debugging.

Improve error handling.

Add mirror selection for FTP installs. To support TBEMD installs
(e.g. powerpc64), this looks in a slightly different default path than now,
specifying both $MACHINE and $MACHINE_ARCH:
MIRROR/pub/FreeBSD/releases/`uname -p`/`uname -m`/`uname -r`

How to handle TBEMD properly probably merits some more discussion, but,
since no such 9.0-CURRENT distfiles exist at the moment, all existing
mirrors presently fail anyway.

Mark any distfiles with no checksum entries in the manifest "Skipped"
instead of "Passed".

Allow setting of parameters for file systems (e.g. softupdates), turn on
SUJ by default, and allow creation and mounting of FAT filesystems from
the installer.

Turn on softupdates by default. We need a UI to set filesystem parameters.

Rewind manifest file in case distributions are in a different order there.

Use the manifest file count, if available, to determine the number of files
in the archive instead of spinning through it to get a file count for the
progress bar. This speeds up installation a lot on systems with slow
CD drives.

Add generation of an installation manifest containing SHA256 checksums as
well as package descriptions and add code in the installer to check the
checksums.

Correct typo.

Only validate the partition setup when the user presses "Save". If the user
wants to exit without saving, it's not a problem if the disk is set up
in an invalid way.

Make installations work when no optional components are selected.

Show disk model numbers in partition wizard screen. Also, since da(4) disks
are probably more likely to be USB or Firewire these days than SCSI, just
call them and anyone unknown "Disk" instead of SCSI disk.

Add a catch-all for miscellaneous RAID volumes while seeking a better
solution here. It would really help if CAM devices were part of newbus.

Add support for optional distributions, installation of the ports tree and
src. Fitting all of this on a single CD along with a full system requires
changing the default compression algorithm from gzip to xz.

Add some error checking on the return values of chdir() and calloc(). The
first might actually happen, so it displays the error message in a prettier
way.

Found by: Coverity Prevent(tm)
CID: 9121, 9122, 9123, 9124

Don't present the user with spurious errors when overwriting a disk with
a deleted, but uncommitted, gpart.

Reported by: brucec

The PC98 gpart module supports labels now.

Import bsdinstall. This is meant to be (eventually in conjunction with
pc-sysinstall) a replacement for sysinstall in the 9.0 release and beyond.
Currently supported platforms are sparc64, pc98, i386, amd64, powerpc, and
powerpc64. Integration into the build system will occur in the coming
weeks.

Merging with pc-sysinstall will use this code as a frontend, while
temporarily retaining the interactive partition editor here. This work
will be done in parallel with improvements on this code and release
integration.

Thanks to all who have provided testing and comments!