Improve patch for SA-15:04.igmp to solve a potential buffer overflow.

Fix multiple vulnerabilities of ntp. [SA-15:07]

Fix bsdinstall(8) insecure default GELI keyfile permissions. [SA-15:08]

Fix Denial of Service with IPv6 Router Advertisements. [SA-15:09]

Approved by: so

MFH (r272173): offer to enable local_unbound after installation

Approved by: re (gjb)

Copy stable/10@r272459 to releng/10.1 as part of
the 10.1-RELEASE process.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r272274:
Change the /var dataset in the default ZFS layout to have the
ZFS property canmount=off, making /var/db/pkg part of the / dataset, so
installed package files are consistent with the package database when
using ZFS boot environments (beadm).

PR: 193971
Reviewed by: Shawn Webb, bcr
Approved by: re (gjb), jmg
Relnotes: yes
Sponsored by: ScaleEngine Inc.

MFC revisions 271553, 271567:
r271553: Rename choices in partitioning method dialog (nwhitehorn)
r271567: ZFS support isn't actually experimental anymore (nwhitehorn)

Approved by: re (gjb)

MFC r271563:
Make the root-on-zfs part of the installer warn a user who booted the
installer via UEFI that we do not support booting ZFS via UEFI yet

PR: 193595
Approved by: re (gjb), nwhitehorn
Sponsored by: ScaleEngine Inc.

MFC r271552:

Make the default choice for the chroot shell at the end be "No". This allows
just pressing enter repeatedly to successfully install a reasonable system.

Approved by: re (gjb)

MFH (r269653): Give a brief error message

MFH (r269074): strip patch level from release name

Fix syntax error thrown at the point of creating the root pool, caused by
an embedded newline appearing within the options string surrounded by
double-quotes. Rework the logic that goes into setting dataset options on
the root pool dataset while we're here -- added two new variables (which
can be altered via scripting) ZFSBOOT_POOL_CREATE_OPTIONS and also
ZFSBOOT_BOOT_POOL_CREATE_OPTIONS for setting pool/dataset attributes at
the time of pool creation. The former is for setting options on the root
pool (zroot) and the latter is for setting options on the optional separate
boot pool (bootpool) implicitly enabled when using either GELI or MBR. The
default value for the root pool variable (ZFSBOOT_POOL_CREATE_OPTIONS) is
"-O compress=lz4 -O atime=off" and the default value for separate boot pool
variable (ZFSBOOT_BOOT_POOL_CREATE_OPTIONS) is NULL (no additional options
for the separate boot pool dataset).

Reviewed by: allanjude

MFC r266107,r266108,r266109:

Here is a patch for the bsdinstall root-on-zfs stuff that adds optional
encryption for swap, and optional gmirror for swap (which can be combined)

Updates to the datasets created by zfsboot.

Set compress=lz4 for the entire pool, removing it from the individual
datasets

Remove exec=no from /usr/src, breaks the test suite.

Fix the "disks" variable reuse.

It starts off being used to track the grammar for the number of disks
(singular vs plural) and then it is reused as the list of available disks.

Replace the variable with disks_grammar and move 'disk' and 'disks' to
msg_ vars so they can be translated in the future.

Submitted by: Allan Jude <freebsd@allanjude.com>
Reviewed by: roberto
Sponsored by: ScaleEngine Inc.

MFC r264841: Update zfsboot to coincide with MFC of r264840
adding GEOM support.

MFC r264488:

When merging docsinstall and zfsboot updates to stable/9 it was discovered
that the slightly older dialog(1) requires --separate-output when using the
--checklist widget to force response to produce unquoted values (whereas in
stable/10 --checklist widget without --separate-output will only quote the
checklist labels in the response if the label is multi-word (contains any
whitespace).

Since these enhancements (see revisions 263956 and 264437) were developed
originally on 10, the --separate-output option was omitted. When merged to
stable/9, we (Allan Jude) and I found during testing that the "always-
quoting" of the response was causing things like struct interpolation to
fail (`f_struct device_$dev' would produce `f_struct device_\"da0\"' for
example -- literal quotes inherited from dialog(1) --checklist response).

To see the behavior, execute the following on stable/9 versus stable/10:

dialog --checklist disks: 0 0 0 da0 "" off da1 "" off

Check both items and hit enter. On stable/10, the response is:

da0 da1

On stable/9 the response is:

"da0" "da1"

If you use the --separate-output option, the response is the same for both:

da0
da1

So applying --separate-output on every platform until either one of two
things occurs 1) dialog(1,3) gets synchronized between stable/9, higher or
2) we drop support for stable/9.

Reviewed by: Allan Jude

MFC r260866,260900,261960 (3 revisions; summarized below):
260866: Fix a bad comparison operator and fix dumpdev comment
260900: Dummy commit; show r260866 was in-part by Christoph Mallon
261960: Add zfsboot to bsdinstall(8) scripted interface

MFC r264448: Fix typo in debug/log statement.

Submitted by: Rick Miller <vmiller@hostileadmin.com>

MFC r260261 (gavin):
Lower the priority of the "connect to any open wireless network" network
description, so guaranteeing that any other defined network will be tried
first.

MFC r260262 (gavin):
Allow bsdinstall to use WPA-Enterprise networks when installing. This
only allows basic username/password config, and does not provide the
ability to set any of the other WPA options. Regardless, this is
generally sufficient to associate.

Perhaps in the future this could allow full configuring (e.g. being able
to set "anonymous identity", and perhaps some of the more obscure WPA
options), though perhaps that will better belong in bsdconfig when that
grows wlan config ability.

MFC r260260 (gavin): Remove what appears to be a stray debugging printf.

MFC r259686:
Move the installer's keymap solution introduced by SVN r259468. The new
location of /etc/rc.local on the install media is more appropriate as it
knows serial vs. non-serial and can also do the change earlier (so that
even the initial Install dialog can benefit from the change).

Discussed with: nwhitehorn

MFC r259276,259468-259470,259472,259474,259476-259478,259480-259481,259570,
259572, and 259597-259598...

r259276: Fix bug in `services' script in adding dumpdev comment to rc.conf
r259468: Ignore spurious escape generated by VMware's Ctrl-Cmd combination
r259469: Mask errors in `config' script from newaliases(1) about non-FQHN
r259470: Set atime=on for /var/mail zfsboot dataset to support mail server
r259472: Accept NULL input for zfsboot SWAP to indicate SWAP of zero bytes
r259474: Multiple changes, including bug-fixes and debugging improvements
r259476: Change default ZFS disk layout, making it easier to resize
r259477: fletcher4 is now the default (zfsboot related)
r259478: De-uglify the geli(8)-setup infobox (zfsboot related)
r259480: Fix ghosted zroot issue by always performing labelclear on swap
r259481: Auto-enable 4k sector alignmet when geli(8) is enabled (zfsboot)
r259570: Fix numerical comparison error (zfsboot)
r259572: Mask spurious rm error in bsdinstall_log from `auto' script
r259597: Fix zfsboot regression when installing to 3+ disks
r259598: Set cachefile property of bootpool so it imports to new system

MFC r259132,r259133,r259143:
Per the resolution of POSIX bug 0000779 (note 0002050) add support for using 'i'
as a case insensitive flag.

Add regression test for recently added 'i' flag in r259132.

the i modifier was added in r259132 since POSIX recently agreed to add
it. Switch uses of /I to /i.

PR: standards/184641

MFC r259113, r259115, r259144, r259148:

r259113 (dteske):
Fix failed attempt to send pkg(8) stderr to /dev/null

r259115 (dteske):
Prevent truncating /tmp/bsdinstall_log each time we
exec a module.

r259144 (dteske):
Fix a regression after successfully installing to encrypted
ZFS root, the passphrase is not accepted and a message about
"incorrect key" is displayed.

r259148 (dteske):
Fix a regression resulting in mountroot prompt after attempting
to install to encrypted ZFS root (caused by a typo in a
variable name -- ZFSBOOT_BOOT_FSNAME -> ZFSBOOT_BOOTFS_NAME).

Sponsored by: The FreeBSD Foundation

MFC r258927: Fix a regression introduced by SVN r257842 that prevents
Encryption from being enabled.

Approved by: re (gjb)

MFC Revisions:
256549: man-page updates
256551: .Dd bump in manpage
256553: bootenv -> ROOT fix
257842: monster list of fixes
257843: minor touchup
257844: copy install log
257845: fixup to last
257872: mdoc fixup
257939: debug check
258021: MBR bootcode fixup

Reviewed by: many
Discussed on: -current
Approved by: re (hrs)

MFC r257749:
Switch to using pkg(8) for the doc install in bsdinstall(8).
This also bootstraps the system with pkg(8) for future use.

Approved by: re (delphij)
PR: 183488
Sponsored by: The FreeBSD Foundation

MFC r256540, r256544:

r256540 (dteske):
Add loader.conf(5) entries to import bootpool after boot when
using full-disk encryption and ZFS.

r256544 (dteske):
Fix loader.conf(5) entries from r256540.

Approved by: re (hrs)
Sponsored by: The FreeBSD Foundation

MFC r256489:
Add executable bit to docsinstall [old] and entropy [new] scripts.

MFC r256541:
Document BSDINSTALL_TMPBOOT environment variable introduced by SVN r256343.

Approved by: re (gjb)

MFC revisions 256321-256323,256331,256333,256335,256343:
Bring in a new zfsboot auto script for performing automatic setup of a boot
pool (optionally encrypted) with many other options, validations, features.
Originally submitted by Allan Jude; modified in collaboration.

MFC revisions 256325,256330,256345:
Rewrite the keymap selection menu to display keymaps and provide a test
mechanism. Test mechanism originally submitted by Warren Block; modified.

MFC r256347:
Prominently display "Wireless" for each wireless network interface. Part
of PR bin/161547; submitted by Warren Block; slightly modified.

MFC r256348:
Remove the dumpdev configuration dialog, merge it into the regular
services configuration and enable it by default. Originally submitted by
Allan Jude; slightly modified.

PR: bin/161547
Submitted by: Allan Jude, Warren Block <wblock@wonkity.com>
In collaboration with: Allan Jude <freebsd@allanjude.com>
Approved by: re (glebius)

MFH (r256338): store some entropy after installation

Approved by: re (gjb)

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Remove ftp5.se.f.o, as per request to -hubs@

Approved by: re (glebius)
MFC after: 3 days

- Remove the is (Iceland) mirror per mail from the admins.

Approved by: re
With hat: clusteradm@

Make sure bootonly.iso for -BETAs and -RCs use the releases/
directory on the FTP mirrors to fetch distributions, since
these are always pushed to releases/ during the release cycle.

MFC after: 3 days
X-MFC-To: stable/9, releng/9.2

Remove ftp2.tr.freebsd.org, it is out of date.

MFC after: 3 days

Update mirror list to more closely match
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/mirrors-ftp.html

MFC after: 1 week

If no resolv.conf has been made for the new system already and the install
media has one already, copy it in lieu of leaving things blank. This
reduces the foot-shooting potential for PXE installs that immediately
add packages.

Properly quote argument.

MFC after: 3 days

Remove some debugging code that snuck in.

Add a scripting frontend. Documentation and release(7) support coming later.

Scripts take the form:

------------------------------------
PARTITIONS=ada0
DISTRIBUTIONS="kernel.txz base.txz ports.txz"

#!/bin/sh
echo "sshd_enable=YES" >> /etc/rc.conf
pkg add puppet
echo "System setup complete"
------------------------------------

The second part of the script (beginning with #!) is run in the newly
installed system after the installation onto ada0 (with default partitioning,
see scriptedpart for more complicated scenarios) is complete.

Note that cpufreq(4) is not available on all hardware.

PR: bin/175139
MFC after: 2 weeks

Add missed BETA* to list.

Approved by: simon (implicit)
MFC after: 3 days
X-MFC-With: r244058

Get 'uname -r' earlier, so it can be used to determine what branch is
being run to set BSDINSTALL_DISTSITE accordingly. This change allows
non-RELEASE branches to use the FTP snapshots directory for bootonly.iso
installations.

Approved by: simon

Remove snapshots.jp.freebsd.org. It stopped working years ago.

Discussed with: hrs

- Remove snapshots.se.freebsd.org [1]
- Add ftp6.se.freebsd.org

Discussed with: brd [1]

Fix bug introduced by r241902 (MANIFEST uses TAB delimiter).

PR: bin/173140
Approved by: adrian (co-mentor)

Optimize syntax to use builtins and reduce unnecessary forking where possible.

Reviewed by: nwhitehorn
Approved by: adrian (co-mentor)

Add the IPv6 tag to ftp4.fr.freebsd.org.

Make the chroot shell more functional by providing devfs.

Reported by: Robert Simmons
MFC after: 4 days

Fix a whitespace nit

Reviewed by: bz

Prevent user astonishment by providing the shell option at the end, after
any installer-provided configuration files have been copied. This allows
users to edit their fstab, if desired, and to see what the installer has
placed in rc.conf.

Requested by: phk
MFC after: 1 day

Add a screen that asks if the user would like to enable crash dumps,
giving them a very brief description of the trade-offs. Whether the
user opts in or out add an entry to what will become /etc/rc.conf
explaining what dumpdev is and how to turn on/off crash dumps. The folks
who handle interacting with users submitting PRs have asked for this.

Reviewed by: nwhitehorn
MFC after: 1 day

Provide an error message and error handling if there are no network
interfaces in the system. This is a non-fatal error except when doing a
network installation.

PR: bin/161950
MFC after: 3 days

Escape the newline so we get a proper line continuation. Without this
the text of the menu selections doesn't get displayed properly and it
makes the installer appear to lock up for no obvious reason.

Reviewed by: nwhitehorn

Move "Exit" to the first entry in the list, so that it is the default
choice.

PR: bin/160913
MFC after: 3 days

Ask if you want to set the keymap before invoking kdbmap to prevent
confusion.

PR: bin/160913
MFC after: 3 days

Inquire about terminal type if starting from a serial console and do
not configure the host's networking if netbooted [1]. Also fix FTP
installations behind some firewalls [2].

PR: bin/159583 [2]
Reported by: stas [1]
Approved by: re (kib)

Fix WPA setup broken by changes to shell argument passing inside the
installer network configuration.

PR: bin/159558
Approved by: re (kib)

Allow WPA PSKs up to 63 characters, the max length according to
wpa_supplicant.conf(5). Up to now, the PSK was truncated
to 15 characters (the length of the input field).

PR: 159559
Submitted by: Gavin Chappell <gavin.chappell@gmail.com>
Approved by: re (kib)

Add ftp4.se.freebsd.org to the IPv4 and IPv6 mirror lists.

PR: 157210, 157211
Submitted by: Niclas Zeising <zeising@lysator.liu.se>
Approved by: re (kib)

Correct a typo.

Submitted by: gcooper
Reviewed by: nwhitehorn
Approved by: re (kib)

Fix two problems:
1) if no default gateway is given the route command will fail
and the script will abort with an error leading to an endless
loop unless configuration is cancelled or changed. [1]
2) In the IPv6 case add -inet6 to not alter the legacy IP default
route.

Reported by: Oleg Ginzburg (olevole olevole.ru) [1]
Tested by: Oleg Ginzburg (olevole olevole.ru) [1]
Approved by: re (kib)

Swap FTP paths to be MACHINE/MACHINE_ARCH instead of MACHINE_ARCH/MACHINE.
This follows what the output of make targets looks like, and reduces the
number of directories needed on the FTP server, since we now have only one
architecture with multiple machine types (i386, which has i386 and pc98),
but 3 machines with 9 architectures between them (powerpc, mips, and arm).

Requested by: kensmith, bz
Approved by: re(kensmith)

Make messages about typos appear on the screen instead of in the log file.

Reported by: lev
Approved by: re (kib)

Fix a bug that prevented docsinstall from being able to use DNS in most
cases and provide a better error handling mechanism during package
installation.

Approved by: re (kib)

Add back ftp4, a new machine has taken its place. ftp2 and ftp8 are both IPv6 capable
mirrors. cvsup4/ftp4 have already been added to the handbook.

Reviewed by: nathanw
Approved by: re (kib)

To inet or not two inet. Unbreak static IPv4 configuration. If you tried
you were stuck in an endless loop.

Reported by: kensmith
Approved by: re (kib) for BETA1
Pointy hat to: bz - wonder why I had not tested legacy IP thoroughly..

Run newaliases at the end of the install process.

Submitted by: kevlo

Per request of the docs team, install docs as packages, instead of via
the normal distfile mechanism. Thanks to Marc Fonvieille for the patch and
for putting up with me taking entirely too long to commit this!

Submitted by: blackend

Remove TODO which is not longer needed and the default.

Fix resolv.conf search list creation:
1) do not print out an empty "search ", things do not like it.
2) the search list is not comma separated.

Sponsored by: The FreeBSD Foundation
Sponsored by: iXsystems

Write the multi step netconfig to a temporary file and only move that
to the final name if netconfig was completely finished. This fixes
reentrance problems even better than r222611.

Suggested by: nwhitehorn
Reviewed by: nwhitehorn
Sponsored by: The FreeBSD Foundation
Sponsored by: iXsystems

Empty the network configuration only after the user decided to pick an
interface. Otherwise an accidental start of the netowrk configuration
and immediate cancel after the install has finished removes the previously
configured settings.

Discussed with: nwhitehorn
Sponsored by: The FreeBSD Foundation
Sponsored by: iXsystems

Fix opening a shell on the new system (prevent the shell's stderr from
ending up in the install log).

Contrary to the rc.conf framework, when manualy enabling IPv6 we have
to -ifdiabled ourselves.

Sponsored by: The FreeBSD Foundation
Sponsored by: iXsystems

While doing it right for current configuration, fix the entry for rc.conf
adding the missing mandatory "inet6" keyword.

Sponsored by: The FreeBSD Foundation
Sponsored by: iXsystems

Split netconfig into three parts:
- netconfig - what auto will call which in turn will check for
IPv4 and IPv6 to be available and ask the user to configure it
by calling
- netconfig_ipv4 doing DHCP and static IPv4 addresses, and
- netconfig_ipv6 doing rtsol and static IPv6 addresses,
and then checking, querying and updating resolv.conf upon return.
Both DHCP and rtsol (in the future) might update resolv.conf already so
we seed ourselves from that file if available.

Reviewed by: nwhitehorn
Sponsored by: The FreeBSD Foundation
Sponsored by: iXsystems

Fix shell-based partitioning.

Fix mirror selection, which was broken by stderr redirection.

Reported by: Garrett Cooper

Update to reflect net install changes.

Make the keymap script continue to work now that stderr is being
redirected to the log file.

Rename the "Reboot" option to "Exit", which reflects what it actually
does. The rc.local wrapper script will also soon grow an option to use the
live environment after installation, so an immediate reboot isn't even
necessarily implied.

Check correctly for whether there are any available wireless networks. This
was tested on a train in the middle of an upstate New York swamp.

Committing while tired is never a good idea. Remove an unconditional error
exit left over from debugging.

Improve error handling.

Add mirror selection for FTP installs. To support TBEMD installs
(e.g. powerpc64), this looks in a slightly different default path than now,
specifying both $MACHINE and $MACHINE_ARCH:
MIRROR/pub/FreeBSD/releases/`uname -p`/`uname -m`/`uname -r`

How to handle TBEMD properly probably merits some more discussion, but,
since no such 9.0-CURRENT distfiles exist at the moment, all existing
mirrors presently fail anyway.

Mark any distfiles with no checksum entries in the manifest "Skipped"
instead of "Passed".

Add generation of an installation manifest containing SHA256 checksums as
well as package descriptions and add code in the installer to check the
checksums.

Correct typo.

Make installations work when no optional components are selected.

Add support for optional distributions, installation of the ports tree and
src. Fitting all of this on a single CD along with a full system requires
changing the default compression algorithm from gzip to xz.

Import bsdinstall. This is meant to be (eventually in conjunction with
pc-sysinstall) a replacement for sysinstall in the 9.0 release and beyond.
Currently supported platforms are sparc64, pc98, i386, amd64, powerpc, and
powerpc64. Integration into the build system will occur in the coming
weeks.

Merging with pc-sysinstall will use this code as a frontend, while
temporarily retaining the interactive partition editor here. This work
will be done in parallel with improvements on this code and release
integration.

Thanks to all who have provided testing and comments!