release: Rework vm_extra_pre_umount

The vm_extra_pre_umount function in vmimage.subr served two purposes:
It removed /etc/resolv.conf and /qemu (if cross-building), and it
provided a function for cloudware to override in order to make cloud
specific changes to the filesystem before constructing a disk image.

This resulted in a number of bugs:
1. When cross-building, the emulator binary was left as /qemu in the
Azure, GCE, Openstack and Vagrant images.
2. The build host's resolv.conf was left as /etc/resolv.conf in the
basic-ci and basic-cloudinit images.
3. When building GCE images, a Google-specific resolv.conf file was
constructed, and then deleted before the disk image was created.

Move the bits needed for running code inside a VM staging directory
from vm_install_base into a new vm_emulation_setup routine, and move
the corresponding cleanup bits from vm_extra_pre_umount to a new
vm_emulation_cleanup routine.

Remove the /qemu and /etc/resolv.conf cleanups from the cloudware
configuration files (where they exist) since we will now be running
vm_emulation_cleanup to remove those even when vm_extra_pre_umount
has been overridden.

Override vm_emulation_cleanup in gce.conf since in that one case (and
*only* that one case) we don't want to clean up resolv.conf (since it
was constructed for the VM image rather than copied from the host).

releng/14.1 candidate.

MFC after: 1 week
Sponsored by: https://www.patreon.com/cperciva

release: remove inet blackhole sysctls from GCE config

Other cloud images do not do this, and it can produce confusing results.

Reviewed by: Jose Luis Duran, delphij
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D41751

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

release: remove a stray backslash

PR: 272354 (related)
MFC after: 3 days
MFC with: 0ed426276fd52e917506ab87c05a61c6641ae298
MFC with: cd8cad0ef5cd534dc7ae1dd9a470cac2057ed957
Sponsored by: GoFundMe https://www.gofundme.com/f/gjbbsd

release: update GCE configuration for python3

Reported by: asomers
PR: 272354
MFC after: 3 days
Sponsored by: GoFundMe https://www.gofundme.com/f/gjbbsd

Correct typo in soacceptqueue setting

I somehow introduced the typo when extracting one part of D34598.

Reported by: Jose Luis Duran <jlduran@gmail.com>
Fixes: 9871ae6aa91f ("Track kern.ipc.somaxconn -> ...")

Track kern.ipc.somaxconn -> kern.ipc.soacceptqueue rename

Fixes: 2bdf61ca29d0 ("Hide the unfortunate named sysctl...")
MFC after: 3 days

release: Remove references to ChallengeResponseAuthentication

This sshd_config keyword was replaced by KbdInteractiveAuthentication in
openssh 8.7, though ChallengeResponseAuthentication is silently accepted
as an alias. However, this means that the code in ec2.conf which
modifies a commented-out line no longer does anything. Apply a minimal
fix.

Reviewed by: cperciva, emaste
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34400

release: allow VM_EXTRA_PACKAGES to be specified in the environment

This is useful for adding extra packages to the build of an AMI.
For example:
env VM_EXTRA_PACKAGES="zsh" make -C release ec2ami

Approved by: gjb
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")

release: fix VMSIZE following 1ca8842f3ad9

truncate(1) is not case-sensitive with regard to setting the size
of a file. makefs(8), however, does not honor upper-case values.
Update release-specific files and the release(7) manual page to
reflect this.

MFC with: 1ca8842f3ad9
Submitted by: ehem_freebsd_m5p.com (original)
Differential Review: https://reviews.freebsd.org/D28979
Sponsored by: Rubicon Communications, LLC ("Netgate")

Revert r348438.

The fix to override the default python version when building
the sysutils/py-google-compute-engine did not work, and there
are still issues that need to be addressed in the port itself.

See bugzilla 238267 for additional details.

MFC after: 6 days
MFC with: r348438
MFC note: no-op to appease the merge tracker
Sponsored by: The FreeBSD Foundation

Fix GCE virtual machine startup.

The ports/head branch recently switched to python3 as the default,
which breaks the sysutils/py-google-compute-engine startup scripts,
as lang/python installs lang/python3{,.x} where lang/python2{,.x}
are needed.

Set DEFAULT_VERSIONS in release/tools/gce.conf to python=2.7, and
remove the lang/python3 inclusion in VM_EXTRA_PACKAGES.

Additionally, unset DEFAULT_VERSIONS in release/tools/vmimage.subr
to prevent persistence of DEFAULT_VERSIONS=python=2.7 in subsequent
VM/cloud image builds.

Note: at present, this affects only 13-CURRENT and 12-STABLE, as
the stable/11 branch had already switched to using the 2019Q2 branch
at the start of the 11.3-RELEASE cycle, so this does not immediately
affect 11.3-BETA, hence the 1-week merge timeout. This had been
manually tested on 13-CURRENT.

Reported by: ler (privately)
MFC after: 1 week
Sponsored by: The FreeBSD Foundation

Increase the default size of the GCE disk image from 3GB to 20GB,
as 3GB is too small as discovered in this week's snapshot builds.

MFC after: 3 days
Sponsored by: The FreeBSD Foundation

Reduce the default image size for virtual machine disk images from
30GB to 3GB. The raw images can be resized using truncate(1), and
other formats can be resized with tools included with other tools
included with other hypervisors.

Enable the growfs(8) rc(8) at firstboot if the disk was resized
prior to booting the virtual machine for the first time.

Discussed with: several
PR: 232313 (requested in other context)
MFC after: 3 days
Sponsored by: The FreeBSD Foundation

Fix NTP query on GCE due to unresolved hostname.

PR: 232456
Submitted by: Lucas Kanashiro
MFC after: 3 days
Sponsored by: The FreeBSD Foundation

Reduce the GCE image size to 27G to be lower than the free
quota limit.

PR: 232313
MFC after: 3 days
Sponsored by: The FreeBSD Foundation

Add a space between a variable and escaped new line.

MFC after: 3 days
MFC with: r337717
Sponsored by: The FreeBSD Foundation

Add lang/python2, lang/python3, and lang/python to GCE images
to help avoid hard-coding 'python<MAJOR>.<MINOR>' in several
scripts in the client-side scripts.

PR: 230248
MFC after: 3 days
Submitted by: gustavo.scalet@collabora.com
Sponsored by: The FreeBSD Foundation

Invoke the growfs rc script for each boot on GCE.

PR: 230275
Submitted by: gustavo.scalet@collabora.com
MFC after: 3 days
Sponsored by: The FreeBSD Foundation

Update and replace old rc daemons for GCE images.

PR: 229000
Submitted by: helen.koike@collabora.com
MFC after: 3 days
Sponsored by: The FreeBSD Foundation

Remove insecure ciphers from GCE sshd configuration

They were added for unclear reasons in r277263. The current OpenSSH
defaults (7.5+) are reasonable, and do not include the insecure rc4 cipher:

chacha20-poly1305@openssh.com,
aes128-ctr,aes192-ctr,aes256-ctr,
aes128-gcm@openssh.com,aes256-gcm@openssh.com,
aes128-cbc,aes192-cbc,aes256-cbc

I think I recall there being a reason for a specific list of ciphers on GCE
at the time, but I do not recall what it was, and cannot find any
current GCE documentation of such a list.

So, just revert the explicit configuration and use sane openssh defaults.

PR: 230092
Submitted by: Gustavo Scalet <gustavo.scalet AT collabora.com>
MFC after: 3 days
Security: yes

Fix a typo.

Submitted by: lidl
MFC after: 3 days
MFC with: r333262
Sponsored by: The FreeBSD Foundation

Ensure the ports and src trees are available on GCE images,
satisfying a requirement to allow FreeBSD to be considered
a top-tier supported OS in Google Compute Engine.

MFC after: 3 days
Sponsored by: The FreeBSD Foundation

Escape trailing newlines in a long variable list for consistency.

Submitted by: garga
MFC after: 3 days
Sponsored by: The FreeBSD Foundation

Remove google_accounts_manager from VM_RC_LIST in the GCE configuration
file, no longer needed.

PR: 221714
MFC after: 3 days
Sponsored by: The FreeBSD Foundation

Remove /etc/resolv.conf from virtual machine images, which is
copied from the build host. It is renamed to /etc/resolv.conf.bak
on boot, so never used anyway.

Noticed by: peter
MFC after: 3 days
Sponsored by: The FreeBSD Foundation

Use py-google-compute-engine instead for releasing Google Compute
Engine (GCE) images with an updated version of Google's tools.

PR: 221714
Submitted by: helen _dot_ koike _@_ collabora_dot_com (original)
MFC after: 5 days
Sponsored by: The FreeBSD Foundation

Partially revert r278118 now that the required logic for deciding whether
freebsd-update can be useful has moved into the firstboot_freebsd_update
script.

Use ifconfig_DEFAULT="" instead of hard-coding vtnet(4)

MFC after: 2 weeks
X-MFC-with: r277458, r277536, r277606, r277609,
r277836, r278118
Sponsored by: The FreeBSD Foundation

Move service_enable="YES" rc.conf(5) additions to
VM_RC_LIST and deduplicate.

Evaluate if firstboot_freebsd_update should be enabled
based on UNAME_r, because it is not supported for the
-CURRENT or -STABLE branches.

MFC after: 2 weeks
X-MFC-with: r277458, r277536, r277606, r277609,
r277836
Sponsored by: The FreeBSD Foundation

Add initial support for the GCE (Google Compute Engine)
cloud hosting provider image.

Many thanks to swills@ for his work on getting this to
this point.

Submitted by: swills
Sponsored by: The FreeBSD Foundation