release: Rework vm_extra_pre_umount

The vm_extra_pre_umount function in vmimage.subr served two purposes:
It removed /etc/resolv.conf and /qemu (if cross-building), and it
provided a function for cloudware to override in order to make cloud
specific changes to the filesystem before constructing a disk image.

This resulted in a number of bugs:
1. When cross-building, the emulator binary was left as /qemu in the
Azure, GCE, Openstack and Vagrant images.
2. The build host's resolv.conf was left as /etc/resolv.conf in the
basic-ci and basic-cloudinit images.
3. When building GCE images, a Google-specific resolv.conf file was
constructed, and then deleted before the disk image was created.

Move the bits needed for running code inside a VM staging directory
from vm_install_base into a new vm_emulation_setup routine, and move
the corresponding cleanup bits from vm_extra_pre_umount to a new
vm_emulation_cleanup routine.

Remove the /qemu and /etc/resolv.conf cleanups from the cloudware
configuration files (where they exist) since we will now be running
vm_emulation_cleanup to remove those even when vm_extra_pre_umount
has been overridden.

Override vm_emulation_cleanup in gce.conf since in that one case (and
*only* that one case) we don't want to clean up resolv.conf (since it
was constructed for the VM image rather than copied from the host).

releng/14.1 candidate.

MFC after: 1 week
Sponsored by: https://www.patreon.com/cperciva

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

release/azure: Update reference URL

Sponsored by: The FreeBSD Foundation

release/azure: Don't use azure-agent deprovision to build image

Deprovision feature of waagent is used for preparing to capture a
running VM and turn it into a VM image. Using it in the process of
building a VM image from scratch will cause some side effects such as
the hostname of the building host getting reset.

Remove calling the deprovision command and use a simpler way to fulfill
the requirements of the Azure VM image.

Sponsored by: The FreeBSD Foundation

Update Azure release bits

Imports the changes for building official images on Azure Marketplace,
which fulfill the requirements of Azure and FreeBSD cloud images like
disk layout and UEFI for Gen2 VM, along with some minor improvements like
configurations to speed up booting.

"CLOUDWARE" list will be updated after some more collaborations with re
completed.

Reviewed by: re (gjb)
Sponsored by: The FreeBSD Foundation
Technical assistance from: Microsoft
Differential Revision: https://reviews.freebsd.org/D23804

release: allow VM_EXTRA_PACKAGES to be specified in the environment

This is useful for adding extra packages to the build of an AMI.
For example:
env VM_EXTRA_PACKAGES="zsh" make -C release ec2ami

Approved by: gjb
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")

Chase r284656 and r284658:

Remove the Azure-local vm_extra_create_disk(), since we no longer
need qemu-img to convert the final VHD image to an Azure-compatible
format.

Although the waagent utility is installed from ports, create the
symlink to /usr/sbin, pending investigation on where this is
hard-coded, so it can be reported upstream. In the meantime, this
is good enough.

MFC after: 3 days
X-MFC-Needs: r284269, r284270, r284271, r284655,
r284656, r284657, r284658, r284659
X-MFC-Note: Required for 10.2-RELEASE, marcel@ has
implicit approval for the required changes
Sponsored by: The FreeBSD Foundation

Fix path to waagent.

MFC after: 3 days
Sponsored by: The FreeBSD Foundation

Remove vm_extra_install_base() for the Azure image, now
that the waagent exists in the ports tree.

Add sysutils/azure-agent to the VM_EXTRA_PACKAGES list.

In vm_extra_pre_umount(), remove the explicit pkg(8) install
list, as dependencies are resolved by sysutils/azure-agent.

Sponsored by: The FreeBSD Foundation

Revert r274773, after I *specifically* objected to this change
numerous times.

Sponsored by: The FreeBSD Foundation

Change how packages are installed into VM images: Rather than chrooting
into the image and running 'pkg install' from there, use 'pkg fetch' to
download packages into a temporary location and then 'pkg add' to install
them into the image.

This simplifies the code by avoiding the need to copy /etc/resolv.conf
into the image and then delete it later, and makes it possible to cross
build (e.g., to create an amd64 image when running on i386 hardware; or
in the future for building disk images for embedded platforms).

Because pkg was implicitly installed when VM_EXTRA_PACKAGES was non-empty,
add it to VM_EXTRA_PACKAGES in azure.conf and openstack.conf to maintain
the current behaviour.

By default repo-FreeBSD.sqlite is copied into the image, (a) to match
previous behaviour, where the file would be downloaded by the chrooted
pkg invocation; and (b) because it may be useful for testing purposes,
e.g., to see why a package didn't get installed. Because this file is
large (46 MB) and not likely to be useful in -RELEASE images which are
being launched into Clouds several months later, it can be disabled by
setting NOREPOSQLITE.

As far as I know this commit does not change the disk images produced in
any filesystem-visible way.

In vm_install_base(), copy the host resolv.conf into
the build chroot before attempting to do anything that
requires working DNS (i.e., pkg bootstrap).

In vm_extra_pre_umount(), remove the resolv.conf before
the disk image is unmounted from the backing md(4).

Reported by: cperciva
Sponsored by: The FreeBSD Foundation

Initial rewrite to consolidate VM image build scripts into one.
There may be some very sharp edges here while refactoring.

- Move amd64/mk-vmimage.sh -> scripts/mk-vmimage.sh.
- Remove vm-base target from Makefile.vm.
- In vm-image target, use getopts flags for argument passing.
- Create tools/vmimage.subr, containing default and prototype
for the following functions that are used to drive the build,
run in this order:

vm_install_base()
vm_extra_install_base()
vm_extra_install_packages()
vm_extra_install_ports()
vm_extra_enable_services()
vm_extra_pre_umount()
vm_create_disk()
vm_extra_create_disk()

- In tools/azure.conf, override:

vm_extra_install_base()
vm_extra_pre_umount()
vm_extra_create_disk()

- In tools/openstack.conf, override:

vm_extra_install_base()
vm_extra_pre_umount()

Sponsored by: The FreeBSD Foundation

Clear VM_RC_LIST.

Sponsored by: The FreeBSD Foundation

Add a separate make(1) target to release/Makefile to
build FreeBSD virtual machine disk images for use on
the Microsoft Azure service.

For now, this target is not directly connected to the
build, however can be manually invoked.

The 'vm-azure' target invokes {amd64,i386}/mk-azure.sh,
which does the heavy lifting to produce proper VHDs.
mk-azure.sh uses a configuration file, defaulting to
tools/azure.conf if otherwise unset.

Sponsored by: The FreeBSD Foundation