release: Build dvd image before staging pkgs

This missing dependency is only visible with -jN (which was otherwise
broken until recently) and only when building DVD images (which aren't
included in weekly snapshots and thereby slipped past testing).

MFC after: 3 days

release: Don't reuse disc1/bootonly directories

The disc1 and bootonly directories have files distributed into them
for use in "full" and "mini" images; the former are disc1.iso and
memstick.img, and the latter is bootonly.iso and mini-memstick.img.

Unfortunately the scripts which package a directory tree into an ISO
or memory stick image also modify the directory, for example to
create an appropriate /etc/fstab file; so creating two images at the
same time breaks.

Resolve this by copying disc1 to disc1-disc1 and disc1-memstick,
and copying bootonly to bootonly-bootonly and bootonly-memstick,
before using those directories for constructing the ISO+memstick
images.

MFC after: 1 week

release: distributekernel before packagekernel

With these as a single make command, `make -j` breaks when it tries to
package up a kernel which hasn't been distributed yet.

MFC after: 1 week

release: make -j compat: cd inside subshell

Place instances of "cd foo && bar" inside subshells for compatibility
with modern make(8) which uses a single shell for the duration of a
makefile target.

MFC after: 1 week

release: rework distributions list

Components like base.txz and ports.txz are called distributions in the
installer, and with the introduction of pkgbase we will start dealing
with normal pkg packages in the installer. Rename EXTRA_PACKAGES to
DISTRIBUTIONS, and move base.txz and kernel.txz to that list.

This introduces no functional change but is a small cleanup in advance
of some pkgbase experimentation.

Reviewed by: cperciva
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D43544

release: stop generating mergemaster databases

mergemaster was deprecated some time ago and will be removed from
FreeBSD 15.

Reviewed by: imp
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D41797

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

release: Fix make :@ modifier syntax

This spits out errors but seemingly isn't actually fatal, so was missed.

Fixes: 6853d893c7fe ("release: Automatically generate MK_LIB${LIBCOMPAT} and lib${libcompat}-dbg lists")

release: Automatically generate MK_LIB${LIBCOMPAT} and lib${libcompat}-dbg lists

Reviewed by: brooks, jhb
Differential Revision: https://reviews.freebsd.org/D41184

release: Set loader_menu_multi_user_prompt to "Installer" for installer

This makes it more obvious that the media being booted is an installer
rather than an installed system, which is otherwise hard to distinguish.
It also provides a more user-friendly, and more accurate, prompt.

Reviewed by: gjb
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D36419

release: Include an etcupdate tree in -DNO_ROOT release image builds

This makes use of the new -N etcupdate flag and merges the resulting
METALOG into base.meta re-rooted to /var/db/etcupdate/current.

Reviewed by: gjb
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D35858

release: Forward ${MAKE} to etcupdate via the new -m flag

This is in preparation for non-FreeBSD builds where make is GNU make and
so etcupdate needs to know the name of or path to the bmake binary to
use for its own builds.

Reviewed by: gjb
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D35855

release: Use in-tree etcupdate for build

This is in preparation for non-FreeBSD and -DNO_ROOT builds. On
non-FreeBSD there is no host etcupdate to use, and -DNO_ROOT will
require additional flags that may not be supported by the host's
etcupdate when building on FreeBSD. Moreover, there's no guarantee
anyway that the host's etcupdate is quite right for the current tree;
upgrading from source only requires that the host's is good enough for
-p which just manually copies master.passwd and group, the rest of the
upgrade is done post-installworld. For example, should a new set of
autogenerated files be added that etcupdate is taught about, the host
won't know about them and so the bootstrapped current tree will
incorrectly contain them, leading to spurious diffs on the installed
system.

Reviewed by: gjb, delphij
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D35853

Makefile.inc1 release bsd.own.mk: Introduce and use TAR_CMD

Our uses of tar rely on BSDisms, and so do not work in environments
where GNU tar is the default tar. Providing a TAR_CMD variable like
some other commands allows it to be overridden to use bsdtar in such
cases.

Reviewed by: brooks, delphij, gjb
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D35813

release: Support -DNO_ROOT image building

This requires a bunch of METALOG mangling to include the files we inject
into the tree. The mkisoimages.sh and make-memstick.sh scripts are now
called with the current directory inside the tree so that the relative
paths in the METALOG match up with the current directory. The scripts do
not require this when not using a METALOG, but for simplicity we always
do so. The Makefile mangles the real METALOG created from the install,
as those files are shared across all uses of the tree, but the shell
scripts create a temporary copy of the METALOG that they mangle as their
tree modifications are specific to that image. We also need to pass -D
to makefs to turn any duplicate METALOG entry errors into warnings, as
we have many (harmless) instances of those.

Whilst dvd1.iso should work, the !NOPKG code will need more work to
support this.

All media will also lack mergemaster and etcupdate trees, since more
work is needed to add -DNO_ROOT modes to them. Users of install media
built this way will have to manually bootstrap them.

Reviewed by: brooks, gjb
Differential Revision: https://reviews.freebsd.org/D33999

sys/*/conf/*, docs: fix links to handbook

While here, fix all links to older en_US.ISO8859-1 documentation
in the src/ tree.

PR: 255026
Reported by: Michael Büker <freebsd@michael-bueker.de>
Reviewed by: dbaio
Approved by: blackend (mentor), re (gjb)
MFC after: 10 days
Differential Revision: https://reviews.freebsd.org/D30265

release: move installworld before installkernel

To support -DNO_ROOT work. The top-level installworld target creates a
new METALOG starting with `#mtree 2.0` so it needs to be first, to avoid
overwriting installkernel METALOG entries.

Reviewed by: gjb
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D29582

release: permanently remove the 'reldoc' target and associates

Following 7b1d1a1658ffb69eff93afc713f9e88ed8b20eac, the structure
for the reldoc target has significantly changed as result of the
ASCIIDoctor/Hugo migration. As the release notes related files
on the installation medium are inherently out of date, purge them
entirely.

Discussed within: re, doceng
No objection: re (silence), doceng (silence)
Timeout: 2 weeks
MFC after: 1 week
MFC to: stable/13, stable/12, and stable/11 only
Sponsored by: Rubicon Communications, LLC ("Netgate")

release: disable the 'reldoc' target after the ASCIIDoctor switch

The 'reldoc' target includes release-related documentation on
installation medium. Since the switch from XML to ASCIIDoctor,
the file locations have moved, and it will take some time to sort
out how this target should work now.

MFC after: 3 days
Sponsored by: Rubicon Communications, LLC ("Netgate")

release: don't checksum images if there are none

For platforms that don't have any of the memstick, cdrom, or dvdrom
release images (i.e. riscv64), the release-install target will trip up
when invoking md5(1) on the non-existent image files. Skipping this
allows the install to complete successfully.

Makefile.inc1:
- New file. Adds logic to search for the git binary, as well
as determining the branch and revision, used in various
places.

Makefile:
- Remove searching for the svn{,lite} binary.

Makefile.ec2:
- Reduce duplicated code, removing searching for the svn{,lite}
binary, in addition to EC2_SVN{BRANCH,REV}.
- Rename EC2_SVN* with GIT* for consistency.

Makefile.mirrors:
- Remove the SRCBRANCH declaration, replaced with the exported
GITBRANCH variable.
- Update _SNAP_SUFFIX from SVNREVISION to GITREV, and remove
the leading 'r' from it, since it will break git hashes.
- Remove yet another instance of duplicated code to search for
the svn{,version}lite binary.

Sponsored by: Rubicon Communications, LLC (netgate.com)

retire amd(8)

autofs was introduced with FreeBSD 10.1 and is the supported method for
automounting filesystems. As of r296194 the amd man page claimed that it
is deprecated. Remove it from base now; the sysutils/am-utils port is
still available if necessary.

Discussed with: cy
Relnotes: Yes
Sponsored by: The FreeBSD Foundation

Fix a typo.

MFC after: 3 days
Sponsored by: Rubicon Communications, LLC (netgate.com)

Remove setting of long-unused MK_NCP option

NCP was removed from the tree in 2013 (r248097) and disconnected some
time before.

Reported by: bz
Sponsored by: The FreeBSD Foundation

Pass _BRANCH in the reldoc target to ensure the correct
src branch is used when generating the hardware.html page.

MFC after: 3 days
MFC before: 12.0-BETA3
MFC with: r339873
Sponsored by: The FreeBSD Foundation

Add debug.witness.trace=0 back to the installer sysctl.conf(5),
incorrectly removed from head when it should have been removed
from stable/12 post-branch.

Reported by: bdrewery
Sponsored by: The FreeBSD Foundation

- Update head to 13.0-CURRENT.
- Bump MACHINE_TRIPLE, TARGET_TRIPLE, FBSD_MAJOR, FBSD_CC_VER,
FREEBSD_CC_VERSION, OS_VERSION.
- Update comment in UPDATING regarding debugging options.
- Remove debug.witness.trace=0 from installation media.
- Bump __FreeBSD_version.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Disable kernels_autodetect on installation media

This feature is disabled on install media as these generally won't have any
interesting kernels to be listed other than the default kernel, so the
potential performance penalty in these situations likely isn't worth it.

Approved by: re (kib)

Fix the hardware.{txt,html} build in the release/Makefile 'reldoc'
target.

The doc/share/mk/doc.commands.mk sets SVN to /usr/local/bin/svn
by default, which is not necessarily installed by the documentation
project textproc/docproj port.

Ensure SVN can be evaluated properly to include the hardware pages
by iterating through /usr/local/bin and /usr/bin and looking for
both svn and svnlite binaries, and pass the SVN variable explicitly
through env(1) in the reldoc target to avoid failures if it does not
exist.

Approved by: re (rgrimes)
Sponsored by: The FreeBSD Foundation

build: skip the database check when generating install media

There are several scripts and targets solely used to generate install
media, make sure DB_FROM_SRC is used in that case in order to prevent
checking the host database, which is irrelevant when generating
install binaries.

Sponsored by: Citrix Systems R&D
PR: 230459
Reviewed by: gjb
Differential revision: https://reviews.freebsd.org/D16638

Fix the reldoc target in release/Makefile following doc commit r51264.

Sponsored by: The FreeBSD Foundation

Run mm-mtree with whatever -j value the build is using.

Reviewed by: gjb
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12807

Fix the 'reldoc' target, following doc commit r51047.

Sponsored by: The FreeBSD Foundation

Allow passing NOPKG= to make(1) to prevent the pkg-stage target
from getting executed.

MFC after: 3 days
Sponsored by: The FreeBSD Foundation

Remove groff from base

All manpages in base are now compatible with mandoc(1), all roff documentation
will be relocated in the doc tree. man(1) can now use groff from the ports tree
if it needs.

Also remove checknr(1) and colcrt(1) which are only useful with groff.

Approved by: (no objections on the mailing lists)

Allow PORTSDIR to be overridden in the pkg-stage target.

Submitted by: Johannes Jost Meixner (original)
PR: 218393
MFC after: 3 days
Sponsored by: The FreeBSD Foundation

Make sure we clean up the rdoc and reldoc directories if NODOC
is not set.

MFC after: 3 days
Sponsored by: The FreeBSD Foundation

Reduce the set of things on the disc1 image to those on the bootonly image.
Notably, this removes the toolchain from the CD-ROM image (it remains on DVD
images) and pushes the CD-ROM image well below the 700 MB mark.

MFC after: 10 days

Remove vestigial cloudware-install bits missed in r300767.

Sponsored by: The FreeBSD Foundation

Remove misspelled and redundant MK_INSTALLIB=no.

Add a new target to touch the ${.OBJDIR}/release file, which
indicates the 'release' target has run (in order to prevent
subsequent invocations that may clobber original build output).

As is, the 'release' target is a dummy target that does nothing
more than depend on subsequent targets. Unless 'make obj' is
invoked prior to 'make release', .OBJDIR and .CURDIR will always
be '/usr/src/release' (or wherever /usr/src is located).

When 'make release' invokes 'make real-release' (and subsequent
targets), .OBJDIR is not updated, which still leads to src/ tree
pollution.

While arguably a hack, 'make release' will invoke the original
dummy targets as originally intended, but instead of touching an
empty file (or returing @true), will call a 'release-done' target
that will trigger the behavior that was intended to prevent
a subsequent invocation.

Discussed with: hrs
MFC after: 3 days
X-MFC-With: r293173
Sponsored by: The FreeBSD Foundation

Merge ^/projects/release-install-debug:

- Rework MANIFEST generation and parsing via bsdinstall(8).
- Allow selecting debugging distribution sets during install.
- Rework bsdinstall(8) to fetch remote debug distribution sets
when they are not available on the local install medium.
- Allow selecting additional non-GENERIC kernels during install.
At present, GENERIC is still required, and installed by default.

Tested with: head@r293203
Sponsored by: The FreeBSD Foundation

Fix path to include .OBJDIR to avoid polluting the source
tree during 'make release'.

MFC after: 3 days
Sponsored by: The FreeBSD Foundation

Deprecate MD5 checksum generation in favor of SHA512.

This was discussed during the 10.2-RELEASE cycle, however
since we were nearing the end of the cycle, we decided to
defer this change until after 10.2-RELEASE.

Reminded by: so (delphij), jmg
MFC after: 5 days
Sponsored by: The FreeBSD Foundation

Whitespace alignment

MFC after: 3 weeks
X-MFC-to: stable/10
X-MFC-with: 287696

The <arch>/mkisoimages.sh script in release knows how to add
extra bits from an "xtra-bits-dir". This feature is unusable
from release/Makefile. Add an XTRADIR setting to use it.

Differential Revision: https://reviews.freebsd.org/D3633
Reviewed by: kmacy
MFC after: 3 weeks
X-MFC-to: stable/10
Relnotes: yes

Remove arm/Makefile.{KERNEL} files, as this project
branch is now going in a different direction than
originally planned.

Sponsored by: The FreeBSD Foundation

Add 'cloudware-install' target to handle copying cloud
disk images to a directory within DESTDIR.

Add CLOUDINSTALL variable to automatically generate the
actual list of install targets. While here, rename the
cloud-specific names from 'vm-' to 'cw-' to avoid naming
collisions.

Add AZURE_DISK, GCE_DISK, and OPENSTACK_DISK output file
variables.

Add 'cloudware-install' to the 'install' target.

MFC after: 1 week
Sponsored by: The FreeBSD Foundation

release: use xz via pipe when compressing the tarballs.

libarchive(3) doesn't support the new liblzma API yet, but this change
allows us to enable multi-threaded xz compression.
``make release'' should now finish in half the time on a machine with
several cores and fast disks (our typical build server).

This behaviour only applies when building a release and it doesn't
affect buildworld/installworld. To disable threaded xz compression,
set XZ_THREADS=1.

Reviewed by: gjb
Tested by: gjb

Initial restructure for FreeBSD/arm release builds.

Sponsored by: The FreeBSD Foundation

Revert r278445.

I was going to use __FreeBSD_version to determine if
xz(1) should be multi-threaded by default, but doing
this will cause problems if/when the changes are merged
from head.

Sponsored by: The FreeBSD Foundation

Enable multi-threaded xz(1) compression, after r278433.
Allow multi-threaded xz(1) to be turned off by specifying
NO_XZTHREADS, and allow number of threads to be overridden
by specifying XZ_THREADS=N.

MFC after: 1 week
X-MFC-needs: r278433
Sponsored by: The FreeBSD Foundation

Define FREEBSD_VERSION in release/Makefile, derived
from sys/sys/param.h.

Sponsored by: The FreeBSD Foundation

Rename the 'system' target to 'disc1', which is specific
to the disc1.iso installer image.

Replace 'system' with 'release' in CLEANFILES, and add
'disc1' to CLEANDIRS.

Ensure the 'dvd' target depends on 'packagesystem'.

Fix 'mini-memstick.img' prerequisite, which should be the
'bootonly' target, not 'disc1' (previously 'system').

Use .TARGET as the target installation directory for the
disc1.iso and bootonly.iso images, which now expand to
'disc1' and 'bootonly' respectively, mimicking the behavior
of the 'dvd' target.

Remove '@true' from the 'release' target, and instead use
'touch ${.TARGET}' to prevent multiple iterations of
'make release' from clobbering previously-built installer
medium.

Tested with: head@r277834
MFC after: 3 weeks
X-MFC-with: r277458, r277536, r277606, r277609
Sponsored by: The FreeBSD Foundation

Provide a recipe of "true" for building the "release" target once its
prerequisites are satisfied, in order to avoid having an implicit rule
triggered by the presence of release.sh.

Approved by: gjb

Rename the 'release' target to 'real-release', and
add two targets, 'vm-release' and 'cloudware-release',
that are invoked if WITH_VMIMAGES and WITH_CLOUDWARE
are not empty.

This fixes an issue where 'make release' would not
build the cloud provider targets because CLOUDWARE was
not yet set. [1]

Move the WITH_VMIMAGES and WITH_CLOUDWARE targets to
Makefile.vm.

Note: There is no 'cloudware-install' target yet, since
some of the disk image names may need to be specific to
the provider, so this is probably best handled by the
build scripts.

Reported by: cperciva [1]
MFC after: 1 month
X-MFC-with: r277458
Sponsored by: The FreeBSD Foundation

Use the correct loader.conf(5) entry for deferring the
mountroot prompt.

Prodded by: ian
Sponsored by: The FreeBSD Foundation

Delay dropping to the mountroot prompt for 10 seconds
in case the root device is not yet ready.

Sponsored by: The FreeBSD Foundation

Fix VOLUME_LABEL when BRANCH contains non-alpha
characters, in particular '-' and '.'.

Submitted by: des
Tested by: des
MFC after: 3 days
X-MFC-10.1: yes
Sponsored by: The FreeBSD Foundation

Add glue to allow enabling building cloud provider
VM images by default.

When WITH_CLOUDWARE is not empty, add CLOUDTARGETS
to the release/Makefile 'release' target.

CLOUDTARGETS is generated from the contents of
CLOUDWARE, which should be a list of all supported
target providers.

Sponsored by: The FreeBSD Foundation

Move virtual machine / cloud provider targets and
options from release/Makefile to their own Makefile.

Sponsored by: The FreeBSD Foundation

Fix a few issues with creating VOLUME_LABEL for the
installation ISOs:

- TYPE, BRANCH, and REVISION are only defined if
OSRELEASE is not defined, so in situations where
one might set OSRELEASE for an in-house ISO build,
VOLUME_LABEL would be empty.

- makefs(8) limits the volume label to 32 characters,
which for the powerpc64 case, OSRELEASE expands to
FreeBSD-11.0-CURRENT-powerpc-powerpc64. Even with
removing the prefixing 'FreeBSD-', the string is 30
characters long, leaving zero room for suffixing the
type of ISO media (BO for bootonly, CD for cdrom, and
DVD for dvdrom).

Resolve these by defining VOLUME_LABEL when defining
OSRELEASE if unset. If OSRELEASE is defined by the
builder, use the OSRELEASE from that definition as the
VOLUME_LABEL.

In addition, for cases where both TARGET and TARGET_ARCH
are used for the VOLUME_LABEL, use TARGET_ARCH if it
differs from TARGET.

There are probably a few sharp edges here yet, but these
problems are going to affect the powerpc/powerpc64 builds
for 10.1-RELEASE, so the immediate concern is fixing the
underlying problem at hand quickly, and less so about the
elegance of the fix.

MFC after: 3 days
X-MFC-10.1: yes, asap
Sponsored by: The FreeBSD Foundation

Add more descriptive metadata to the ISO images.

PR: 146079
Submitted by: Roman Bogorodskiy
MFC after: 3 days
X-MFC-10.1: yes
Sponsored by: The FreeBSD Foundation

Add a separate make(1) target to release/Makefile to
build FreeBSD virtual machine disk images for use on
the Microsoft Azure service.

For now, this target is not directly connected to the
build, however can be manually invoked.

The 'vm-azure' target invokes {amd64,i386}/mk-azure.sh,
which does the heavy lifting to produce proper VHDs.
mk-azure.sh uses a configuration file, defaulting to
tools/azure.conf if otherwise unset.

Sponsored by: The FreeBSD Foundation

Remove two stray comments added during the initial iterations
of testing, no longer needed.

Sponsored by: The FreeBSD Foundation

Add WITH_COMPRESSED_VMIMAGES variable, which when set enables
xz(1) compression of the virtual machine images.

This is intentionally separate to allow more fine-grained
tuning over which images are compressed, especially in cases
where compressing 20GB sparse images can take hours.

Sponsored by: The FreeBSD Foundation

After evaluating WITH_VMIMAGES is non-empty, ensure
the mk-vmimage.sh script exists before running it.

Sponsored by: The FreeBSD Foundation

Avoid using env(1) to set values passed to mk-vmimage.sh,
and instead pass the values as arguments to the script,
making it easier to run this by hand, without 'make release'.

Add usage_vm_base() and usage_vm_image() usage helpers.

Sponsored by: The FreeBSD Foundation

Separate release/scripts/mk-vmimage.sh to machine-specific
scripts, making it possible to mimic the functionality for
non-x86 targets.

Move echo output if MAKEFLAGS is empty outside of usage().

Remove TARGET/TARGET_ARCH evaluation.

Sponsored by: The FreeBSD Foundation

release/Makefile:
Include .OBJDIR in DESTDIR in the vm-base target.

release/release.sh:
Provide the full path to mddev.

Sponsored by: The FreeBSD Foundation

release/Makefile:
Connect the virtual machine image build to the release
target if WITH_VMIMAGES is set to a non-empty value.

release/release.sh:
Add WITH_VMIMAGES to RELEASE_RMAKEFLAGS.

release/release.conf.sample:
Add commented entries for tuning the release build if the
WITH_VMIMAGES make(1) environment variable is set to
a non-empty value.

Sponsored by: The FreeBSD Foundation

Use VMBASE in place of a hard-coded filename in the CLEANFILES list.

Sponsored by: The FreeBSD Foundation

Initial commit to include virtual machine images as part
of the FreeBSD release builds.

This adds a make(1) environment variable requirement,
WITH_VMIMAGES, which triggers the virtual machine image
targets when not defined to an empty value.

Relevant user-driven variables include:

o VMFORMATS: The virtual machine image formats to create.
Valid formats are provided by running 'mkimg --formats'

o VMSIZE: The size of the resulting virtual machine
image. Typical compression is roughly 140Mb, regardless
of the target size (10GB, 15GB, 20GB, 40GB sizes have been
tested with the same result).

o VMBASE: The prefix of the virtual machine disk images.
The VMBASE make(1) environment variable is suffixed with
each format in VMFORMATS for each individual disk image, as
well as '.img' for the source UFS filesystem passed to
mkimg(1).

This also includes a new script, mk-vmimage.sh, based on how
the VM images for 10.0-RELEASE, 9.3-RELEASE, and 10.1-RELEASE
were created (mk-vmimage.sh in ^/user/gjb/thermite/).

With the order in which the stages need to occur, as well as
sanity-checking error cases, it makes much more sense to
execute a shell script called from make(1), using env(1) to
set specific parameters for the target image than it does to
do this in make(1) directly.

Sponsored by: The FreeBSD Foundation

Merge ^/projects/release-debugdist into ^/head:
r262491, r262493, r262516, r267345, r267397:

r262491:
Add DEBUG_DISTRIBUTIONS, and set it to include base and
EXTRA_DISTRIBUTIONS, excluding 'doc', since the documentation
distribution does not have corresponding debug information.

Use DEBUG_DISTRIBUTIONS in the 'distributeworld installworld'
and 'packageworld' targets, to reduce the number of occurances
of excluding distributions that do not have .debug files.

r262493:
In release/Makefile, explicitly set WITHOUT_DEBUG_FILES=1
for dvdrom and cdrom targets. (Later reverted.)

Exclude the *.debug.txz distributions from dvdrom and
cdrom images, but include them for ftp distribution.

r262516:
Rename ${dist}.debug.txz to ${dist}-dbg.txz to prevent the
following output:
eval: ${base....}: Bad substitution
eval: ${doc....}: Bad substitution
eval: ${games....}: Bad substitution
eval: ${lib32....}: Bad substitution

This also follows other naming conventions seen in the
wild.

r267345:
Explicitly set MK_DEBUG_FILES=no, which overrides the
WITH_DEBUG_FILES=1 and WITHOUT_DEBUG_FILES=1 collisions
previously experienced.

This change allows us to create the {base,kernel}_debug.txz
distributions without accidentally installing the *.debug
files on the medium itself.

r267397:
Remove evaluations of MK_DEBUG_FILES where not needed.
If DEBUG_DISTRIBUTIONS is empty, which is true if
MK_DEBUG_FILES evaluates to 'no' above, the loop does
nothing.

MFC after: 1 month
Tested on: head@r267801
Reviewed by: brooks [1], emaste, imp [1]
[1] earlier version
Sponsored by: The FreeBSD Foundation

Switch release/Makefile knobs from WITHOUT_FOO=1 to MK_FOO=no.

This allows greater granular control over the release
build, and avoids WITH_FOO=1 and WITHOUT_FOO=1 collision.

This change was verified to produce correct results
compared to an earlier build by diffing find(1) output
of the resulting bootonly/, dvd/, and release/ directories.

Tested on: head@r267449
MFC after: 1 month
X-MFC-Note: Requires commits not yet MFC'd, so likely
longer than 1 month.
Sponsored by: The FreeBSD Foundation

Strip out the test suite from the installation media.

This affects the disc1.iso, dvd1.iso and bootonly.iso files.

Obtained from: gjb

Fix indentation level.

MFC after: 3 days
X-MFC-With: r267326
Sponsored by: The FreeBSD Foundation

Add empty pkg-stage file to CLEANFILES if WITH_DVD=1.

MFC after: 3 days
Sponsored by: The FreeBSD Foundation

Turn off the full witness trace on console output.

On head/, or more specifically, when WITNESS is in
the kernel config, the console is spammed excessively
with lock order reversal between isofs and devfs.

Set debug.witness.trace=0 in the installer sysctl.conf
to avoid printing the full KDB stack backtrace. This
does not prevent printing the lock order reversal has
happened, only lessens the console spam.

Sponsored by: The FreeBSD Foundation

Allow overriding xz(1) path.

Requested by: marck
MFC After: 5 days
X-MFC-With: r264027,r264028,r264029,r264030,r264046
Sponsored by: The FreeBSD Foundation

Use xz(1) instead of gzip(1) to compress release images
when WITH_COMPRESSED_IMAGES is used.

Requested by: delphij, brooks, Nikolai Lifanov
MFC After: 1 week
X-MFC-With: r264027,r264028,r264029,r264030
Sponsored by: The FreeBSD Foundation

If WITH_COMPRESSED_IMAGES is set, add the compressed images to
the CLEANFILES list.

MFC after: 1 week
X-MFC-With: r264027,r264028,r264029
Sponsored by: The FreeBSD Foundation

Fix logic error: s/||/&&/

Pointyhat: gjb
MFC after: 1 week
X-MFC-With: r264027,r264028
Sponsored by: The FreeBSD Foundation

Clean up trailing whitespace in release/Makefile.

MFC after: 1 week
X-MFC-With: r264027
Sponsored by: The FreeBSD Foundation

Add a new release build variable, WITH_COMPRESSED_IMAGES.

When set to a non-empty value, the installation medium is
compressed with gzip(1) as part of the 'install' target in
the release/ directory.

With gzip(1) compression, downloadable image are reduced in
size quite significantly. Build test against head@263927
shows the following:

bootonly.iso: 64% smaller
disc1.iso: 44% smaller
memstick.img: 47% smaller
mini-memstick.img: 65% smaller
dvd1.iso: untested

This option is off by default, I would eventually like to
turn it on by default, and remove the '-k' flag to gzip(1)
so only compressed images are published on FTP.

Requested by: wkoszek
MFC After: 1 week
Sponsored by: The FreeBSD Foundation

In release/Makefile, remove exclusion of CVS directories in the
src/ and ports/ distributions.

While I am thinking about it, exclude .git directories for src/
and ports/, as somewhat of a followup to r262499.

Sponsored by: The FreeBSD Foundation

etcupdate should use the src tree from where the release is built, not
the default (/usr/src) tree.

MFC after: 2 weeks

Update the pkg-stage target to be more compatible with pkg-1.2:

- Add a release-dvd.conf pkg(8) configuration file to override
the default FreeBSD.conf configuration.
- Remove architecture-specific pkg-stage.conf files, consolidate,
and move their contents to scripts/pkg-stage.sh.
- Use 'pkg -vv' to determine the ABI, which is used as the
cache directory.

Prior to these changes, it would be possible for pkg-stage to fetch
conflicting binary packages from multiple repositories.

Tested against: head@r260522, stable/10@r260522

MFC after: 3 days
X-Insta-MFC: possibly
Sponsored by: The FreeBSD Foundation

Bootstrap etcupdate(8) as part of the release build, similar
to what is done for mergemaster(8). This allows etcupdate(8)
to work out-of-box after the first upgrade of a system.

Submitted by: jhb
MFC after: 3 days
Sponsored by: The FreeBSD Foundation

Add a pkg(8) repository configuration file for cdrom-based package
installation.

As part of the 'pkg-stage' target, copy the configuration file
to the 'packages/repos/' directory on the DVD filesystem.

MFC after: 3 days
X-MFC-Before: -RC3
Sponsored by: The FreeBSD Foundation

Reduce disc1.iso size by 74MB by removing lib32 libraries, sendmail,
and atf.

MFC after: 3 days
X-MFC-To-10: immediate (-RC2 requirement)
Sponsored by: The FreeBSD Foundation

Ensure WITH_DVD is not empty to prevent 'WITH_DVD= ' from evaluating
to true.

Submitted by: hrs
MFC after: 3 days
X-MFC-to-10: immediate
Sponsored by: The FreeBSD Foundation

Turn off the dvdrom target by default. dvd1.iso is now built by
specifying 'WITH_DVD=1' during 'make release'.

This caused some problems during the freebsd-update builds for
10.0-BETA4.

MFC after: 3 days
Sponsored by: The FreeBSD Foundation

Use standard CLEANFILES/CLEANDIRS and clean target in bsd.obj.mk.

- Prevent TARGET and TARGET_ARCH from being defined as empty when
TARGET="" and/or TARGET_ARCH="" is specified.
- Remove extra ().

Add NOPKG to disable pkg-stage.

Document the dvdrom target and NODVD variable, forgotten with r258310.

MFC after: 3 days
Sponsored by: The FreeBSD Foundation

Remove WITHOUT_PROFILE=1 for the dvd1.iso medium.

Requested by: hrs
MFC after: 3 days
X-MFC-With: r258310
Sponsored by: The FreeBSD Foundation

Add the 'dvd1.iso' target. This mimics the 'release.iso' target,
with the additional step of fetching packages for inclusion on the
dvd image.

The 'pkg-stage' target is used to run 'scripts/pkg-stage.sh' if
the '${TARGET}/pkg-stage.conf' configuration file exists (currently
only amd64 and i386).

Allow dvd1.iso to be skipped if NODVD=1.

MFC after: 3 days
X-MFC-With: r258305, r258307, r258308, r258309
Sponsored by: The FreeBSD Foundation

Use the IMAGES variable to determine which image files to remove
as part of 'make -C /usr/src/release clean'.

MFC after: 3 days
Sponsored by: The FreeBSD Foundation

Remove extra target from 'memstick' that I forgot to remove before
previous commit.

MFC after: 1 week
X-MFC-After: 10.0-RELEASE
X-MFC-With: r257805
Sponsored by: The FreeBSD Foundation

Add a 'mini-memstick.img' release target, which will use the
'bootonly.iso' components to create a smaller memory stick image.

This is useful for system recovery, where a full memstick.img image
is not necessarily needed (or wanted). In addition, it is possible to
do bootonly-style installation, where the base.txz, kernel.txz, etc.
are fetched from a remote source.

Provide backwards-compatible target (mini-memstick), to keep in sync
with the targets documented in release/Makefile.

Prompted by: wblock
MFC after: 1 week
X-MFC-after: 10.0-RELEASE
Sponsored by: The FreeBSD Foundation

Reduce disc1.iso image size by installing the userland with
the WITHOUT_PROFILE=1 option set, trimming 56MB from the image.

Approved by: re (glebius)
MFC after: 1 week
Sponsored by: The FreeBSD Foundation

Remove BIND.

Approved by: re (gjb)

For consistency with previous releases, name the cdrom install
cd 'disc1.iso'.

Discussed with: re@ team
Approved by: kib (mentor)
MFC after: 3 days
X-MFC-To: stable/9 only

Create memstick images with the '.img' filename extension.

Requested by: many
Approved by: kib (mentor)
MFC after: 3 days
X-MFC-To: stable/9 only

Convert old make variable modifiers :U and :L to bmake :tu and :tl.

Reviewed by: sjg

- Add CHECKSUM.* support in Makefile[1].
- Use ln -fs to create a symlink.
- Remove pkgadd for docports.
- Use WITHOUT_JADETEX=yes instead of WITH_JADETEX=no.
- Add {WORLD,KERNEL}_FLAGS to [BTWK]MAKE.
- Use makefs(8) and gpart(8) for sparc64 ISO image[2].
- Add publisher option to makefs(8)[2].

Based on work by: gjb[1]
Discussed with: marius, nwhitehorn[2]

Prevent including .zfs snapshot directories in the src.txz
distribution. This can happen if the src/ tree checkout is
within its own ZFS dataset, and the 'snapdir' ZFS property
is set to 'visible.'

Approved by: hrs
MFC after: 3 days
X-MFC-To: stable/9 only

Exclude potential circular symlink

This does not occur with normal release builds but can happen when reusing
these release parts for other purposes.

Exclude potential circular symlink when creating src tarball.

Normal release builds are not affected by this but it can happen if using
these release bits against a work tree.

Sponsored by: ADARA Networks

Make sure not to tar up CVS directories if this ports tree is a CVS
checkout.

PR: bin/165868
MFC after: 5 days

- Clean up extra ${.OBJDIR}.
- Add ${IMAGE} for the supported image files. This fixes the install target
on FreeBSD/pc98.
- Use "mkdir -p" instead of "-mkdir" consistently.

Reviewed by: nwhitehorn

Restore functionality to pack several kernels into release. All
kernels specified by KERNCONF are built and packed into release.
The first one is packed into kernel.txz, all others to
kernel.CONFIG.txz.

The first one is installed on bootables in /boot.

Allow release generation from a WORLDDIR that is not /usr/src

PR: misc/159666
Submitted by: Test Rat <ttsestt@gmail.com>
Reviewed by: Nathan Whitehorn <nwhitehorn@freebsd.org>
Approved by: re (kib)

Per request of the docs team, install docs as packages, instead of via
the normal distfile mechanism. Thanks to Marc Fonvieille for the patch and
for putting up with me taking entirely too long to commit this!

Submitted by: blackend

Make the bootonly CD smaller by discarding static libraries that are not
useful without a toolchain.

Do not install the rescue tools onto the install CD. Since it is read
only, they are very unlikely to be needed and take up a great deal of
space.

As requested by many people, with final prodding from Jason Hall, fix this
so that running make release causes make obj to be run before doing
anything. This fixes a bug wherein, when run for the first time, and
without -DNOSRC, make release would attempt to recursively tar up the src
directory including its own output and enter an infinite loop.

While here, make the cross-building stuff work a little more the way it
should if only one of TARGET/TARGET_ARCH is specified.

Mention the memstick target here.

Add an install target to release/Makefile instead of grubbing around
(and requesting that users grub around) in /usr/obj. Also make sure that
make clean removes the bootonly media and clean up that target a little.

Add infrastructure for generation of boot-only ISO images. These have a
reduced world, though still reasonably complete for fixit purposes.

Cause the release target to set up its object directory. It is designed
to be a meta-target, and having it do this avoids it spewing files all
over the source tree by default (and possible recursive tarring if
distributing a src tarball).

Use labels to find release media instead of hard-coded device paths. This
makes booting more reliable (and working at all on USB sticks). While here,
move responsibility for setting up fstab into the various platform mk-*.sh
scripts.

Suggested by: many

Thanks to sysinstall for its 15 years of service. And now to sleep,
perchance to dream...

Improve the distributeworld target in Makefile.inc1 and update the release
infrastructure to use it. make distributeworld can now be used without
preparing its environment first and installs games into its distribution
using the regular make distribute logic instead of post-processing with
a script.

Also add two new targets, packageworld and packagekernel, that tar up the
results of distributeworld and distributekernel (also new), respectively.

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Get powerpc64 install CDs a little closer: some powerpc things should be
used on powerpc64 as well (build scripts, for instance), and the MFS root
must be adjusted upward on powerpc64 as well.

This doesn't completely solve things: sysinstall stil needs some
persuasion to deal with a default kernel named GENERIC64.

Connect powerpc64 to the build. It is not presently part of make universe,
which will be added soon.

Reviewed by: imp

Bump version number from 8.0 to 8.1. Belatedly bump stable/7 to stable/8
for SVNBRANCH.

Approved by: re (implicit)

Merge r206423:

> Shift the version of perl used by the release build process over to
> perl-5.10. This aligns the release build process with the current
> default version of perl in the ports tree.

Merge r206422:

> Pass the HTTP_PROXY and FTP_PROXY environment variables through in addition
> to FTP_PASSIVE_MODE so release building works for a machine that needs
> to use a proxy.

PR: misc/137688
Submitted by: Michael Leun

Shift the version of perl used by the release build process over to
perl-5.10. This aligns the release build process with the current
default version of perl in the ports tree.

I'll MFC this to all active branches.

MFC after: 3 days

Pass the HTTP_PROXY and FTP_PROXY environment variables through in addition
to FTP_PASSIVE_MODE so release building works for a machine that needs
to use a proxy.

PR: misc/137688
Submitted by: Michael Leun
MFC after: 2 weeks

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

MFC: revision 201391

Do kgzip to the loader on pc98, too. Now pc98's boot2 works for ELF.

MFC r200775:

Add FreeBSD- to the beginning of the ISO image filenames.

Do kgzip to the loader on pc98, too. Now pc98's boot2 works for ELF.

MFC after: 2 week

Add "FreeBSD-" to the beginning of the filenames for the ISO images.
People who collect ISOs from more than just us have been requesting
this for a while.

MFC after: 1 week

MFC r197313:
Build a separate livefs CD for sparc64.

Approved by: re (kib)

Have sparc64 join the separate livefs crowd. Including the docs packages
on disc1 (needed now by sysinstall) overflows the target CDROM iso size.

MFC after: 3 days

MFC: r196588

Log:
Update some comments to match the removal of the docset compilation
during the release build.

Approved by: re (kib)

MFC 196522:
Invoke the recently added mm-mtree.sh release script to store a pre-built
mergemaster mtree database in the 'base' dist.

Approved by: re (kib)

MFC 196521:
Fix a few issues with the lib32 dist so that it includes ldd32.
- Use a better find invocation to purge empty directories from all the dist
trees during a release build. The previous version did not purge
directories whose contents were all empty directories.
- Explicitly blacklist a few files from the lib32 dist instead of using a
whitelist. A better longterm solution is to fix the few offenders to not
install data files during a lib32 install.

Approved by: re (kib)

Update some comments to match the removal of the docset compilation
during the release build.

Invoke the recently added mm-mtree.sh release script to store a pre-built
mergemaster mtree database in the 'base' dist.

MFC after: 3 days

Fix a few issues with the lib32 dist so that it includes ldd32.
- Use a better find invocation to purge empty directories from all the dist
trees during a release build. The previous version did not purge
directories whose contents were all empty directories.
- Explicitly blacklist a few files from the lib32 dist instead of using a
whitelist. A better longterm solution is to fix the few offenders to not
install data files during a lib32 install.

MFC after: 3 days

Make head 9.0-CURRENT in preparation for lifting code freeze.

Approved by: re (implicit)

MFC 196221:

Add the ability to build a release from an SVN checkout instead of a CVS
checkout. If SVNROOT is specified, then the source tree will be checked
out from that SVN repository instead of using CVS. ports and docs still
use CVS. If SVNROOT is not specified, then the source tree will be checked
out using CVS. An explicit SVN branch can be specified using SVNBRANCH
(e.g. SVNBRANCH=stable/8). If SVNBRANCH is not set but RELEASETAG is set
to a CVS branch (such as RELENG_8) the appropriate SVN branch will be
inferred from the CVS branch using svnbranch.awk.

Note that there are still several open questions about using SVN instead
of CVS in the release process. However, this does enable one to build a
release from an SVN repository if needed.

Approved by: re (kensmith)

Add the ability to build a release from an SVN checkout instead of a CVS
checkout. If SVNROOT is specified, then the source tree will be checked
out from that SVN repository instead of using CVS. ports and docs still
use CVS. If SVNROOT is not specified, then the source tree will be checked
out using CVS. An explicit SVN branch can be specified using SVNBRANCH
(e.g. SVNBRANCH=stable/8). If SVNBRANCH is not set but RELEASETAG is set
to a CVS branch (such as RELENG_8) the appropriate SVN branch will be
inferred from the CVS branch using svnbranch.awk.

Note that there are still several open questions about using SVN instead
of CVS in the release process. However, this does enable one to build a
release from an SVN repository if needed.

Approved by: re (kensmith)

Adjust for RELENG_8.

Approved by: re (implicit)

- release/* update to use freebsd-doc-* packages instead of building
FreeBSD docset during 'make release' this will speed up release
builds;
- sysinstall(8) has also been updated to use these packages with a new
menu allowing people to choose what localized doc to install;
- mention in UPDATING that docs from the FreeBSD Documentation project
are now installed in /usr/local/share/doc/freebsd instead of
/usr/share/doc.

Approved by: re (kensmith)

Make a DVD for ia64.

Restore MAKE_FLOPPIES for pc98.

Turn off the parallel make options accidentally turned on by the previous
commit. Sigh.

Noticed by: rdivacky

Turn off the building of boot floppies for amd64/i386.

This is being committed from a sparc64 (US-III, thanks Marius!) that
was installed from a DVD so apparently it works... :-)

Enable building DVDs for sparc64.

Move an echo that accidentally got put under the .if defined(MAKE_DVD).

Pointy hat: kensmith

Oops, add missing @.

- Import the HEAD csup code which is the basis for the cvsmode work.

The thought of making more than one DVD image for a release really
freaks me out. But it turns out we might be able to generalize
a few of the other things RE uses to assemble the package trees
for releases if the DVDs use a naming theme close to what is used
for the CDROMS (disc1, disc2, etc). So change the name to dvd1.
Hopefully this way src/release/scripts/{package-split.py,package-trees.sh}
can be generalized instead of copied-and-hacked.

MFC after: 5 days

Add a build knob MAKE_DVD to control on a per-architecture basis whether
or not to build a tree used for the creation of a DVD image. If that is
enabled set up a DVD tree by installing everything we normally install
to the individual CDROM trees into the one DVD tree. The result is one
image with all the install bits, livefs bits, and doc bits suitable for
burning to a DVD instead of CDROM.

Enable building the DVD for amd64 and i386.

MFC after: 1 week

When making release with NOPORTS, we'll checkout only the
mininal set of ports required to make the docs. However,
we also need ports/sysutils/cdrtools in order to make the
ISO images. When a platform doesn't have packages, the
release will fail in that case. Add ports/sysutils/cdrtools
to RELEASEPORTSMODULE for the DOMINIMALDOCPORTS case to
handle the NOPORTS release build.

Note that this change doesn't try to handle the NOPORTS with
NODOC case. For we have NOPORTSATALL set and it seems wrong
to check out a ports module in that case.

Enable GCC stack protection (aka Propolice) for userland:
- It is opt-out for now so as to give it maximum testing, but it may be
turned opt-in for stable branches depending on the consensus. You
can turn it off with WITHOUT_SSP.
- WITHOUT_SSP was previously used to disable the build of GNU libssp.
It is harmless to steal the knob as SSP symbols have been provided
by libc for a long time, GNU libssp should not have been much used.
- SSP is disabled in a few corners such as system bootstrap programs
(sys/boot), process bootstrap code (rtld, csu) and SSP symbols themselves.
- It should be safe to use -fstack-protector-all to build world, however
libc will be automatically downgraded to -fstack-protector because it
breaks rtld otherwise.
- This option is unavailable on ia64.

Enable GCC stack protection (aka Propolice) for kernel:
- It is opt-out for now so as to give it maximum testing.
- Do not compile your kernel with -fstack-protector-all, it won't work.

Submitted by: Jeremie Le Hen <jeremie@le-hen.org>

Don't use disklabel for the MFS root. The DoFS.sh script creates
a bogus partition table and puts the file system on the whole
partition. geom_part doesn't expose the 'c' partition as it's an
artifact of the BSD label and not to be used. Secondly, gpart(8)
is the preferred tool for partitioning disks on ia64.

Make it easier to experiment with alternate compression.

Reviewed by: kensmith

When compiled with the combination of flags used to build crunched
binaries for the fixit floppy bin/ed/main.c causes a gcc warning
message about argc possibly being clobbered by longjmp or vfork.
We have threatened to ditch floppies for 8.0 but I don't want to
do quite that much rototilling yet so for now turn off -Werror while
building ed (and everything else) for the fixit floppy.

Thanks to jb for pointing out NO_WERROR.

Remove kernel and module symbol files from disc1 (and the livecd if it
is enabled). This saves around 80MB (for i386) on the ISO images.

Note that this is only from the install media, not the installed
system where the symbols are still installed, if part of the release.

Should anybody want the symbols which match disc1/livecd, they can
just be extracted from the kernel distribution files.

Reviewed by: kensmith

The i386 disc1 size wandered above what fits on 650Mb media which has been
our target previously. So i386 joins the others that have switched over
to a separate livefs iso image.

pccard.conf is gone.

To honor the birth of RELENG_7 bump HEAD to 8.0-CURRENT.

Approved by: re (implicit)

Remove one more reference to the installation notes, this time in the
floppies directory.

Pointed out by: kensmith
Approved by: re (blanket for installation notes removal)

Don't copy the installation guide to the release media (i.e.
floppies or ISO images). We retain the concept of MD
release documentation for now, although it's fairly unlikely
that we'll ever do this again.

Approved by: re (blanket for installation guide removal)

The amd64 monthly snapshots began overflowing disc1 (and that's without any
packages...).

Reflect that the hardware notes is now an MI document.

Boot floppy for i386 overflowed during Jan. 2007 snapshot build, fix
it before Feb. 2007 snapshot build...

Teach release builds and release documentation snapshots that release
notes are no longer MD documents.

Move the documentation to its own separate disc to make room for more
packages on disc2. This will also let users decide if they want to
have a CD of the docs at all - unless they're disconnected from the
net they will probably find the Web site more useful.

Reviewed by: ru
MFC after: 3 days

Enforce RELEASENOUPDATE if EXTSRCDIR is used.

Suggested by: jb

The AUTO_KEYBOARD_DETECT is useful on amd64 as well.

Remove hint.atkbd.0.flags="0x1" from the installation device.hints;
kbdmux(4) is available during install and should take care of USB
keyboards.

Make it possible again to build -CURRENT snapshots on -STABLE.

Reported by: Angelo Turetta

Stop cutting the tarball names to eight symbols. This makes
it possible to e.g. distribute kernels with config names larger
than eight symbols, without the clash. Previousy, LALALALA
and LALALALA-SMP would be the same tarball. (I think this
comes from the old days where tarballs were put on the MS-DOS
formatted diskettes.)

MFC after: 3 days

Remove alpha left-overs.

When deleting empty directories prior to building the release tarballs,
pass the "-empty" flag to find(1). This change has no effect other
than to get rid of a few pages of "rmdir: ...: Directory not empty"
error messages. (Note that the "-empty" flag has been supported by
find(1) since 4.3-RELEASE.)

Make a bootable CD on pc98.

Fix the acpi.ko path for boot.flp.

PR: misc/96466
Submitted by: Nobuyuki Koganemaru
MFC after: 1 day

Assert that /dev/null is a character device in a chroot early. Otherwise,
we might get surprises later, like /dev/null having error in the 4th line
reported by make(1). :-)

Tested by: Dmitriy Kirhlarov (who attempted to make release in a jail)

make sure install scripts are executable

Submitted by: Ceri Davies

Convert NO_PROFILE and NO_LIB32 to new style.

Reimplementation of world/kernel build options. For details, see:

http://lists.freebsd.org/pipermail/freebsd-current/2006-March/061725.html

The src.conf(5) manpage is to follow in a few days.

Brought to you by: imp, jhb, kris, phk, ru (all bugs are mine)

Remove NO_SENDMAIL leaked from the days when building non-crypto
stuff for release was evil.

fix handling of default kernels

Reviewed by: ru

Fix this to what I've actually submitted.

handle KERNELS as before

Submitted by: ru, andrew

Revamp base system packaging of kernels to enable up/smp selection
at runtime and to support distributing additional kernels:
o remove kernel from the base tarball
o add new kernel tarballs
o build + package both SMP and GENERIC kernels when an <arch>/conf/SMP
config file is present
o add sysinstall support for multiple kernels
o update sysinstall to probe for the number of cpus on a system
and auto-select smp/up kernel accordingly
o add a post-kernels install hook to fixup /boot/kernel
o add -ldevinfo to boot crunch for sysinstall's cpu probing logic

Notes:
1. On HEAD this code is not currently used because GENERIC kernels
include SMP. This work is mainly intended for RELENG_6 where the
GENERIC kernel is UP. If HEAD changes to match then just enable
WITH_SMP in sysinstall/Makefile.
2. The cpu probing support is done with acpi and MPTable; this means
some systems will require work for auto-detection to work.
3. The handling of /boot/kernel may need to be revisited; for now
we rename one kernel at the last moment (SMP if installed, otherwise
GENERIC). There are other, possibly better, approaches.

Lots of help from ru, emaste, scottl, and jhb.

Fix a bug which puts an incorrect checksum of CHECKSUM.MD5
into the CHECKSUM.MD5 file itself.

Approved by: re (scottl)
MFC after: 3 days

Enable splitting up the MFSROOT floppy on amd64, the contents now overflow
one floppy's worth of space.

Fix the missing CD_VOLUME=1 in the cdrom.inf file on disc1 for the
livecd != disc1 case (i.e. ia64). The line was appended to the
non-existing cdrom.inf file, which was created only later. Move the
line to after the file is created.

MFC after: 1 day

Enable boot floppies for amd64. I just copied the needed variables over
from i386.

Tested by: Markus Trippelsdorf markus at trippelsdorf dot de
Approved by: scottl
MFC after: 1 week

Add EXTPORTSDIR like EXTSRCDIR and EXTDOCDIR.

PR: misc/83287
Submitted by: Gabor Kovesdan
MFC after: 1 week

Request boot floppy after loading mfsroot floppy because beastie.4th exists
in boot floppy.

Submitted by: Nobuyuki Koganemaru < n-kogane at syd dot odn dot ne dot jp >
MFC after: 1 week

Remove one more reference to usbd.conf.

Be cross-arch friendly.

Refactor (some more) installation of kernel and module objects.

Try to make everyone happy: David (to have debug kernels installed
by default), Warner (to be able to override that), and myself (for
actually making it all work and to be consistent).

Now, if kernel was configured for debugging (through DEBUG=-g in
the kernel config file or "config -g"), doing "make install" will
install debug versions of kernel and module objects with their
canonical names,

kernel.debug -> /boot/kernel/kernel
if_fxp.ko.debug -> /boot/kernel/if_fxp.ko

Installing a kernel not configured for debugging, or debug kernel
with INSTALL_NODEBUG variable defined, will install non-debug
kernel and module objects.

Also, restore the install.debug and reinstall.debug targets that
are part of the existing API (they cause some additional gdb(1)
scripts to be installed).

Add SHA256 checksums to the release build.

Requested by: cperciva (so)
Reviewed/improved by: ru
MFC after: 2 days

Unbreak release builds. Now that NO_INFO also affects the installation
of texinfo and we need makeinfo to build world, we cannot have NO_INFO
anymore when we populate the chroot.

Adjust for HEAD being 7.0 now.

Approved by: re (implicit)

Adapt the mfsroot environment of sysinstall to the needs of the new dhclient.
This involves having passwd bits available so that seteuid("_dhcp") work,
and creating /var/empty so that chroot(_VAR_EMPTY) works. My gut feeling
is that the better solution is to make privsep and chroot optional in
dhclient, but this works well for now and is low-risk.

Approved by: re

Provide 32-bit runtime support on amd64 as a separate distribution, lib32.

Prodded by: obrien
Nodded by: peter
Approved by: re

Make the liveCD separate ala ia64 - the combined live/install is
too large.

Strip the initial world some more.

Allow specifying which INDEX file to use by setting PKG_INDEX. By default
the INDEX file is taken from the package source tree as defined by the
PKG_TREE variable. This change allows using the (possibly incomplete)
packages on pointyhat.

MFC after: 2 days

- Move the doc project docs over to disc2 to free up some room for future
bloat on disc1.
- Output a message letting the user know that we are generating MD5 sums
during the long pause after the last mkisofs invocation in the iso.1
target.

MFC after: 3 days

What should be the last major changes to the ISO layout:
- When a separate livefs ISO is used, change the name of the tree directory
to be R/cdrom/livefs and change the name of the iso to livefs.iso rather
than using disc2. Instead, disc2 is now always going to be the packages
disc regardless of the presence of a separate livefs which makes things
simpler.
- Build a very simple disc2 tree under R/cdrom that just includes the
cdrom.inf file that the packages need. Also, build a disc2.iso image
in the iso.1 target. Disc 2's volume label is "FreeBSD_Packages".
- Retire CD_EXTRA_BITS and replace it with CD_PACKAGE_TREE. CD_EXTRA_BITS
was specific to disc1, but CD_PACKAGE_TREE instead should point to a
directory that has disc1 and disc2 subdirectories that contain the extra
bits for each image.
- Rename the bootonly volume label to "FreeBSD_bootonly" and just hardcode
the disc1 and livefs volume names and iso suffixes.

The live CD itself no longer has /stand since a long time ago; however,
there are still a couple of places under src/release such like fixit.profile
assumed that system binary can be retrivied from /stand(ex: boot with the
live CD and run "disklabel -e" in the fixit CD shell).

Since /stand is still functional in fixit floppy, and there are more than
one places in src/release needs to be updated(document for example) if we
want to make use of something like /mnt2/rescue/vi. This commit try to
deal with aforementioned inconsistency with minimal effort by simply
create a symbolic link to /rescue.

Reviewed by: jhb, sam (mentor)

Add a new package splitting mechanism to replace print-cdrom-packages.sh.
The new system tries to be more automated so that there is less work for
the re's to do. It also no longer uses a /usr/ports tree as its input,
but uses the generated package build including its INDEX file as its input.
It parses the INDEX file, determines which packages should go on which ISO
images, and then builds full-fledged trees of packages that can be added
as an argument to mkisofs along with the tree built by 'make release' to
build a full CD image. The INDEX files in the populated trees are
generated with volume media number to make use of sysinstall's multiple
volume support so that the user is kindly prompted to insert the
appropriate disc for a package if it is not on the current disc. There is
still some more tweaking to be done here, but this part needs to be
committed. This stuff will all be used to prep the 5.4 release as well.

Tested by: kensmith, others on re@
Reviewed by: re

When building release documentation for release builds or for
release documentation snapshots, force URLS_ABSOLUTE. This
allows us to relocate HTML renderings anywhere without worrying
about breaking relative links to pages on the Web site. This can
happen when we use &base; as part of a URL.

I wonder if we should do this for all release documentation builds,
but I'm not quite sure what all the cases are.

Always use a separate disc for the livefs on ia64 since the merged image
doesn't fit on one disc.

Tested by: marcel
X-MFC after: 30 seconds

Rework how we layout the CD/DVD ISO images for releases. Note that this
is only the first stage and does not yet handle packages (which will move
to their own disc(s)).
- By default, include the live filesystem on disc1, so that disc1 is now
both an install disc and a fixit disc. The images can still be split
into two separate discs by specifying SEPARATE_LIVEFS=yes.
- Remove /usr/ports from the live filesystem disc as the merged images
do not fit on even i386 otherwise. Also, the merged image already
contains a tarball of the entire ports tree, so /usr/ports was a
duplicate anyway.
- Stop building port READMES for the release sinc they are no longer used
(/usr/ports in the live filesystem was the last remaining user).
- Split cdrom.1 into three targets. cdrom.1 builds the live filesystem
image. cdrom.2 builds the disc1 image. Note that if SEPARATE_LIVEFS
is not specified these both write to the same area. cdrom.3 builds
the bootonly image and the UFS miniroot.
- The various distribution directories are now in a <buildname>
subdirectory of disc1 rather than being in the root so that the merged
image's root is not quite as cluttered.
- The disc1 image is now called disc1.iso and we no longer build a
miniinst.iso image. If CD_EXTRA_BITS is defined, then the iso.1 target
will simply include those bits in the disc1 image that it builds.

MFC after: 1 day
Requested by: re (scottl, etc.)

Use the shiny, new top-level 'distribution' target when populating the
chroot area. This fixes make release of HEAD on systems that have a
/usr/src with bsd.endian.mk and a matching /usr/obj but haven't installed
the world in /usr/obj and thus have no bsd.endian.mk /usr/share/mk.

Safer version of the chflags command.

NOCLEAN -> NO_CLEAN
NOCLEANDIR -> NO_CLEANDIR

NOHTML -> NO_HTML

NOGAMES -> NO_GAMES

NOSHARED -> NO_SHARED

NODOCCOMPRESS -> NO_DOCCOMPRESS
NOINFO -> NO_INFO
NOINFOCOMPRESS -> NO_INFOCOMPRESS
NOLINT -> NO_LINT
NOPIC -> NO_PIC
NOPROFILE -> NO_PROFILE

Start the dreaded NOFOO -> NO_FOO conversion.

OK'ed by: core

Actually put hint.atkbd.0.flags="0x1" in /boot/device.hints on CD-ROMs.
(Brought to my attention by Brooks while sitting together on the floor
in "Kairo".)

Pass WORLD_FLAGS down to the distributeworld target, so it's
possible to build releases with customizations such as with
NOPROFILE.

Problem found and fix tested by Roman Neuhauser.

It turns out the ia64 architecture's disc2 overflows if we put the
ports/ tree on it. This makes putting the ports/ tree on disc2 at
all dependent on the NOPORTREADMES knob as well as the NOPORT knob
(at the moment NOPORT may be set while NOPORTREADMES isn't, that
should probably be revisited). And it hardcodes ia64 for NOPORTREADMES
by request the ia64 release builder.

While here really get rid of the temp file 'make index' leaves behind.

Tested by: i386 and ia64 'make release'
Reviewed by: marcel
MFC after: 1 day

Fix-ups for a few problems reported with how ports are handled in the
release:

- run 'make index' as part of release build, INDEX* files in
CVS no longer updated and likely to be removed from CVS soon
- don't include README.html files in ports.tgz tarball because
they cause cvsup problems for users later
- put a copy of /usr/ports on disc2 that does include README.html
files so users have access to them

Reviewed by: ru (found one bug, several clean-up suggestions)
Tested by: 'make release'
MFC before: BETA6 build (this isn't likely to be tested in -current
anyway...)

Don't attempt to install the (obsolete and disconnected) Early Adopters
Guide on release media.

The early-adopter doesn't exist in HEAD anymore. The source is there,
but it's disconnected from the build. Remove it from the architecture
independent release documentation set, because it breaks "make release"
when NODOC is undefined.

Reviewed by: hrs

Use "cp -f" instead of "rm" and "cp" to break the hardlink to
device.hints.

Submitted by: ru

Add 'hint.atkbd.0.flags="0x1"' back to /boot/device.hints on the
install media on i386 and amd64. While the current default is very
useful in server environments, the ability to use USB keyboards is
vastly more important then the ability to use PS/2 keyboards the admin
forgot to plug in during the install process.

PR: kern/71443

Correct a typo in a comment.

Submitted by: Xin LI

Document the following knobs: CD_EXTRA_BITS, EXTRA_SRC, HTTP_PROXY
and FTP_PROXY. Removed bitrot (BOOT_CONFIG).

PR: misc/71465
Submitted by: Jon Passki <cykyc@yahoo.com>

Start the de-orbital burn of our previous FreeBSD version compat libs
living in usr/src. We need to use them from ports to record dependencies.

Discussed with: re(scottl)

Fixed the shell arithmetic to work with 4.x /bin/sh.

Tested by: make release of HEAD on 4.10-STABLE

Meet the new era of 6.0-CURRENT.

PPC definitions required for 'make release'

Reviewed by: ru
Submitted by: ssouhlal

Bring back 'NO_FLOPPIES', but as a user external knob.
The internal logic knob remains MAKE_FLOPPIES.

Suggested by: ru

Follow 'MAKE_ISOS' in naming and s/FLOPPIES/MAKE_FLOPPIES/g

Submitted by: ru

Accept the fact that floppies either aren't applicable or relevant on most
platforms. So make the floppy logic easier to read.

Now that make(1) passes variables that were given on the command line
to subordinate make(1) invocations through MAKEFLAGS, we cannot add
CFLAGS onto the make(1) command line. This will conflict with the
individual makefiles wanting to append to it, which is not respected
when CFLAGS is given on the command line. Hence build breakage.
So, put CFLAGS in the environment instead.

Clean up after mechanical renumbering.

Crypto is now part of the "base" distribution, and the old "release.3"
target no longer exists. Renumber the old release.[4-9] targets to
release.[3-8].

Reminded by: ru

Join the 21st century: Cryptography is no longer an optional component
of releases. The -DNOCRYPT build option still exists for anyone who
really wants to build non-cryptographic binaries, but the "crypto"
release distribution is now part of "base", and anyone installing from a
release will get cryptographic binaries.

Approved by: re (scottl), markm
Discussed on: freebsd-current, in late April 2004

Remove some serious foot-shooting potential from the release Makefile. For
some unknown reason, when LOCAL_PATCHES and LOCAL_SCRIPT were originally
added, they were silently ignored if the actual file did not exist. As a
result, if one mistyped the pathname to a patch or script, then the release
silently succeeded. However, it was not built with the desired changes and
no warning was given to inform the builder either. This commit explicitly
checks to see that all of the defined patches and scripts exist up front
and bails if any of them do not exist. I lost several hours of valuable
sleeping time this evening due to this "feature" so I've finally gone and
ripped out. I've tripped over this in the past several other times as
well.

Glanced at by: scottl

Die if make buildworld fails -- don't wait 'till installworld can't find
what it needs.

Add URLS_ABSOLUTE=YES into the doc.1 target.

Sorry for doing this backwards, but as with most release related stuff
we only pay attention to it during release time and at the moment it's
a -stable release in progress.

This is a MFS I guess, moving rev 1.536.2.113 to HEAD. I ran out of
time to follow through with doing this last night. Sorry for the delay.

Adding a couple of variables to make tweaking the CD-ROM label and .iso
filenames easier.

- Only build a fixit floppy on i386 and pc98.
- Reenable boot floppies for Alpha.

- Use a stock loader with forth support on the boot floppies.
- Use the regular forth scripts including the stock device.hints file on
the boot floppies.
- gzip all the various text files on the boot floppy that are larger than
one sector to save on space.
- Generate a loader.conf to load the kernel, acpi.ko, and mfsroot making
use of the 'module_before' variables to prompt for a floppy to be entered
when required and use a stock loader.rc. This enables the beastie menu
on i386.
- PC98 has been changed to split the mfsroot for now so that the floppies
fit. When the PC98 bootstrap is updated to work with ELF files (which
needs to happen anyway), then SPLIT_MFSROOT can be turned off for PC98.

- Tweak order of arch fs options to stick all floppy options together and
all MFS root options together and in the same order on all archs.
- Drop support for using the no longer existing 2.88 floppy image for CD
booting on i386 via the EMUL_BOOT variable.
- Use a stock loader on the boot floppy rather than one without forth.
- Force the gzip of the mfsroot in release.9 to succeed. This is useful
when doing lots of rereleases when testing floppy changes.
- If an acpi kernel module exists, stick it on the boot floppy and for
now always load it after the kernel.

Two nits from rev. 1.835:

- Fixed spelling of NO_FLOPPIES to unbreak snapshot building on
architectures that don't support them (as installation source).

- Don't rely on CVS setting execution bits on the shell script.
(This is also consistent with the rest of this makefile.)

Reviewed by: jhb

The name of small boot floppy should be boot-small.flp to avoid conflicting
normal size floppy.

Reviewed by: jhb

Revert revision 1.833 -- the reinstallkernel.debug target calls the
installkernel.debug target (in the kernel build directory) which is
only defined if the kernel was configured for debugging which it is
not the case for GENERIC kernels on release branches.

Only i386 and Alpha have the compat4x distribution.

Overhaul our boot floppy generation system so that it scales better and
requires minimal care and feeding for future releases.
- Consolidate multitude of floppy related constants down to a normal
FLOPPY set for 1.44 floppies and on PC98 a SMALLFLOPPY set for 1.2
floppies. Also, cleanup the i386 arch section by not duplicating
constants that are the same on both machine types (i386 and pc98).
- Update the ZIPNSPLIT macro to generate a file chunks that will actually
fit onto 1.44 floppies formatted with UFS1. Unfortunately, split(1)
seems to be somewhat buggy, so the files generated are slightly larger
than the argument passed to split.
- Split the release.10 target into 3 targers: floppies.1, floppies.2 and
floppies.3 that are added to EXTRAS only if NOFLOPPIES is defined.
floppies.1 builds the install floppies, floppies.2 builds the fixit
floppy, and floppies.3 generates the md5 sums and READMEs for the
floppies/ directory.
- Drop the by now largely obsolete and less useful boot.flp picture. This
was more useful when the mfsroot lived inside the kernel rather than
being loaded from a separate file by the loader.
- Only build a single mfsroot containing no modules that is used for all
installation methods.
- Use split-file.sh to split up a gzipped GENERIC kernel into however many
floppies it takes for the boot kernel. Currently, a stock 5.2 GENERIC
kernel including WITNESS, INVARIANTS, DDB, and other assorted bloat fits
onto 2 additional floppies besides the boot floppy with some room to
spare.
- If SPLIT_MFSROOT is defined, the mfsroot.gz file is similar split into
however many floppies are needed. Currently it is not defined as the
mfsroot.gz fits onto the current boot.flp with room to spare.
- Add a 'makeFloppySet' target which builds a floppy set for a file that
was split using split-file.sh.
- Rename the doMFSKERN target to 'buildBootFloppy' as that more closely
matches what it does now. We no longer build a custom BOOTMFS kernel for
each boot floppy.
- We no longer build a 2.88 boot.flp image to use with emulated CD booting.
The non-emulated cdboot works for almost everyone who boots off of CD and
if it doesn't work on a particular machine, the user can always boot from
the 1.44 floppy images.
- We no longer build a driver floppy or stick kernel modules in the mfsroot
since we now use a stock kernel when booting from floppy.

- Run makewhatis(1) in etc/Makefile at the end of "install".
- Removed redundant and undocumented NO_MAKEDB_RUN knob.

Use the reinstallkernel.debug target, now that we have it.

Fix "5.x release on 4.x box" support.

It is required to check /etc/rc.d/ldconfig before use, because
${CHROOT} sandbox is sometimes 4.x world. If /etc/rc.d/ldconfig
is not there, run ldconfig(8) directly.

Discussed with: imp, obrien, ru

Once upon a time we had both "crypto" and "krb5" distributions,
and rebuilt some bits with crypto but without Kerberos support
(most notably SSH) during "make release", to put them into the
"crypto" distribution.

Now that we don't ship the separate "krb5" distribution anymore
(it's now part of the "crypto" distribuion), don't waste time
recompiling SSH bits without crypto and without Kerberos support
in an attempt to put them in the "base" distribution -- it just
doesn't work as SSH always uses crypto code.

We avoid this by not rebuilding KPROGS from kerberos5/Makefile in
release/Makefile and adding "libpam" to SPROGS in secure/Makefile
to ensure it's still rebuilt without crypto support for the "base"
distribution. (Disabling crypto (NOCRYPT) also disables building
of Kerberos-related PAM modules, and it's OK to depend on this.)

This should be a no-op change saving some "make release" time.

MFRELENG_5_2: Default the snapshot versioning to 5.2.

PR: misc/61402
Submitted by: Chris Knight

Add missing '\' in previous change.

* if-like-constructs now use <TAB>body consistently.
* Long statements with '\' continuation now use <4 spaces>continuation
consistently.
* wraps some long lines
* More consistent on how very simular repeated commands are wrapped.

Use /etc/rc.d/ldconfig script rather than the ldconfig command directly.

Submitted by: matusita

Run ldconfig in the chroot environment to find shared libraries in
/usr/local/lib.

Automatically generate the CHECKSUM.MD5 file for ISO's.

Approved by: scottl

Generate minimal /etc/group file for mfsroot tree.

This fixes newfs failures at installation.

Reviewed by: re (scottl)

Fix ${LOCAL_PATCHES} (chdir was ignored).

Combining "all" and "install" in one invocation is a very bad idea as
conditional evaluation is done only once in this case, and this may
not account for the fact that some conditionals would be evaluated
differently if "all" has already been run, like is the case with the
loader.help existance check in sys/boot/alpha/common/Makefile.common,
beforeinstall target. This should fix Alpha snapshot builds.

Correct the description of what the release.3 stage does nowadays.

Make LOCAL_PATCHES accept multiple patch files, as its name suggests.

For boot floppies, build a lightweight version of loader(8)
without Forth support. Saves 28K of kern.flp and boot.flp
on i386.

Reviewed by: jhb

2nd part of making the name of the release running script overrideable.

Make the name of the release running script overrideable.

Added the new knob MINIROOT that, when set, causes the creation
of the bootable UFS image, miniroot.ufs, fully identical to the
contents of the "bootonly" CD-ROM. The image is made available
on FTP as miniroot/miniroot.ufs.gz, for resemblance with NetBSD.

This replaces the boot.flp image for sparc64, making the latter
as coming with NO_FLOPPIES.

Reviewed by: jhb
Tested by: hrs

- Eliminate the "rm" command that was causing us trouble in rev. 1.799.

- Normalize doRELEASE to avoid seeing "`ftp.1' is up to date." warning
when doing "make rerelease", and ftp.1 was already built.

Reviewed by: jhb

- Unbreak Alpha snapshots (we were still attempting to build a
no longer existing "fixit" crunched binary).

- Simplify the CRUNCH_TARGETS variable's initialization.

- Simplify the release.5 logic a bit by accounting for the fact
that release/*_crunch.conf files were removed two years ago.

Back out LC_ALL=C since tr here supposed to run under clean env -i
environment always.

Pointed by: ru

LANG->LC_ALL

Pointed by: ru

Prepend LANG=C to all tr 'a-z' 'A-Z' and vice versa.
Note: this is POSIX requirement, c-c ranges are portable only in POSIX locale

- Removed vestiges of the IA64 boot floppy support.
- Fixed the bootable CD-ROM support for IA64.

Reviewed by: marcel
Tested by: marcel

- Boot-only CD-ROM support (bootonly.iso).

Submitted by: jhb, ru

- Moved the creation of the ports distribution to release.7.
- Call MFS root floppy mfsroot.flp even on a small PC98 disk.

Submitted by: jhb

Reviewed by: ru, jhb

From now on, Alpha releases will come with NO_FLOPPIES.

Approved by: consensus on freebsd-alpha

A replacement (miniboot.iso) is currently under development.

Revert last delta, sys/conf/kern.pre.mk,v 1.33 does the job.

OK'ed by: scottl

The -Os kernel build with GCC 3.3.1 also requires -fno-strict-aliasing
to survive.

Discussed with: bde
Tested on: i386 and pc98

Initialize the FIXCRYPTO, BINMAKE and related variables (that use
the != operator) only when needed.

This change allows me to check out the current version of release/
makefiles only (co -l) to /tmp/release, and use that directory to
build a release (supplying the correct WORLDDIR).

Without this, attempt to "make release" caused an endless fork bomb
while trying to evaluate FIXCRYPTO, and the only way I could get
away from this on a remote box was to "kill -INT 1", thanks to
tcsh(1) and its internal "kill" command.

Embed a simplistic version of the dokern.sh script directly into
release/Makefile.

Reviewed by: marcel

The original patch also removed ia64/dokern.sh, but this was OBE.

Moved the MFS root floppies creation code from release.10 to
doMFSKERN (where it logically belongs).

Make IA64 a second port that comes with NO_FLOPPIES.

Reviewed by: marcel

Bah, I go and get all preachy, and then commit from the wrong branch. Revert
the use of '-j8'.

Apologies to: ru

Fix the release build with a forgotten '-f'.

Make the PC98 small boot floppy usable again.

Spotted by: nyan
Reviewed by: nyan

The RELEASE9 knob is now OBE.

OK'ed by: jhb

Moved the boot floppy generation code out from release.9 to release.10.
The release.9 target is now responsible only for generation of MFS root
file systems, that are built for all architectures, even those that do
not provide a floppy installation option. The release.10 target is now
responsible for creation of a set of boot, MFS root, and fixit floppies,
and the NO_FLOPPIES variable now affects only this target.

Also, replaced the FIXIT_TARGET variable with a check of whether the
*FIXITSIZE variables are present for a given architecture, similar to
how this is done when generating boot floppies.

Discussed with: jhb

Get the correct set of cryptographically-infected binaries
automagically from the (secure|kerberos5)/Makefile:[KS]PROG.

Replaced the hardcoded target names in the touch(1) commands with
${.TARGET}, to ease maintenance.

Added the (undocumented) EMUL_BOOT variable (for TARGET=i386 only)
that causes the bootable ISO images to be created using the floppy
emulation (the old method) as opposed to the new "cdboot" method.

Only copy boot.flp to the 2nd CD-ROM if this variable is defined.

Reviewed by: murray

Added the WORLDDIR variable (defaulting to ${.CURDIR}/..) that
points to a directory where "make buildworld" was run. Useful
for building 5.x snapshots on 4.x.

Make NO_FLOPPIES really mean "no floppies", including not even
creating the "floppies" directory on FTP and CD-ROMs. Enforce
NO_FLOPPIES on amd64.

Tested for: amd64, i386 with -DNO_FLOPPIES

Revert revision 1.788, and explain a bit the intent of having
the boot.flp image on the second CD-ROM.

Explained by: jhb
Desired by: vendors

Do not define BIGBOOTSIZE and the friends for amd64; it serves
no useful purpose other than wasting CPU time in "make release"
creating useless boot.flp.

Desired by: peter

/floppies/boot.flp is no longer necessary on bootable i386 CD-ROMs.

Forgotten by: jhb

Fixed the last-minute bug I made in previous commit.

Fixed style bugs related to parentheses in Makefile.inc1.

Submitted by: bde

Fixed nearby bug: propagate the root Makefile's idea of
the appropriate "make" binary down to release/Makefile.

MFRELENG_5_1: Default the release makefile to 5.1.

Be laconic.

Add amd64 support

TARGET_ARCH == amd64 also has /boot/mbr.

Have the port readmes follow the same logic as buildworld. We skip
doing make readmes if /tmp/.skip_ports exists. This file is created
after a complete run, which avoids that we do the lengthy operation
again for a rerelease.
When NOPORTS or NOPORTREADMES are defined, we create the file prior
to starting the release.

Add some missing items to the make clean target.

Remove alpha-specific floppy build target for now.

Approved by: re(jhb)

Move the isp driver back to the alpha BOOTMFS kernel, and trim the size
of inodes for the alpha driver floppy. With this patch, alpha actually
built a full release!

Submitted by: ru

For EXT*DIR, use -H in addition to -R, so that a symlink may be
used in the EXTSRCDIR case, for example, if one finds it useful
to symlink /usr/src to an actual source tree, and use /usr/src
as the EXTSRCDIR.

This has no effects on release builds not using external directories.

Reviewed by: ru
Approved by: scottl (re)

disklabel(8) is spelled bsdlabel(8) or sunlabel(8) these days.
Also make sure bsdlabel(8) (along with the disklabel(8) compat
link) still appear on the fixit floppies of platforms that use
it natively (alpha, i386, and pc98).

Approved by: re (scottl)

release.9 has become the biggest festering pile of spaghetti s(*^% code.
Duplicate the target for Alpha so I can start hacking and slashing bits out.

Approved by: re(blanket to make Alpha build)

Turn MAKE_KERBEROS5 into NO_KERBEROS by negating the logic. Some extra
cleanups were necessary in release/Makefile, and the tinderbox code
was syntax checked, not run checked.

Move the setting of the BOOTABLE flag outside of .if MAKE_ISOS so that
bootable CDs can still be created if the iso.1 target is called
directly.

PR: misc/47191
Submitted by: Joost Bekkers <joost@bps.jodocus.org>
MFC after: 1 week

KerberosIV removal. Also crunch down the "krb5" distribution into
"secure".

Removing comment in some text files for fixit floppy overflow.

Approved by: re (murray)

Presumably fix sparc64 "make release". (The breakage was due to
sunlabel(8) no longer being linked to disklabel(8).)

Reviewed by: phk

continuation not needed

Don't play with MAKEFLAGS in one other place too; it's dangerous.

Makefile.inc1 rev. 1.341 broke release.5 -- the fuss with MAKEFLAGS
here interferenced with one in Makefile.inc1. Since we can't avoid
the jugglery in Makefile.inc1 (we want -m /usr/src/tools/build/mk
to come first, before any other -m specified on the command line),
avoid it here.

Reported by: imp

Revert revision 1.639 -- the "nodev" mount(8) option restriction
does not apply to ${CHROOTDIR} file system since revision 1.712.

Strip the .comment section out from the BOOTMFS kernel.

s/loaded as a module/available as a module/g

Submitted by: ru

Use the target arhitecture's disktab(5) file.

Pass the target machine architecture down to disklabel(8).
Do it so that systems without the -m option in disklabel(8),
e.g. 5.0-RELEASE, can still build current snapshots.

While here, drop the -s option from doFS.sh; we have not
been using the .size files for a long time.

In rev 1.603 I should have changed it to mdconfig rather than remove it.
I don't recall now, but for some reason I didn't think it was needed.
Also place the mdconfig right before the chroot call, to make it stand out
this needs to be done in the standard environment, as we've tried doing it
in the chroot environment before.

Unbreak iso.1.

Use positive logic rather than harder-to-read negative logic.

Make PREFETCHDISTFILES the default. Set NO_PREFETCHDISTFILES if you don't
like this. Also make it easier to issue the distfile fetching on command.

Add PORTREADMES_FLAGS which can really speed up the build on SMP machines
when set to -j<3*#CPU's>

KerberosIV de-orbit burn continues. Disconnect from "make release".

o Run make readmes (ports) from within CHROOTDIR to avoid pollution
and/or breakages due to /etc/make.conf on the host. Move it after
make world and put it in an if-block so that we can include the
commands in the script even if we're not going to make the readmes.
o When building docproj, make sure we propagate the setting of the
FTP_PASSIVE_MODE environment variable. Not propagating the envvar
may cause fetch failures. This increases the number of cases for
which one does not have to prefetch the packages.

If we are told to update the sources in "make rerelease", we also
probably want to re-buildworld.

If we are not going to rebuild (make rerelease -DRELEASENOUPDATE
and the file ${CHROOTDIR}/tmp/.world_done exists), do not touch
${CHROOTDIR}/usr/src/sys/conf/newvers.sh, as the only effect that
is has is upsetting release.2. Hey, have you ever seen this
"touch: not found" during rerelease? :-)

At least we don't need the whole <bsd.prog.mk> glue.

(I did test build the original change, but apparently
forgot to include it to LOCAL_PATCHES. It happens.)

mk file needed for 'make obj' done by $CHROOTDIR/mk

Fixed synopsis.
Mention that md(4) should be active (from release(7)).
Yet, this has nothing to do with bsd.prog.mk.

Pass FTP_PASSIVE_MODE, FTP_PROXY, and HTTP_PROXY variables to chroot sandbox.
These variables will be used by pkg_add, which installs mkisofs package
while MAKE_ISOS=YES. See src/release/${arch}/mkisoimages.sh if you doubt.

No objection heard from: current@FreeBSD.org
Suggested by: des, nectar (about *_PROXY variables)

Run kgzip(8) in a right environment and pass it the correct loader object.

Part 2/3 of unbreaking cross releases:

Use the right strip(1) binary.

Use checksum-recursive rather than fetch-recursive-list in PREFETCHDISTFILES
so as to catch errors ASAP rather than later between the ports cvs checkout
and docs cvs checkout. If a distfile doesn't checksum OK, it is very
hard to restart the release with "make rerelease" as that target assumes
all the cvs checkout's happened OK, and if that is not the case the cvs
update that is attempted fails..

Fix 'make release' from erroring out if this is an arch without driver
floppies.

Submitted by: nyan
Reviewed by: jhb

If there is a driver floppy for this arch, then add a line to the loader.rc
on the boot floppies to set the 'driver_floppy' variable in the loader so
that it ends up in the kernel environment and sysinstall will prompt for the
diskette. Booting from CD-ROM does not use this loader.rc file, so folks
booting from CD to do an install will not be prompted.

X-MFC after: as soon as I finish committing to current
Approved by: re (blanket)

Make iso.1 more readable.

Backout phk's backout of obrien's change.

Hat: core

Reduce size of individual bits of the dists to 240640 bytes again
so snapshots can be installed.

I have no idea what breaks, but at least the passive FTP install
goes haywire.

Setup a symlink to the distribution name in the root of the first CD image
so that it can be used as a anon-ftp area.

Change 'mfsroot' filename in mfsroot-small floppies from 'mfsroot-small'
to 'mfsroot'.

This is MFC candidate.

Noticed by: Toru Morimoto <too@os.gulf.or.jp>

MFS: Split export list.

Split files into 1.44MB chunks rather than a tiny, high-ftp overhead size.

Approved by: rwatson

Strip out any .comment ELF sections from crunchgen'ed binaries.

Move the creation of the DRIVERS.TXT file to a more appropriate area
of this file, so it will be created for other platforms that use a
third floppy disk, not just i386.

Approved by: re

Add ${BUILDNAME} and ${TARGET} to the filenames of ISO images
generated during release builds.

For the most part, re@ was doing this manually before putting ISO images
on ftp-master, but not always. This should make naming more consistent
and descriptive.

Requested by: jason andrade <jason@dstc.edu.au>, hubs@
Approved by: re (rwatson)
MFC after: 1 month

Only create the DRIVERS.TXT file on i386 for now.

Approved by: re

Create DRIVERS.TXT file to describe the contents of drivers.flp.

Approved by: re

Add lib/libfetch and usr.bin/fetch to FIXCRYPTO since they need the crypto
libraries for SSL support.

Approved by: re (rwatson)

Export NO_CPU_CFLAGS and NO_CPU_COPTFLAGS to chroot.

Copy from ${EXTLOCALDIR} to ${CHROOTDIR}/usr/local if EXTLOCALDIR is defined.

Rework the checking out of the doc, src, and ports trees a bit to make it
more manageable.
- Add some helper variables (CVS_{SRC,DOC,PORTS}ARGS) to be used when
using CVS to checkout files. We stick release tags in these helper
variables if they are defined and then use only one cvs command instead
of two cvs commands with an .ifdef to choose between them.
- rm the old src/doc/ports directories as separate commands from the
CVS comands so that the rm commands don't need to be duplicated.
- Simplify the DOMINIMALDOCPORTS case by overriding RELEASEPORTSMODULE to
be ${MINIMALDOCPORTS} thus removing yet another nearly-duplicate cvs
command in an .ifdef.
- Add support for grabbing src/ and doc/ from external directories
specified via EXTSRCDIR and EXTDOCDIR instead of from CVS. The same
is not done for ports/ quite yet as the DOMINIMALDOCPORTS case is a
bit tricky.

The rerelease target scripts have not been changed to use the helper
variables yet, so there is still some room for improvement.

Submitted by: kuriyama (4)

Allow a person to specify NOCDROM to skip the cdrom.1 target during a
release build.

PR: 45300
Submitted by: David Yeske <dyeske@yahoo.com>

release.9 for ia64 cannot be currently cross-built -- gpt(8)
is built on ia64 only.

Don't ever disable ACPI on ia64. We cannot life without it.

o Introduce variable DOFS_SH to hold the path of the doFS.sh.
o ia64: Set DOFS_SH to point to an alternate version.
o ia64: Lower BIGBOOTSIZE from 120M (LS-120) to ~5M.
o ia64: Set BIGBOOTLABEL to efi so that we can check for it
in the alternate doFS.sh script. If the label is not efi, we
can invoke the generic doFS.sh.

Fixes to unbreak Alpha releases:

Strip ${RD}/mfsfd/stand/etc/defaults/rc.conf.

Don't copy loader.help to ${RD}/mfsfd/boot, it serves no useful purpose.

Removed the now no-op commands to copy ${RD}/trees/base/dev bits to
${RD}/mfsfd/dev, now that DEVFS is the default.

Removed UFS_ACL, PSEUDOFS, and COMPAT_FREEBSD4 options from the BOOTMFS
kernel.

Moved miibus network module to the drivers floppy.

isp(4) and mpt(4) now have corresponding modules, move them to the third
floppy as well.

Caution! With these changes, mfsroot.flp has only 11K left, and
kern.flp has even less -- 5K.

Make a small (1.2M) fixit floppy image for pc98.

Use ${TARGET_ARCH} for ia64 tests.

Move the modules in the mfsroot to /modules instead of /stand/modules so
that they aren't copied over into /stand of new installations.

Submitted by: matusita
Tested on: i386

Use ${RD}/trees/base/boot/loader for kern.flp instead of /boot/loader.

Add the necessary tweaks for FreeBSD/ia64 releases. Note that this is
in no way final. A typical ia64 wart is that there are no boot blocks.
Instead, we need to create disks with EFI partitions if we want auto
boot to work. All this functionality is not present yet.

Add CVSARGS for cvs arguments (vs. cvs subcommand arguments).
Allow the cvs command to be prefixed by using CVSPREFIX.

- Copy the mfsroot.gz file to /R/stage/mfsroot/ so that the CD-ROM layout
can fetch it from there instead of looking in the boot floppy's
directory.
- Add a NO_FLOPPIES variable that disables building of any boot floppies.

Tested on: alpha

Hook the early adopter's guide into the release build. Its basename
is "EARLY", which is kind of an odd name, but I couldn't think of
anything better that fit the traditional 8.3 naming convention for
release documentation files.

Remove unneeded variables.

Make floppy-making part of release.9 conditional on *BOOTSIZE
rather than on "pc98" (rev. 1.710) and "sparc64" (rev. 1.711).

Nuke pccard.conf from FreeBSD/i386 mfsroot (no longer needed).

Suggested by: nyan

Fixups to get make release going again for i386:

o mount (and unmount) devfs in the chroot'd filesystem
o handle umounting devfs in case of a keyboard interrupt
o remove MAKEDEV from the fixit floppy so things fit again (mount_devfs was
added in a separate commit)
o explicitly remove mfsroot.gz and loader.conf when building the cd-rom area
as otherwice cp -Rp aborts when a previous run left identical files around
o increase the number of inodes on the drivers floppy; moving drivers from
the kernel there (to shrink it's size) caused things to overflow
o while we're increasing the driver's floppy inode config, separate out all
parameters from the boot floppy

Approved by: re

Add support for building sparc64 releases.

Reviewed by: murray

Fixed disk full problem for pc98 distribution. Now, it makes 4 installation
floppies.

kern-small.flp and mfsroot-small.flp are for old machines that don't
support 1.44MB floppy. These floppies don't have pci related devices.

kern.flp and mfsroot.flp are normal 1.44MB floppy images.

Pass -o option to crunchgen(1) to ensure object files end up
in seperate directories. This mostly affects RELENG_4 where
not doing so results in ${CHROOTDIR}/usr/src/release being
filled with object files, and "srelease" distribution having
them in.

PR: misc/43825
Prodded by: re (murray)

Don't make an error even if drivers.conf does not exist.

Tested on: snapshots.pc98.jp.FreeBSD.org

ALLLANG is long dead, garbage collect it. Instead, make it possible
to use DOC_LANG to specify which languages the doc/ is built for.

Note: be aware that the DOC_LANG setting in /etc/make.conf will now
take the effect on "make release". (This is probably the desired
behavior anyway.)

PR: docs/42924

Kernel MIB name for disabling ACPI was changed.

Pointed out by: iwasaki

Initiate deorbit burn for the i386-only a.out related support. Moves are
under way to move the remnants of the a.out toolchain to ports. As the
comment in src/Makefile said, this stuff is deprecated and one should not
expect this to remain beyond 4.0-REL. It has already lasted WAY beyond
that.

Notable exceptions:
gcc - I have not touched the a.out generation stuff there.
ldd/ldconfig - still have some code to interface with a.out rtld.
old as/ld/etc - I have not removed these yet, pending their move to ports.
some includes - necessary for ldd/ldconfig for now.

Tested on: i386 (extensively), alpha

Use ${TARGET}/dokern.sh instead of ${TARGET_ARCH}/dokern.sh.
(pc98/dokern.sh is (repository) copied from i386/dokern.sh.)

Collapsed the export list, added CD_EXTRA_BITS to it.

Two more variables to export: KERNELS and AUTO_KEYBOARD_DETECT.

Submitted by: Alan Judge <Alan.Judge@eircom.net>

Remove a stray ';'.

Remove a '-' that is not needed.

Submitted by: ru

Add a third floppy to releases. It will only be built if there is a
release/${TARGET}/drivers.conf file which list drivers that have to
go into the third floppy.

Also shuffle i386/drivers.conf so that the floppies don't overflow
anymore. Anybody with real/better usage statistics is welcome to
shuffle it differently.

Reviewed by: ru

Stop eating all process memory if either of ${.CURDIR}/../kerberosIV
or ${.CURDIR}/../kerberos5 does not exist.

Spotted by: Alexandr Listopad <laa@reis.zp.ua>
MFC after: 3 days