Copy stable/10@r272459 to releng/10.1 as part of
the 10.1-RELEASE process.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC rev. 265926:
(NANO_CONFIG): New variable containing path of config file, so that
the configuration can reference additional files relative to its own
location.

(NANO_MODULES): If set to "default", install all built modules.

Reviewed by: imp

MFC rev. 265260:
Add appropriate quoting to allow building with a KERNCONFDIR containing
spaces.

PR: kern/162736

MFC: r258705,r258709,r263140,r263184,r263185,r263186,r263187,r263188,
r263189,r263190,r263192,r263193,r263287,r263288,r263890,r264281,
r264655,r264656,r264856,r264857,r264858,r264859,r264860,r264861,
r264891,r264892,r264909

Catch up on all the referenced changes in -current, in brief:

r258705 | n_hibma | Use NANO_PMAKE for build, but not install
r258709 | mr | pkgng goo + bootstrap
r263140 | glebius | Axe IPX.
r263184 | imp | Fix build with spaces in names.
r263185 | imp | Make pcengines config files compile again.
r263186 | imp | Use UFS2 by default, but allow fallback to UFS1
r263187 | imp | Print an error message when we exit out early.
r263188 | imp | Sometimes only copy MBR when backing via swap
r263189 | imp | Update save_cfg
r263190 | imp | Be more explicit about setting SRCCONF to /dev/null
r263192 | imp | Rely on default UFS type
r263193 | imp | Remove TARGET_BIG_ENDIAN. It's no longer relevant.
r263287 | bdrewery | pkg(8) has no limits
r263288 | bdrewery | Remove comment meant for removal in r263287
r263890 | imp | Up default media size to 1GB.
r264281 | imp | Ignore .hg and .git files
r264655 | imp | rm -x for safety
r264656 | imp | new example: dhcpd derived from FreeNAS/BSDRP
r264856 | imp | Override the parallel make target to use all CPUs.
r264857 | imp | No need for true here, remove it.
r264858 | imp | NANO_PKG_META_BASE
r264859 | imp | stylize umount hack
r264860 | imp | switch dhcpd to pkg(8).
r264861 | imp | CR and CR0 added from dhcpd
r264891 | imp | Try to alwaays use () for functions
r264892 | imp | bash whines about r264857, revert
r264909 | imp | Move back to {} for functions that need global effects

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Replace a silly command sequence with a proper if-then-else.
Generate images sparsely. This saves space and time, especially when
generating images inside a VM (PR 173482).
Add a 'true' statement to last_orders to prevent some version of sh from
tripping over an empty function.

Doing a cpio from /var/empty if dir was not specified or non-existent
copies its mode to the destination. This is not desirable.
Rephrase this code to be more sensible.

PR: 173483
MFC after: 1 week

Don't need (and can't use) -L to copy links here.

Restore BOOT2CFG, accidentally removed in r212938.

PR: 157020

If there's no package directory, don't try to install packages from
it. Instead, report that 0 packages are reported.

PR: misc/140436

Add ${} around variable dereference...

Submitted by: garrett coopper

Implement -f to inhibit copying s1 partition out for speed.
(-z not taken from PR, it is no longer needed since we basically always
do that now)

PR: misc/151698
Submitted by: lev@

Copy symbolic links as files rather than recreating the links.

PR: misc/151697
Submitted by: lev@
MFC after: 2 weeks

Use ${NANO_WORLDDIR}/var/empty as copy source since it has no schg flag set.
Copying over /var/empty's schg flag had unpleasant side effects (schg flag on /etc and /cfg)
during nanobsd boot before.

Bump the media size from approx 600MB to approx 750MB. The great
hob-nailed tennis shoe of progress demands it!

Submitted by: phk

Put in the other half of the SRCCONF patch.

Submitted by: phk

o Mount the device async when we're doing the copy.
o Create a sparse file instead of a fully zerod one. This trades the
possibiltiy of running out of space during the build for the speed
gain not having do write all those zeros...

Submitted by: n_hibma

_WITHOUT_SRCCONF has too much baggage. Instead, use the simpler
SRCCONF=/dev/null.

Insulate the nanobsd build from the current system by opting out of
the SRCCONF processing.

Noted by: gcopper@

Build make.conf when the world is not selected to build, but the
kernel is.

PR: 151696
Submitted by: lev@

Document NANO_CFGDIR and NANO_DATADIR

Submitted by: n_hibma@

- Set -x flag when executing customisation scripts to aid in debugging them.
- Use KERNCONFDIR with KERNCONF instead of copying the kernel config into the source tree
so included kernel configs work.
- Put more stuff in the _.bk/_.ik log file, not just make statements.
- Add the kernel config name to the pprint during kernel installation.
- Add NANO_MODULES providing a list of modules to build and install.

Reviewed by: imp
MFC after: 2 weeks

Make the labels match the device name that's mounted, not just the
slice they are on. When NANO_LABEL is not defined, the fstab
generates entries that specify /dev/ad0s1a. When NANO_LABEL is
defined, it generates /dev/usb/${NANO_LABEL}s1a. The prior code
created the file system with a label of ${NANO_LABEL}s1, leading to
problems on boot.

Pointy hat to: imp@

Support new variable NANO_LABEL. When set to a non-null string,
nanobsd will build a system that uses this label (via
/dev/ufs/${NANO_LABEL}sX) in preference to NANO_DRIVE (well, it forces
NANO_DRIVE to be ufs/${NANO_LABEL}). This allows images that will
boot off usb stick or CF card easily well.

There is no change if you don't set this variable.

Allow / in the NANO_DEVICE

PR: 149729
Submitted by: Thomas Quinot <thomas@cuivre.fr.eu.org>

Allow boot.config to have different values than just -h.

Submitted by: bsdrp by way of freenas and olivier cochard-labbe
MFC after: 3 days

Allow population of /cfg and /data. Begin the move to making all
slice creation overrideable too, but there's a few problems doing that
for the duplicated partitions (s1 and s2), so just comment that it
needs work.

MFC after: 3 days

Create a make.conf. not needed for runtime, but some ports want to spam it
at compile or install time.

MFC after: 3 days

make these convenience functions more convenient by accepting all
args, not just the first. makes mechanical conversion of old style
more forgiving.

MFC after: 3 days

Fix typo in usage message. Add -n to the list of valid arguments.

MFC after: 1 day

Fix comment about NANO_ARCH. It can be set now. Also add a warning
about how it confuses TARGET and TARGET_ARCH and the need to separate
the two out into independent variables.

Stop hard coding i386 as the arch for the build. Instead, default to the
processor we're running on. Also, supply amd64 version of create_diskimage
that's the same as i386's.

# didn't fix the confusion between using the processor for this and using
# the machine (which would be more appropriate). NANO_ARCH smashes the two
# together right now.

MFC after: 7 days

- When we run our trap cleanup handler, echo that we are running this
handler to make it more clear why we are 'suddenly' running df,
umount, and mdconfig.
- Remove trap handler again after we have unconfigured the memory
device etc. Before we could end up running the trap handler if a
later stage failed, which was a bit confusing and not really useful.

MFC after: 2 weeks

- Make pprint print through fd 3, so it can be used in customisation
functions to print something to the screen.
- Prefix each line with the running time (bikeshed).

Submitted by: Rick van der Zwet (Wireless Leiden)

- Don't hard code _.disk.full. Use the variable.
- _.bk should be created in MAKEOBJDIRPREFIX as well
- Remove a misplaced and unnecessary message.

Reverse some stuff I accidentally committed in the previous commit:

- creation of sparse files to speed up the build process. This was
discussed with phk 2 years ago and he disagreed with this change.
- handling of negative data partition sizes.

Can I have the ... green pointy hat, please?

Allow building world into a separate dir (for reuse in multiple images):
- buildworld and buildkernel are built into MAKEOBJDIRPREFIX
- installworld and installkernel are performed on NANO_OBJ.

No change of functionality if MAKEOBJDIRPREFIX is not set. If it is sea,t
clean_world deletes NANO_OBJ instead of NANO_WORLDDIR. By starting nanobsd.sh
with the -b option the existing world can be reused to build a new world
reducing time and disk space considerably.

While there:
- Fix two cases where (in comments) MAKEOBJDIRPREFIX should have been
NANO_DISKIMGDIR.
- Simplify an 'if (not wrong); then true; else action; fi' into
'if wrong; then action; fi'. 'if ! false; then echo hello; fi' produces hello.

Note: Make sure you use NANO_OBJ were you use MAKEOBJDIRPREFIX now in your
nanobsd.conf files if you want to split out.

Fix NanoBSD when the data partition size is defined as a negative
number.

It is possible to ask nanobsd.sh to create a 'data' partition,
separate from the system or configuration partitions, and
furthermore, by specifying a negative value for its size
to request that it use all space unused by those partitions
for its own size.

Because the two lines of code that calculate how much space
is available for this data partition are written in perl-like
syntax, the awk code that does the processing performs the
calculation incorrectly.
[note - this was already fixed by r174936]

Furthermore, a comparison later down fails to newfs the
partition when the size is negative.

PR: misc/127759
Submitted by: Cyrus Rahman <crahman@gmail.com>
MFC after: 1 week

Make it possible to cherry-pick packages out of a directory.

add -n option to suppress clearing the build tree and add -DNO_CLEAN
to buildworld and/or buildkernel

Avoid failing if the directory already exists (when restarting at customize).

MFC after: 2 days

In cust_install_files() we ignore CVS directories. In a similar way now
also ignore .svn directories.

- Change all "echo #..." into using a progress print function to make it
possible to make NanoBSD output more quite or verbose. The default
output should remain mostly unchanged. [1]
- Add missing shift for -i.
- Clean up usage() so it's now (mostly) sorted alphabetically.
- Make command line argument handling more consistent in the code and
remove redundant semicolons.

Reviwed by: phk [1]

* Add USB boot support.
* Allow the image name to be renamed via NANO_IMGNAME.
* Propagate TARGET_ARCH into src top level make targets
explicitly to support cross-building.
* Increase the default size of NanoBSD media from 488MB to
584MB to accomodate a -CURRENT world.

Reviewed by: phk

Mark the first slice of a NanoBSD image 'active' by default.
This fixes USB boot (not yet merged to HEAD) with 3 flavours
of BIOS I've seen.

Approved by: phk

Include $NANO_NAME in the completed message. This is nice if you are
building multiple NanoBSD images at once to keep track of what is
running and what isn't.

Change space -> tab in printed usage output to make it look consistent.

Fix typo in comment.

Accept tty[ud]0 for console device

fix syntax

Add late customize hook

Submitted by: Patrick M. Hausen
Sat on for far too long: phk

Add -i option to supress image building.

Submitted by: keramida

Track /etc/ttys change to uart(4)

unset TARGET_CPUTYPE and TARGET_BIG_ENDIAN when building the kernel
to silence compiler complaints when cross-building

Reviewed by: phk
MFC after: 2 weeks

- Add support for using swap backed md(4) devices for building the
disk image. In some cases this can be a significant speed-up, if
most of the image can be kept in RAM while being populated.

On the 2GB image I'm currently working with, the build time,
excluding buildworld/buildkernel, goes from ~17 minutes to ~6
minutes.

This is not enabled by default, as it might have the opposite effect
on low-memory systems.

- During the generation of the image file be a bit more verbose in the
log file so it is possible to see what's being done.

- Add a NANO_DISKIMGDIR variable which makes it possibly to place the
final images somewhere other than ${MAKEOBJDIRPREFIX}. The default
value for NANO_DISKIMGDIR is $MAKEOBJDIRPREFIX.

Go for it: phk

- Fix calculation of data slice size when NANO_DATASIZE=-1.

Due to a typo, setting NANO_DATASIZE=-1 resulted in the data slice
being the same size as entire image instead of the size of the
remaining space on the image.

- Fix detection of overcommit of the slices.

This fix mainly result in a nicer error than when newfs etc. tries to
write beyond the end of the disk image.

MFC after: 2 weeks
X-MFC after: RELENG_7 is open again

Add option to not fill second code image, this makes the full disk
image compress much better.

Respect pkg_add's internal 200 package limitation.

Approved by: re (bmah)
Spelling fixes by: bmah

Add cust_pkg() which iteratively tries to install packages.

Move /usr/local/etc to /etc/local so that installed packages can be frobbed.

Use a more compatible 512MB card geometry.

Default to R/O root filesystem

Fix the fstab on the second image, just like updatep2.sh does.

NanoBSD modifications:

* Break out the boot0 loader selection into a variable - NANO_BOOTLOADER -
so people like me with VGA consoles can override the default (which is
to use boot0sio)
* Put the boot0 configuration options in NANO_BOOT0CFG in case you want
to override the defaults.
* Modify nanobsd.8 to reflect the changes and hint the console default is
serial.

MFC after: 2 weeks

Shorten the KERNCONF filename to the basename in the install target as
well as in the build target, although it is not used in this target.

Only use the filename part of the kernel configuration file as an argument to
KERNCONF after the file has been copied to the sys/${ARCH}/conf directory. This
allows the use of one kernel config file for multiple images. E.g.:

NANO_KERNEL=../../../../software/nanobsd/default/SOEKRIS

MFC: after 6.2

Move the setup of "diskless" /etc magic files from install_etc() to a
separate function which is called right after install_etc(). This makes
it simpler to (ab)use nanobsd.sh as a framework to build more normal
FreeBSD images where a normal /etc is used, since setup_nanobsd_etc()
can just be redefined.

OK'ed in principle by: phk
MFC after: 1 week

'make distribution' must be run from src, not src/etc, so that the proper
set of /usr/share/mk/* is available. This is necessary to build a 7.x
NanoBSD image on a 6.x host.

Reviewed by: phk

When creating the NanoBSD disk image, install boot blocks from the
newly-built world, in preference to those from the host system.

Reviewed by: phk

Create the diskless magic files in /etc during the install_etc() step
instead of setup_nanobsd(), because this gives customize tasks a chance
to fiddle the details.

Inspired & tested by: Jordan Coleman <jordan@JordanColeman.com>

In addition to -b (suppress builds) add -k (suppress kernel build) and
-w (suppress world).

Add two new variables:

# Size of the /etc ramdisk in 512 bytes sectors
NANO_RAM_ETCSIZE=10240

# Size of the /tmp+/var ramdisk in 512 bytes sectors
NANO_RAM_TMPVARSIZE=10240

And a last_orders() shell function which can be redefined to copy
finished images away, order peanuts or whatever else is appropriate

Fix a bug in disk geometry calculation.

Fix NANO_TOOLS usage in FlashDevice()

Make it possible to set the NANO_TOOLS variable to a directory outside
of $NANO_SRC, which can e.g. be used if the nanobsd scripts are not in
the source tree being built.

Improved by: phk
Approved by: phk

Add a convenience function to set NANO_MEDIASIZE, NANO_HEADS and NANO_SECTS
for commonly used flash devices by

FlashDevice Sandisk 256MB

in the config file.

Create an /etc/nanobsd.conf in the built image and put the disk drive name
there for scripts to use.

Create a noauto fstab entry for the configuration partition (/cfg).

Add NANO_TOOLS env-var to point to the nanobsd sources relative to
NANO_SRC.

Add -h argument which prints a usage.

Add -b argument which skips build steps and goes directly to install steps.

Complain about extraneous arguments, it's usually a forgotten '-c'

Add convenience function to register customization function.

Add some sample customization functions:
cust_comconsole
cust_allow_ssh_root

Rename setup_diskless() to setup_nanobsd(), it makes more sense.

Add various comments etc.

Dump the environment for reference.
Add a function to add customization commands.

Create fstab before linking stuff into /conf

Missing ')'

Correct a comment to match the configuration parameter it describes.

MFC after: 3 days

Increase flexibility in disk-layout

Put more stuff in logfiles instead of stdout

Cleanup and polishing.

After much experimentation, settle for the simplest model for customize
scripts.

Prune empty directories in /usr

Move argv parsing.

polish various bits.

Add the first snapshot at rewriting the nanobsd build to be a shellscript.

The overall idea is to get a single build script which will not need
editing, but instead overrides and changes happen by config files passed
as arguments.