makefs: Make it possible to silence warnings about duplicate paths

When generating a VM image from an installworld mtree manifest, makefs
spits out several thousand warnings about duplicate paths in the
manifest. These are harmless and have been around for a long time (see
the phabricator revision for some more details), so let's at least have
a way to make makefs quieter.

Reviewed by: brooks, imp, emaste
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D43513

makefs: Fix ZFS cross-reference sections

MFC after: 1 week
Fixes: 240afd8c1fcc ("makefs: Add ZFS support")
Pull Request: https://github.com/freebsd/freebsd-src/pull/979

Explain the -d option.

makefs: remove ISO9660 Archimedes extension support

We have no need to create ISO images with RISC OS specific metadata.

Reviewed by: imp
Relnotes: yes
Obtained from: OpenBSD a435da44c07f
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39344

Remove $FreeBSD$: two-line nroff pattern

Remove /^\.\\"\n\.\\"\s*\$FreeBSD\$$\n/

makefs: Add ZFS support

This allows one to take a staged directory tree and create a file
consisting of a ZFS pool with one or more datasets that contain the
contents of the directory tree. This is useful for creating virtual
machine images without using the kernel to create a pool; "zpool create"
requires root privileges and currently is not permitted in jails.
makefs -t zfs also provides reproducible images by using a fixed seed
for pseudo-random number generation, used for generating GUIDs and hash
salts. makefs -t zfs requires relatively little by way of machine
resources.

The "zpool_reguid" rc.conf setting can be used to ask a FreeBSD guest to
generate a unique pool GUID upon first boot.

A small number of pool and dataset properties are supported. The pool
is backed by a single disk vdev. Data is always checksummed using
Fletcher-4, no redundant copies are made, and no compression is used.
The manual page documents supported pool and filesystem properties.

The implementation uses a few pieces of ZFS support from with the boot
loader, especially definitions for various on-disk structures, but is
otherwise standalone and in particular doesn't depend on OpenZFS.

This feature should be treated as experimental for now, i.e., important
data shouldn't be trusted to a makefs-created pool, and the command-line
interface is subject to change.

Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35248

makefs: sort cd9660 sysid entries in man page

"efi" was added at the end of the sysid list rather than in alpha
order. Move it as a diff reduction against NetBSD.

Allocate extra inodes in makefs when leaving free space in UFS images.

By default, makefs(8) has very few spare inodes in its output images,
which is fine for static filesystems, but not so great for VM images
where many more files will be added. Make makefs(8) use the same
default settings as newfs(8) when creating images with free space --
there isn't much point to leaving free space on the image if you
can't put files there. If no free space is requested, use current
behavior of a minimal number of available inodes.

Reviewed by: manu
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D29492

makefs: connect cd9660 El Torito EFI boot image system type

Sponsored by: The FreeBSD Foundation

makefs.8: update history

- ported to FreeBSD and first appeared in 8.0
- Christos Zoulas added the FAT support that I imported

makefs: add msdosfs (FAT) support

Add FAT support to makefs by copying some files from sys/fs/msdosfs/ and
updating others with changes from NetBSD.

The six files copied from sys/fs/msdosfs at r348251 and modified are:
denode.h direntry.h fat.h msdosfs_fat.c msdosfs_lookup.c msdosfsmount.h

I would prefer to avoid the duplication, but reluctance to doing so was
expressed in a previous review (D11197); for now copy the files and
revisit in the future.

Submitted by: Siva Mahadevan
Discussed with: cem, imp
MFC after: 1 month
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D16438

makefs.8: expand description of image size

Submitted by: ryan_freqlabs.com, Siva Mahadevan
Differential Revision: https://reviews.freebsd.org/D21316

makefs.8: style updates from igor

Sponsored by: The FreeBSD Foundation

Synchronise with NetBSD's version of EFI handling for El Torito images.

When I implemented my EFI support I failed to check if the upstream version
of makefs in NetBSD had done the same. Override my version with theirs to
make it easier to stay in sync with them in the future.

Reviewed by: imp, mav
Obtained from: NetBSD
MFC after: 1 week
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D14913

makefs: add -O (offset) option

NetBSD revs:
ffs.c 1.60
makefs.8 1.44
makefs.c 1.48
makefs.h 1.33
ffs/buf.c 1.20
ffs/mkfs.c 1.27

Obtained from: NetBSD
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D10780

makefs: Add soft-updates option

Add the ffs option to enable soft-updates.
The option is only processed is ufs2 has been selected.

Reviewed by: emaste, bapt (earlier version), allanjude (earlier version)
Sponsored by: Gandi.net
Differential Revision: https://reviews.freebsd.org/D10773

Point out that -F probably does not do what the user expects.

Users attempting to create images from mtree METALOG files created by
installworld often use -F when they should be passing the METALOG file
in place of a directory. This is often produces difficult to debug
error reports.

Reviewed by: emaste
MFC after: 1 week
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D10038

makefs: sync NetBSD IDs with upstream for changes that we already have

May 22 21:51:39 2011 +0000 (christos):

From Nathan Whitehorn (nwhitehorn at freebsd dot org):
Add code to generate bootable ISOs on Powermac and CHRP systems.
Synthesize some partition maps (APM and MBR, respectively) pointing
to (a) the whole disk, and (b) relevant El Torito boot images that
have been added by other code. These partition maps are a little
bit funny looking, but they seem to work. FreeBSD has been using
this successfully in their release generation on powerpc, as well
as generating all non-SPARC install media. SPARC support could
probably be added as an extension of this patch.

makefs.8 1.33

Tue Aug 23 17:09:11 2011 +0000 (christos):

PR/45285: Martin Matuska: makefs does not properly convert ISO level 1 and 2
filenames (buffer overflow)

makefs does not properly verify the maximum filename length in the
special "." case for both ISO level 1 and ISO level 2 filename
conversion. This creates broken images or causes a buffer overflow
(ISO level 2).

ISO level 1:
If a filename contains only dots or up to 8 characters followed by
dots the 8+3 limit check doesn't work.

ISO level 2:
If a filename contains a dot in the first 30 characters and a dot
on the 30th character, the length limit check doesn't work and the
buffer is overflowed.

$ mkdir level1
$ touch level1/12345............
$ makefs -t cd9660 -o isolevel=1 test.iso level1

$ mkdir level2
$ touch level2/1234567890.2345678901234567.....34567890123456789012345
$ makefs -t cd9660 -o isolevel=2 test.iso level2

cd9660.c 1.32

Sun Oct 9 21:33:43 2011 +0000 (christos):

add support for setting the ufs label. (Nathan Whitehorn)

ffs.c 1.45
ffs.h 1.2
mkfs.c 1.22
makefs.8 1.37

Obtained from: NetBSD

makefs: reorder 'usage' alphabetically

From NetBSD, Mon Aug 15 14:45:01 2011 +0000 (wiz)

Re-order `usage' alphabetically;
rename option arguments in the manpage's `SYNOPSIS' section to
match those from `usage' (not the other way around; the `usage'-line
(and other parts of makefs.c) contain the correct names);
minor punctuation improvements.

From Snader_LB.

makefs.8 1.36
makefs.c 1.30

Obtained from: NetBSD

makefs: Provide a -T option to set timestamps to a consistent value

This is taken from the NetBSD versions listed below and adapted to the
makefs version in FreeBSD, along with a bug fix from cem@ that will be
sent to NetBSD.

Reviewed by: pfg
Approved by: re (gjb)
Obtained from: NetBSD
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D6835

Bump .Dd

Follow up to r290259 dealing with makefs(8)

- Don't use contractions (don't -> do not)
- Change "throw away" to "discard" when describing the -o keep-bad-images option
- Revert author e-mail split I brought over from NetBSD, effectively reverting
the change bapt made in r267668

MFC after: 3 days
Submitted by: bjk
Sponsored by: EMC / Isilon Storage Division

Fix spelling of `isolevel` cd9660 option

MFC after: 1 week
X-MFC with: r290260
Sponsored by: EMC / Isilon Storage Division

Bump .Dd for r290259 and r290260

Document undocumented long options for -t cd9660

Note which options have been implemented and which options haven't
been implemented

Submitted as the following NetBSD PRs: bin/50390 and bin/50392

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Sync makefs(8) content a bit with src/usr.sbin/makefs/makefs.8@1.53

Sections involving unimplemented filesystems (chfs, msdosfs, udf, v7fs) and
options have been omitted.

MFC after: 1 week
Obtained from: NetBSD
Sponsored by: EMC / Isilon Storage Division

Follow up to roundup feature addition in r289203

- Rename -r to -R to avoid the clash with makefs -r in NetBSD
- Note that -R is an FFS-specific option because it's not implemented
in cd9660 today
- Rename the roundup variable to "roundup-size" in the manpage and help
text for consistency with other variables.
- Bump .Dd (missed in r289203)

PR: 203707
MFC after: 1 week
X-MFC with: r289203
Differential Revision: https://reviews.freebsd.org/D3959
Reviewed by: adrian (earlier patch), emaste
Sponsored by: EMC / Isilon Storage Division

Revert r289694

I committed some other undesirable local changes by accident

Add some rudimentary [smoke] testcases for makefs

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

makefs: introduce a new option to specify what to round the resulting
image up to.

From ticket:

While trying to run FreeBSD/mips on some device having very small flash media,
one is forced to compress file system with mkulzma(8) utility. It is desirable
to specify small UFS block/fragment sizes like 4096/512 bytes for makefs(8)
and big compression block size like 65535 bytes to mkulzma at the same time.
Then one obtains very good comression ratios (like 75% and more) but faces
the following problem.

geom_uncompress kernel module reports GEOM provider size rounded up to its
compression block size. Generally, this changes original media size and now
it fails to match the size of embedded UFS file system that leads to other
problems, f.e. geom_label kernel module does not like this and skips the
file system while tasting the GEOM and looking for UFS label.

This makes it impossible to refer to the file system using known UFS label
instead of something like /dev/map/rootfs.uncompress.

The following patch introduces new command line option "-r roundup" for makefs
that makes it round up the image to specified block size. Hence, geom_uncompress
does not change GEOM media size for images rounded that way and geom_label
accepts such GEOMs just fine.

With the patch applied, one can use following commands:

$ makefs -t ffs -r 65536 -o bsize=4096,fsize=512,label=flash optimization=space fs.img fs
$ mkulzma -s 65536 -o fs.img.ulzma fs.img

PR: bin/203707
Submitted by: <eugen@grosbein.net>

use .Mt to mark up email addresses consistently (part2)

PR: 191174
Submitted by: Franco Fichtner <franco@lastsummer.de>

Mark the makefs(8) '-p' flag as deprecated in preference for
the '-Z' flag for compatibility with NetBSD.

Submitted by: Eric van Gyzen (via stable@)
MFC after: 3 days

Remove contractions.

Add a -D flag that causes duplicate entries in an mtree manifest to be
treated as warnings rather than errors.

Reviewed by: marcel
Sponsored by: DARPA, AFRL

Remove trailing whitespace.

Sort options.

Add -p flag to create the image as a sparse file.

Submitted by: Shesha Sreenivasamurthy
PR: bin/167779

Allow contents of multiple directories to be merged to the current image.
Note this patch was submitted to NetBSD and they already adopted it.

http://mail-index.netbsd.org/source-changes/2012/01/28/msg031078.html

MFC after: 1 week

- remove device keyword from makefs manpage

PR: docs/144408
Submitted by: gcooper
Approved by: gjb
Approved by: sahil (mentor)
MFC after: 1 week

Add support to makefs(8) to add UFS labels to images.

Submitted by: avg
MFC after: 3 days

Add support for using mtree(5) manifest files to define the image
to be created. The support is based on mtree version 2.0, as used
in libarchive, but adds new features on top of it.

The current implementation is fully functional, but is envisioned
to grow at least the following additional features over time:
o Add support for the /include special command so that manifest
files can be constructed using includable fragments.
o Add support specifying a search path to locate content files.
o Content file filters: commands that provide file contents on
stdout.

The manifest file eliminates the need to first construct a tree
as root in order to create an image and allows images (releases)
to be created directly from object trees and/or source trees.

Reviewed by: deo
Sponsored by: Juniper Networks, Inc

Add analogs to the -chrp-boot and -prep-boot options to mkisofs.

Sync with the latest version from NetBSD. It notably addds ISO9660 support.

Submitted by: bapt

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Fix typos and spelling mistakes.

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

Correct copy-paste typo from previous option description.