- Copy stable/10 (r259064) to releng/10.0 as part of the
10.0-RELEASE cycle.
- Update __FreeBSD_version [1]
- Set branch name to -RC1

[1] 10.0-CURRENT __FreeBSD_version value ended at '55', so
start releng/10.0 at '100' so the branch is started with
a value ending in zero.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFV r248590,248594:
Update libarchive to 3.1.2

Some of new features:
- support for lrzip and grzip compression
- support for writing tar v7 format
- b64encode and uuencode filters
- support for __MACOSX directory in Zip archives
- support for lzop compresion (external utility)

Update libarchive to 3.0.4

Update libarchive to 3.0.3

Some of new features:
- New readers: RAR, LHA/LZH, CAB reader, 7-Zip
- New writers: ISO9660, XAR
- Improvements to many formats, especially including ISO9660 and Zip
- Stackable write filters to write, e.g., tar.gz.uu in a single pass
- Exploit seekable input; new "seekable" Zip reader can exploit the Zip
Central Directory when it's available; the old "streamable" Zip reader
is still fully supported for cases where seeking is not possible.

Full release notes available at:
https://github.com/libarchive/libarchive/wiki/ReleaseNotes

Use contrib sources for building libarchive, tar and cpio.
Make "make test" fully operational.

MFC after: 2 weeks

Sync bsdtar with vendor branch release/2.8:

Revision 3769:
Merge r3744 from trunk: Correctly return errors when reading
an archive using @archive extension.

Obtained from: http://code.google.com/p/libarchive
MFC after: 2 weeks

Update libarchive, tar and cpio to version 2.8.5

The following additional vendor revisions are applied:

Revision 3740:
Use archive_clear_error() to clear the error markers.

Obtained from: http://code.google.com/p/libarchive
MFC after: 2 weeks

In usr.bin/tar/tree.c, if you really want to poke to NULL, you must use
volatile, otherwise the indirection will not be emitted.

MFC after: 1 week

Typo from previous commit. Urgh.

Bring in the --gid --gname --uid and --uname implementation
from libarchive.googlecode.com.

MFC after: 3 days

Correctly link bsdcpio and bsdtar against libmd and libcrpyto
by applying the change from r221472 (libarchive).

Reviewed by: kientzle
Approved by: re (kib)
MFC after: 3 days

Fix accidential commit part:
Include "bsdtar_platform.h" instead of "lafe_platform.h"

MFC after: 2 weeks

Update bsdtar to 2.8.4
Use common code from lib/libarchive/libarchive_fe

Approved by: kientzle
MFC after: 2 weeks

The --newer-than test should descend into old
directories to look for new files.

PR: bin/150890
Submitted by: Tobias Herre
MFC after: 3 weeks

If there is a read error reading Y/N confirmation from the keyboard,
exit immediately with an error.

If there is an error opening or reading a file to put into the archive,
set the return value for a deferred error exit.

PR: bin/154407

Revert most of r210764, now that mdocml does the right
thing with empty quotation macros.

Requested by: Alex Kozlov

Recognize both ! and ^ as markers for negated character classes.

Submitted by: Mykola Dzham

mdoc: make sure to pass at least one argument to quotation macros

Fix typos.

If the compressed data is larger than the uncompressed,
report the compression ratio as 0% instead of displaying
nonsense triggered by numeric overflow. This is common
when dealing with uncompressed files when the I/O blocking
causes there to be small transient differences in the
accounting.

Thanks to: Boris Samorodov

mdoc: move remaining sections into consistent order

This pertains mostly to FILES, HISTORY, EXIT STATUS and AUTHORS sections.

Found by: mdocml lint run
Reviewed by: ru

Enable liblzma support in libarchive
Adjust dependencies for programs using libarchive
Add xz and linkage against liblzma to rescue system

Approved by: kientzle, delphij (mentor)
MFC after: 2 weeks

FreeBSD is now using bsdtar 2.8.3.

Config updates.

Various manpage updates, including many long-option synonyms that were previously undocumented.

If a file is specifically both included and excluded, then:
* It is not extracted (because it is excluded)
* If it's not present in the archive, then an error is
reported (because the file was requested and not found)
* If it is present in the archive, no error is reported.
Previously, this would always report an error because the
exclusion prevented the entry from matching the inclusion.

Also, tar is now more reluctant to report unmatched inclusions.
Previously, "tar x file1 'file*'" against an archive that contained a
single entry "file1" would match file1 and then report an error for
the second pattern because it wasn't matched. It now considers both
inclusions to be matched and reports no error.

Diff reduction against bsdtar 2.8.3

Diff reduction against bsdtar 2.8.3

Fix -X.

Consistently specify O_BINARY when opening files.

Remove an unnecessary include and conditionalize some code.

Correct the markup for the -s option,
document the iso9660:rockridge and mtree:indent options

Fix several typos in macros or macro misusage.

Found by: make manlint
Reviewed by: ru
Approved by: philip (mentor)

Fixed dependencies (make checkdpadd).

Fix common misspelling of hierarchy

Pointed out by: bf1783 at gmail
Approved by: np (cxgb), kientzle (tar, etc.), philip (mentor)

Simplify, remove unnecessary code.

Merge a bunch of refactoring from Joerg Sonnenberger to
isolate common code used by tar and cpio (and useful to other
libarchive clients). The functions here are prefixed with
"lafe" (libarchive front-end) to indicate their use.

Style & Portability: Use archive_entry methods to examine
file information, change some functions to static, remove
some unused headers.

Various portability workarounds for non-FreeBSD platforms.

Restructure the logic that determines when we're crossing a mount
point. In particular, this carves out a place for detecting and
excluding synthetic or network filesystems.

Minor code rework.

Trim out some unused configuration variables, remove
some unused headers, etc.

Fill in some missing error handling, be a little more careful about
error reporting, prefer int64_t to off_t.

bsdtar doesn't actually know what compression is supported by libarchive
and it should not pretend that it does. It should just pass along the
user's request and handle an error if it's not supported.

Refactor the siginfo/sigusr1 handling. The read/write reporting
is sufficiently different that it was simpler to just put separate
reporting functions into read.c and write.c rather than try to have
a single all-purpose reporting function.
Switch to a custom function for converting int64_t to a string; in
the portable version, this saves a lot of configuration headaches
trying to decipher the platform printf().

Remove all traces of an experiment for handling "root" on Windows systems.

Reformat the Makefile slightly.

Allow -b up to 8192. I've had reports from people who routinely
use -b 2048 (1MiB block size). Setting the limit to 8192 should
allow some room for growth while still helping people who mistakenly
put in byte counts here instead of block counts.

Diff reduction compared to portable bsdtar 2.8: Move the
program name into a global, which eliminates an extra
argument from a lot of places.

Build usr.bin/ with WARNS=6 by default.

Also add some missing $FreeBSD$ to keep svn happy.

Revert most part of 200420 as requested, as more review and polish is
needed.

Remove unneeded header includes from usr.bin/ except contributed code.

Tested with: make universe

Make sure to use up-to-date libarchive header files from source tree when
compiling tar and not the ones from /usr/include/.

Reviewed by: kientzle
Approved by: kientzle

Make -lcrypto usage dependent on whether or not we're building with OpenSSL.

Merge remaining changes from libarchive.googlecode.com:
* Add xz and lzma compression options
* Rename --format-options to simply --options
* Add --same-owner for GNU tar compat
* Add -lmd and -lcrypto to fix link
* Documentation

Set options before opening the archive.
Catch and report close-time errors.

Minor portability improvement in calls to ctype.h macros.

Merge from libarchive.googlecode.com: Numerous Windows-specific build tweaks.

Update version to 2.6.901a to indicate this is synced up with r745 of
libarchive.googlecode.com (except for the lzma/xz support).

Merge r492 from libarchive.googlecode.com: First cut at exposing the
new options mechanism to userland. Documentation pending...

Match a comment to reduce differences with libarchive.googlecode.com.

Merge r709,r710 from libarchive.googlecode.com: More work on
Windows support.

Merge r687-689,691,693-701,720 from libarchive.googlecode.com:
Translate getdate.y into C for portability. Make the get_date()
function easier to test as well:
* Have it accept a time_t "now" to use as a reference so that test
code can verify relative time specifications against known starting
points.
* Set up default date after parsing the string so that we
can use the specified timezone (if any) instead of the local
default. Otherwise, local DST makes it almost impossible to
reliably test time specifications such as "sunday UTC"

Merger r629-631,633-646,648,654,678,681,682 from libarchive.googlecode.com:
Many changes for Windows compatibility. bsdtar_test now runs successfully
on both POSIX platforms and Windows.

Merge r368,496,625,626 from libarchive.googlecode.com: A number of
style and portability tweaks to the test harness. Most significantly,
don't use getopt().

Merge r622 from libarchive.googlecode.com: Avoid warning on platforms
that lack regex.h.

Merge r552,r559 from libarchive.googlecode.com: High-resolution time
support on Tru64, AIX, and GNU Hurd, thanks to Björn Jacke.

Merge r529 from libarchive.googlecode.com: Fix how we read ext2fs_fs.h
headers on Linux.

Merge r435,r443 from libarchive.googlecode.com: Let the compiler options
determine how to read config.h.

Merge r283,r423 from libarchive.googlecode.com: Use libarchive's new
archive_read_disk API to pull metadata off of disk. This
removes a lot of platform-specific knowledge of things like
ACLs, file flags, and extended attributes from bsdtar.

Merge r374 from libarchive.googlecode.com: Stupid typo in open() call. <sigh>

Merge r369 from libarchive.googlecode.com: Test -s option.

Merge r278 from libarchive.googlecode.com: Reduce the number of
patterns tested here from 200 to 170, which seems to be the
most that Cygwin can handle.

Merge r273 from libarchive.googlecode.com: Use open() correctly.

Fix multiple -s options.

MFC after: 7 days

When copying file data to the archive, don't write more
than was read. This seems to have only affected the shar
writer, since other formats proactively truncate output
to the originally-advertised size.

PR: bin/131244
MFC after: 7 days

Spell "substitution" correctly.

PR: bin/128616
Submitted by: Jaakko Heinonen
Pointy hat: Tim Kientzle
MFC after: 30 days

bsdtar is synced up with 2.5.903a

bsdtar always supports long options now, so we can remove
a documentation disclaimer.

Only use GCC-specific __dead2 marker when building with GCC.

When deciding which characters (of a filename, for instance)
are safe to print, try to take into account the current locale.
This iterates over output strings using mbtowc() to identify
multi-byte sequences. If iswprint() claims the corresponding
wide character is printable, the original bytes are passed
through. Otherwise, we expand characters into C-style
\-escape sequences.

Submitted by: Michihiro NAKAJIMA
MFC after: 30 days

After visiting a subdirectory, we use chdir("..") or
fchdir() to return back to the parent. If those fail,
we're just dead in the water. Add a new error value
TREE_ERROR_FATAL to indicate that directory traversal
cannot continue. Have write.c honor that by exiting
immediately.

MFC after: 30 days

If we're unable to chdir() to a directory that we know exists,
that should result in a non-zero return value.

In particular, this should address the issue that David Wolfskill
ran into with a somewhat flaky NFS mount resulting in a damaged
archive even though tar returned success.

MFC after: 4 days

Include more detailed explanation of this case, since it's pretty
subtle why it comes out the way it does. Once you realize that it
depends on the archiving order, it's also important to realize that
filesystem differences aren't going to break this case. (Some of the
other tests have had to be extensively rewritten to make them
independent of the order in which a particular filesystem returns file
entries.)

(This commit also serves to note the PR number that I accidentally
omitted from the previous commit.)

PR: bin/128562
MFC after: 30 days

Test --strip-components and fix it to actually work. Jaakko did a
good job writing this test; it exercises a lot of subtle cases. The
trickiest one is that a hardlink to something that didn't get
extracted should not itself be extracted. In some sense, this is not
the desired behavior (we'd rather restore the file), but it's the best
you can do in a single-pass restore of a tar archive.

The test here should be extended to exercise cpio and newc formats as
well, since their hardlink models are different, which will lead to
different handling of some of these edge cases.

Submitted by: Jaakko Heinonen
MFC after: 30 days

New command-line parser for bsdtar.

This replaces the getopt()/getopt_long() wrapper, the old-style
argument rewriter and the associated configuration glue with a more
straightforward custom command parser. In particular, this ensures
that bsdtar will have consistent option parsing on every platform,
regardless of whether the platform supports getopt_long().

MFC after: 30 days

When comparing, cast to the larger size, off_t in this case.
Once we know which one is smaller, then we cast to the smaller size.

Thanks to Xin Li (delphij@)
Pointy hat: /me

Fix compile warnings building on amd64. This is modified slightly
from Jaakko's original patch: I have misgivings about the portability
of the 'z' printf modifier so opted to cast the arguments to (int)
instead.

PR: bin/128561
Submitted by: Jaakko Heinonen
MFC after: 30 days

Clean up flags support just a tad: FreeBSD support depends on
HAVE_STRUCT_STAT_ST_FLAGS, Linux support depends on the
existence of the appropriate ioctl() options. In particular,
this should fix some nagging compile errors on Linux platforms
that don't have e2fsprogs-devel installed.

Test handling of restores relative to symlinks.
In particular:
* tar -x -P follows symlinks to existing dirs, but not without -P
* symlinks to files are always replaced
* broken symlinks are always replaced

Minor cleanup of the -q test: Assert that stdout/stderr are empty for each
extraction.

Don't clean before running tests.

Explain how the test_option_q test works.

Test for -q (aka --fast-read).
Fix the error uncovered by this test.

The results for test 2 here are short enough to just be included inline.
There's no need to go through the hassle of having a checked-in uuencoded
reference file for comparison.

Add some more tests to verify that "./foo" matches "foo" but "/foo" does not.

Always display the unedited pathname in -t output.
I would like to provide a way to preview the effects of pathname edits,
but pattern selection has to happen against the unedited path, so it
seems that we have to show people the unedited path to help in
designing selection patterns.

Better comment the pattern tests; adjust the filenames for the
reference files to match the corresponding source.

MFC after: 3 days

Fix SVN r181821 by not using FNM_LEADING_DIR where
it shouldn't be used.

Strip leading "./" or ".//" from patterns and filenames
so that "./foo" matches "foo" (and vice versa).

This is related to PR bin/119141.

PR: bin/119141
MFC after: 4 days

Don't strip a leading '/' from a pattern to be matched.
This aligns the pattern-matching behavior with GNU tar
and restores the behavior of bsdtar prior to FreeBSD 6.3.

PR: bin/117909
MFC after: 3 days

Test updates: Handling of patterns on command line, error messages.

Mark functions as __dead2 in order to help the LLVM static checker
understand which code paths aren't possible.

This commit eliminates 117 false positive bug reports of the form
"allocate memory; error out if pointer is NULL; use pointer".

- Fix mtree example so it works.
- Document --version.

Reviewed by: kientzle
MFC after: 1 week

Move duplicated code from tar_mode_[cru] into archive_write.

Fix a bug I introduced 7 minutes ago: clean up properly from archive_write
if we exit the argv-handling loop due to -C not having an argument.

Use malloc in write_archive to allocate a 64kB buffer for holding file data
instead of using 64kB of stack space in copy_file_data and write_file_data.

Clean up write_entry by eliminating fd except in the #ifdef __linux block
where it is used. [1]

Don't leak file descriptors in write_entry_backend if archive_write_header
returns ARCHIVE_FAILED.

Found by: Coverity Prevent [1]

Revert CVS revision 1.68; it is now possible for entry to be NULL at the end
of write_entry. (This was perfectly safe, since archive_entry_free(NULL) is
a no-op, but adding the check back makes the style more consistent.)

FreeBSD-CURRENT bsdtar is synched up with libarchive-portable 2.5.5.

Remove HAVE_STRUCT_STAT_ST_RDEV, as it's not used anywhere.

Stop using the deprecated linux/ext2_fs.h.

Use %H:%M for strftime() time formatting instead of the non-portable %R.

Pass the entry down into the core write loop, so we
can include the filename when reporting errors.

Thanks to: Dan Nelson

MfP4: Minor portability fix.

MfP4: test harness cleanup.

Make one-bit fields unsigned instead of signed. This has no effect,
since they are only tested for zero/nonzero; but it's arguably a bad
idea to set a {-1, 0} variable to 1 (as happens in this code).

Found by: Coverity Prevent

Rework code to avoid using a pointer after freeing it. Aside from the
possibility of memory becoming undereferenceable when it is freed, this
change should have no effect on bsdtar behaviour.

Found by: Coverity Prevent

Compatibility fix: define REG_BASIC if it isn't already. In particular,
glibc has a suitable regex implementation, but doesn't define this
constant.

Thanks to: Diego "Flameeyes" Pettenò

MFp4: bsdtar 2.5.4b

In addition to a number of bug fixes and minor changes:
* --numeric-owner (ignore user/group names on create and extract)
* -S (sparsify files on extraction)
* -s (regex filename substitutions)
* Use new libarchive 'linkify' to get correct hardlink handling for
both old and new cpio formats
* Rework 'copy' test to be insensitive to readdir() filename ordering

Most of the credit for this work goes to Joerg Sonnenberger, who
has been duplicating features from NetBSD's 'pax' program.

The value le->name cannot be NULL when we're freeing an entry in the
hardlink table for two reasons: 1. If le->name is set to NULL, the
structure le won't be inserted into the table; 2. Even if le somehow
did manage to get into the table with le->name equal to NULL, we would
die when we dereferenced le->null before we could get to the point of
freeing the entry.

Remove the unnecessary "if (le->name != NULL)" test and just free the
pointer.

Found by: Coverity Prevent

Improve portability via s/struct siginfo/struct siginfo_data/. This was
running into a namespace collision on an avian operating system.

Detect if argv[1] is "" and avoid calling malloc(0). Prior to this commit,
running 'tar ""' would print 'No memory' instead of the correct error
message, 'Must specify one of -c, -r, -t, -u, -x' if malloc is set to
System V mode (malloc(0) == NULL).

There's no way for entry to possibly be NULL at the end of write_entry
(in fact, there has never been any way for it to be NULL, going all the
way back to revision 1.1 of this file), so remove the check and
unconditionally free entry.

Found by: Coverity Prevent

Add SIGINFO (and for portability to SIGINFO-lacking systems, SIGUSR1)
handling to bsdtar. When writing archives (including copying via the
@archive directive) a line is output to stderr indicating what is being
done (adding or copying), the path, and how far through the file we are;
extracting currently does not report progress within each file, but
this is likely to happen eventually.

Discussed with: kientzle
Obtained from: tarsnap

Add --keep-newer-files option (as in GNU tar: When in -x mode, ignore
files if the existing file is newer than the archive entry).

Currently if any files are ignored, bsdtar will exit with a non-zero
exit status; this is likely to change in the future, but requires some
API changes in libarchive.

Discussed with: kientzle
Obtained from: tarsnap

Documentation updates:
* --format can be used with -r or -u
* -o is a synonym for --format=ustar when used with -c, -r, or -u
Also, fix the erroneous sanity check that suppressed --format with -r or -u.

bsdtar --version should succeed.

New bsdtar test harness. Still rather skimpy, but a lot easier
to run and maintain than the old scripts that used to be here.

Allow -r with -T even if there are no files on the command line.

PR: bin/123246
MFC after: 3 days

Fix --fast-read by decrementing the remaining patterns to be matched,
instead of incrementing.

Pointy hat: me
Thanks to: Tomasz Przygoda
MFC after: 3 days

style(9) fix to add whitespace around an '=' sign and restore
a comment that got accidentally dropped.

Support NetBSD's --insecure as a synonym for -P.

Document --chroot.

Support -q (from NetBSD) as a synonym for --fast-read (the old FreeBSD
extension to GNU tar).

Thanks to: Joerg Sonnenberger

Also accept --uncompress as a synonym for -Z.

Thanks to: Joerg Sonnenberger

--chroot implementation thanks to Joerg Sonnenberger.

Support --compress as a synonym for -Z.

Enable and document the -Z option for selecting "compress" compression,
now that libarchive supports it.

Allow appending to archives on block devices as well as
archives in regular files, since both are seekable.

Thanks to: Reinoud Zandijk (@netbsd.org)

Make -l always have the SUSv2 meaning of "check links."
GNU tar changed -l to match SUSv2 a couple of years ago,
so bsdtar no longer needs to pander to this particular GNUism.

Thanks to: Debian maintainers
MFC after: 7 days

Remove the old bsdtar test scripts; something much
better is almost ready to commit.

A couple of miscellaneous fixes:
* prototypes for optarg/optind on platforms that don't already have them
* Disambiguate version number macros
* Remove unnecessary PACKAGE_NAME macro
* Hook for forthcoming bsdtar test suite
* Sync version number up with the portable distribution

Fill in the approximate date when tar(1) first appeared.

Include a suitable stub definition of __FBSDID() for non-FreeBSD platforms.

Use archive_entry_strmode() instead of a local bsdtar_strmode().
(This does a couple of things that the standard library's strmode()
doesn't; it proved useful in bsdcpio as well, so I pushed it down
into libarchive.)

Include Guido's copyright for the fnmatch() code I cribbed from.
(It's in the C source, just wasn't in the COPYING file until now.)

Add --posix as a synonym for --format=pax, for GNU
tar compatibility.

Approved by: re (bmah)

Fix for PR bin/115033. This corrects a crash when long options
are specified with the "-W option=arg" syntax and the '=' is omitted.

MFC after: 3 days
Approved by: re (rwatson)

Fill in some casts that are needed (according to GCC 4.1)

Thanks to: Joerg Sonnenberger
Approved by: re (hrs)
MFC after: 3 days

Don't exit immediately on libarchive warnings, just
set the delayed return value and keep going.

Approved by: re (hrs)
MFC after: 7 days

bsdtar 2.2.3:
* Implement --use-compress-program using new libarchive feature.
* Minor portability improvement by adjusting casts used to
print out uids, gids, and device numbers.

Thanks to: Joerg Sonnenberger for the --use-compress-program implementation.
MFC after: 15 days

Mention the formats supported by bsdtar; include an
example that shows how to work with an ISO 9660 image.

Set the program return value non-zero if we enounter
an error while skipping the body of an entry during
"tar -t" processing.

Thanks to: Spencer Minear
PR: misc/112521

If lstat(2) fails, have bsdtar return a non-zero exit code if the
failed path is one which was specified on the command line.

This is a compromise between the situation prior to revision 1.57
(where a race between tar(1) and rm(1) could cause tar(1) to
spuriously report an error) and the situation after revision 1.57
(where "tar -c /no/such/path" prints a warning but returns with
an exit code of zero).

Inspired by: rafan
MFC after: 1 week

Ignore trailing '/' when comparing pathnames, so that
"tar -u" works again. Otherwise, "tar -u" wants to
treat every dir as new and re-adds it.

When testing basic functionality, strip trailing '/' from
dir names, so they match the names generated by 'find'.

In libarchive: Downgrade ARCHIVE_FATAL and ARCHIVE_FAILED errors which
occur on the write side of extracting a file to ARCHIVE_WARN errors
when returning them from archive_read_extract.

In bsdtar: Use the return code from archive_read_data_into_fd and
archive_read_extract to determine whether we should continue trying to
extract an archive after one of the entries fails.

This commit makes extracting a truncated tarball complain once about
the archive being truncated, instead of complaining twice (once when
trying to extract an entry, and once when trying to seek to the next
entry).

Discussed with: kientzle

Portability.
Thanks to: Martin Koeppe for testing on Interix

Make it easier to support more platforms.
Thanks to: Joerg Sonnenberger for pointing out the need and the technique.

Clean up error handling in archive_append to match how errors are
handled in write_hierarchy.

Approved by: kientzle

Simplify the logic for handling header read
errors.

Add test to confirm that piping a tar archive created by bsdtar through
'bsdtar -cf- @-' doesn't alter it.

Add tests for "bsdtar -t". These are useful primarily because they test
the archive_read_data_skip code.

Split the append_archive function (used for processing @<archive> directives)
into separate append_archive and append_archive_filename functions; the first
takes a "struct archive *" as input, while the second takes a filename, opens
the archive, and calls the first.

There should be no changes in behaviour as a result of this commit; it simply
reorganizes code to make more sense. At some point in the future it may be
possible to share code between append_archive and read_archive, but not yet.

Discussed with: kientzle

Issue a warning if there's a non-zero exit value.

Remove pathlen argument from write_entry function. It has never been used.

Approved by: kientzle
MFC after: 3 days

Fix logic bug; we want to do_chdir if arg doesn't start with / _and_ it
doesn't start with @/ either.

This unbreaks "tar -c -C /no/such/directory @/path/to/archive".

MFC after: 3 days

Don't consider an lstat(2) failure to be an error (in the sense of
affecting the return value from bsdtar), since (a) it usually occurs
due to a perfectly innocent (and unavoidable) race condition where a
user deletes a file in the window between bsdtar reading a directory
and attempting to read the file; and (b) aside from printing a warning
message, bsdtar behaves exactly as if the file had been deleted prior
to bsdtar reading its parent directory.

Reviewed by: kientzle
MFC after: 6 days

Reduce the risk of inducing heart attacks, by printing the right path when
complaining about lstat(2) failing. It's a bit scary to find the message
tar: /: Cannot stat: No such file or directory
printed while doing a backup.

MFC after: 1 week

Clarify the test comments in test-basic.sh. Have config.sh do a
better job searching for the bsdtar binary to test and the gtar binary
to use for inter-operability testing. It should now find the built
(but not installed) binary if there is one, then search for an
installed binary in a number of standard locations.

bsdtar 2.0.23:
* New test scripts exercise some basic functionality
* Most header inclusions are now protected (portability)
* read.c now relies on security checks in libarchive instead
of trying to do its own (optimization)
* -p now enabled by default for root, add --no-same-permissions
to disable it
* Comments, minor style fixes.

Argh. Support for reading ACLs off of disk was inadvertently disabled
as part of an old configuration shuffle. As a result, although
ACL restore has been working, ACLs haven't been written into archives
for some time. <sigh>

Pointy hat: You know.
MFC after: 3 days

Make the file tests robust against broken symlinks and other
sources of stat()/lstat() failure.

If we already have stat() data, we might be able to
determine if this is a physical dir without an lstat().
While I'm in here, try to clarify the comments around
the _is_dir() and _is_physical_dir() tests.

Andrew and Colin each pointed out to me that truncating the backup
of a growing file should not be considered a "bad thing."

PR: bin/108990
MFC after: 7 days
Pointy hat: /me

Correctly handle writes beyond the end of the archive entry
(as determined by the initial size given to the header).
Libarchive recently changed to correctly return the amount
of data actually consumed in this case, which revealed this
bug in bsdtar.

Update -r handling:
* Create file if it doesn't exist.
* If archive is "empty", then append to it with pax restricted
* If user specified a format, use that if it's compatible with
the existing format.

Correct the copyright messages: Make this a standard vanilla
2-clause BSD license, update the year to 2007.

If append_archive fails while writing an archive header, output the error
message from the archive being written (not the message from the archive
being read, where no error has occurred).

MFC after: 3 days

Clean up the struct archive used for reading an archive in the handling
of @archive commands. This bug should be harmless as long as you don't
use an excessive number of @archive commands.

MFC after: 1 week

Handle errors which occur during archive_write_data and archive_write_close
by printing an error message and exiting with a non-zero status code.

MFC after: 1 week

Fic typo in previous commit.

MFC after: 3 days

Add two checks for inappropriate options: -U only makes sense in "x"
mode, and --strip-components only makes sense in "x" and "t" modes.

MFC after: 3 days

Correct the names of some options.

MFC after: 3 days

Remove -F option from getopts string -- this option has never done
anything apart from invoking usage(), and apparently slipped in by
accident.

Approved by: kientzle
MFC after: 3 days

Fix "1 day ago", "yesterday" and similar relative-time phrases
when parsing dates.

PR: bin/104934: bug in some date handling of bsdtar
Reported by: Herve Boulouis
MFC after: 3 days

Remove two unused variables.

Thanks to: Stefan Farfeleder

Conditionally include sys/mkdev.h on platforms (such as Solaris) that need it.
While I'm here, correctly alphabetize 't' after 's'.

Thanks to: VMiklos

In the examples, use "tar.gz" as the extension for
gzip-compressed tar archives.

Thanks to: Mike Hunter

If you see a broken symlink, don't try to follow it,
just archive it as a symlink, even if -h was specified.

Thanks to: Jin Guojun
PR: bin/95175
MFC After: 1 week

Extended attribute support on write for Linux; FreeBSD hooks are
forthcoming. This commit also has a number of style(9) fixes and
minor corrections so the code works better with the build system being
used for non-FreeBSD builds.

Many thanks to: Jaakko Heinonen, who proposed a mechanism for extended
attribute support and implemented both the machine-independent portion
and the Linux-specific portion.

Remove non-FreeBSD source from FreeBSD repository.

Remove configure.ac.in and the 'distfile' target from
the FreeBSD makefile. This is a step towards cleaning
out the non-FreeBSD bits from the FreeBSD source tree.

Add a check for Linux' "libacl" library and headers.

Change the output of --version to match GNU "gnits" standards.
(I'm not using GPL, but I still think there are good
ideas in the GNU projects. ;-) Among other things,
this should make it easier for clients of bsdtar to
recognize it automatically:
bsdtar --version | grep bsdtar

Fix a format mismatch (%d integer format, platform-dependent
argument type).

If opendir(".") fails after descending into a subdirectory via
chdir(), be sure to undo the effects of the chdir before continuing.

Without this, after hitting a directory with mode 0111 (for example),
tar will get lost, and won't add any yet unvisted files to your
archive. (Or possibly add the wrong files, I suppose...)

Reviewed By: kientzle@

Use the .Fx macro.

MFC after: 3 days

Remove the -B option from the directory-copying examples. The -B
option is undocumented because it does nothing. It does nothing
because bsdtar never needs it. It is accepted because gnutar does
sometimes need it and many scripts use it.

Reported by: Pawel Jakub Dawidek

Portability: remove AC_FUNC_MALLOC from configure.ac.in.
According to Dan Nelson, this fixes the build on AIX 5.2.

Edit pathnames for -x but not for -t. Otherwise, people get confused
when list the archive contents, then try to extract selected files
(file selection always works against unedited pathnames). With this change,
-t always shows the pathnames as they appear in the archive.

Thanks to: Robert Watson

In mode 'u', check for an error return from archive_read_open.
This causes attempts to update a non-existent file to report
an actual error instead of triggering an assertion failure.

PR: bin/87911
Thanks to: roemer.ulrich
MFC after: 3 days

Note: This does not entirely fix bin/87911. I need to decide on
the "correct" response when someone tries to update a non-existent
archive file.

Include <sys/param.h> in order to get current __FreeBSD_version
in order to properly set HAVE_NL_LANGINFO. <whew!>

Thanks to: Andrey Chernov
PR: bin/88013
MFC after: 3 days

Spell "--format" correctly in error message. (It hasn't been
called "-F" for a very long time.)

PR: bin/86915
Thanks to: Gary W. Swearingen
MFC after: 3 days

Fix -u with absolute paths (e.g., "tar -uf foo.tar /bar") by handling
pathname edits before comparing pathnames on disk to those in the archive.

Thanks to: Gareth Bailey, Lowell Gilbert

Detect Mac OS X's broken ACL library.

Thanks to: Diego "Flameeyes" Petten?

Style fix: Correct a compiler warning from GCC 4

Thanks to: Divacky Roman
PR: bin/84993

The bsdtar_warnc() reporting function requires the program name to be
set up before it is called, so move the progname initialization before
the first possible call to bsdtar_warnc().

Thanks to: Stanislav Sedov
PR: bin/83366
MFC after: 7 days

Fixed troff warnings.

Approved by: re (blanket manpages)

Clarify some error messages.

bsdtar now uses the "tree" package instead of "fts" for walking
directory heirarchies.

Correct autoconf/automake invocation to work with devel/gnu-auto* from
ports.

Thanks to: Dag-Erling Sm?rgrav (des@)

Minor fix to the configure logic to properly detect ACL support.

Overhaul getdate.y. In particular, remove minutes-based logic
(combine with existing seconds-based), treat '-' as punctuation rather
than a negative number indicator (eliminates several special cases),
use a single list of special words instead of several separate lists,
use table-driven abbreviation logic (eliminate duplicate word entries
and special-case abbreviation and plural handling). The result is
shorter, simpler (judging from comments, earlier maintainers didn't
understand the special handling for "negative years"), handles more
cases (e.g., "tu" is now a recognized abbreviation for "tuesday",
"3rd" is now equivalent to "third") and it has 2 fewer shift/reduce
conflicts.

Point users to tcsh(1) for information about shell globbing patterns,
rather than 'XXXX'.

MFC after: 1 week

A very minor tweak to the handling of leading '/' characters.

A number of fixes to the autoconf-generated build system. In
particular, acl support is no longer enabled on FreeBSD 4, acl support
should be correctly enabled on Linux, dirent.d_namlen should be
correctly detected on platforms that support it.

Thanks to: Greg Lewis, Juergen Lock, and Jaakko Heinonen

Configure script needs to test for dirent.d_namlen.

Thanks to: Juergen Lock

Update "make distfile" to use newest automake/autoconf from ports.

Thanks to: Juergen Lock

Support path-rewriting options (including --strip-components) for both
extraction and creation. While I'm here, fix a bug reported by Garrett
Wollman: when stripping the leading '/' from the path "/", don't produce
an entry with an empty name; produce "." instead.

Add directory hierarchy copying example.

Deorbit gtar, step 1: Remove WITH_GTAR knob.
"tar" is now always a synonym for "bsdtar".
People who need gtar can access it as "gtar."

Approved by: No objections on arch@

A handful of minor portability and style improvements.

Style correction: one tab after #define.

Re-unbreak the distfile target.

Also, reduce the WARNS level to 5 since different build environments
end up using different Yacc skeletons. The BSD one does not
predeclare yyparse, the FSF one does, so it's not really possible to
consistently enforce both -Wmissing-prototypes and -Wredundant-decls.

Fix a compile warning, fix the build.

"make depend" with .y files is trickier than it looks. <sigh>

Brain-o. Missing quote.

Add --newer-ctime, --newer-mtime, --newer-ctime-than, and --newer-mtime-than
switches to support selecting files by time of modification.

Special thanks to: Steven M. Bellovin, Rich $alz, and Jim Berets,
authors of the public-domain getdate.y date-parsing code.

Add --strip-components option, per bin/77666.

Thanks to: Sangwoo Shim

Document the --null, --one-file-system, and -I options.
Clarify that -T can be used in 'x' mode as well as 'c' mode.

Sort sections.

Added the EXIT STATUS section where appropriate.

Add a fallback version of fnmatch() to the bsdtar
source code. Include configure logic to pick up the
system one when it exists and use the fallback version
when it doesn't exist. Set the default for FreeBSD
to use the system version.

With this, bsdtar should now be quite portable.

GC an unused #include

Fix -W long option handling.
In particular:
-W excl=text
fails because "excl" is a prefix of both "exclude" and "exclude-from". But,
-W exclude=text
is okay because it matches "exclude" exactly.

Thanks to: Jose F Nieves
MFC after: 7 days

Portability: the included fts.c should now
compile on many non-FreeBSD systems, including
Linux, Solaris, Darwin, etc.

Thanks to: many people who pointed out the same portability problem

Use a relative symlinking so that "tar" also works in /stand.

FreeBSD does have uintmax_t, so it can correctly
print out very large file sizes.

Thanks to: Yar Tikhiy

Correct error handling on failed writes
to the archive.

getopt(3) prints an "unrecognized option" warning for me, so I
shouldn't print another.

MFC after: 15 days

Cosmetic goof: Use two spaces in "Jan 03 2004" but only one in "Jan 03 12:32"
MFC after: 15 days

Make -C and -T work correctly together.

MFC after: 3 days
Thanks to: Guy Helmer

Document the -O option's new behavior in the presence of -t.

MFC after: 3 days

Support -t -O as in gtar. Perversely enough, -O means "send to stderr"
when used with -t, "send to stdout" when used with -x.

Thanks to: Ryan Hamilton for pointing out this odd beast
MFC after: 3 days

Permit -P to be combined with -t. (It's a no-op then, just as in gtar.)

Tell getopt to accept the (already-implemented) -I option.

Make -I and --files-from be synonyms for -T

gtar uses -F for another purpose, so bsdtar no longer treats it as a
synonym for --format. Update the man page to reflect this. While
I'm here, change the man page to document "tar" rather than "bsdtar,"
update some comments about -l compatibility and fix a few grammar nits.

Better low-memory handling: If the link cache runs out of memory, just
throw out the whole thing and stop tracking links entirely. That will
break all remaining hardlinks, but should free up enough memory to
let everything finish.

Move the smart chdir logic into a couple of utility functions in util.c.
Then use them to provide consistent -C support throughout the program.
Thanks to: Christoph Mallon

It shouldn't be an error to specify the same mode twice: allow -cc but not -ct.
Thanks to: Christoph Mallon (whose proposed patch was actually
simpler than what I ended up with)

Another pointy-hat problem: PACKAGE_VERSION macro is set in
CFLAGS, it shouldn't be in bsdtar_platform.h.
Thanks to: Andrey Chernov for looking over my shoulder so much lately.

Minor portability fixes. bsdtar now compiles on Win32/cygwin, though
it doesn't yet seem to work entirely correctly.

Add --totals option. Unlike gtar, this reports the bytes actually
written to the archive, not the ones written to the compressor.
For uncompressed archives, these numbers are the same, of course.

Linux port corrections:
* Add a more reasonable default device for linux
* Add an autoconf check for the FNM_LEADING_DIR extension
(which isn't yet obeyed in the code)

Add "make distfile" capabilities to bsdtar, including informational
COPYING file and some conditional compilation cleanups.

Ensure that there's always a space between the user name and
group name in -tv output format.

GNU and POSIX disagree about -o and -l.

For -l, upset everyone by breaking it. Specifically, -l now produces
a lengthy error message that suggests --check-links (POSIX -l) or
--one-file-system (GNU -l) instead. However, if POSIXLY_CORRECT is set,
use the POSIX interpretation.

For -o, please everyone by making it work both ways:
* -xo uses POSIX behavior
* -co uses "almost GNU" behavior (as close as we can get until
libarchive implements a true V7 tar format)

Correct the explanation of the -X option.
Thanks to: Pav Lucistnik

Improve portability to FreeBSD 4.
Thanks to: Barry Bouwsma

Fix comment.

Thanks to: Johan Karlsson

Add a --version option to bsdtar that prints the versions of
both bsdtar and libarchive. Of course, this requires that bsdtar
have a version number. Let's call this 1.00, shall we? ;-)

A bunch of style and security fixes (error checking return values, etc),
mostly from: Tim J Robbins

Of course, I meant POSIX.1-1996, not 1997.
Thanks to: Andrey Chernov

A bunch of stuff from Christoph Mellon:
* Whitespace fixes
* Check some malloc calls
* Simplify long_help formatting
* Spell "LINUX" -> "linux"
* A few other miscellaneous style improvements

My bad: /dev/sa0, no 'r'

Thanks (and many apologies) to: Cristoph Mallon

Guard against argv[0] being NULL.

Thanks to: Tim J Robbins

Fix some misspellings, document the TAPE environment
variable and the default tape device.

Remove unused user_uname variable.
Add range-checking to argument of -b.

Thanks to: Tim J Robbins

Make bsdtar the default system tar. This makes /usr/bin/tar a symlink
pointing to /usr/bin/bsdtar by default. To make it point to /usr/bin/gtar,
you can define WITH_GTAR.

Validate -o usage. Strictly speaking, -o only makes sense with -x,
of course, but I make an effort to accomodate GNU tar scripts that
use -o with -c (with a meaning that totally contradicts SUSv2) by
only issuing a benign warning message in that case.

Make the day/month ordering dependent on the current locale by
testing the locale at program startup and setting a flag, then
using that flag to determine appropriate strftime() arguments.

Mimic ls(1) by putting an extra space before the year in old dates

Fix tab/space screwup in long_help()

Fixed cross-references in SEE ALSO.
Emininated double space and hard sentence breaks.

Pass the pointy hat, please:
All of --help should go to stdout, not some to stdout and some to stderr.

Mechanically kill hard sentence breaks.

Add yet another synonym for -n.

Update -p:
* Don't change the umask; the library now ignores the umask if
you set EXTRACT_PERM
* Set the EXTRACT_ACL and EXTRACT_FFLAGS bits (used to be
controlled by EXTRACT_PERM).

Warn about stripping leading '/' when creating archives.

Augment the -T handling:
* Add --null option (sort #defines here)
* Add process_lines function to util.c that reads newline-terminated
or null-terminated lines (with self-sizing buffers, etc) and iteratively
invokes a provided function. Use this to dramatically simplify:
-T handling for -c, --exclude-from-file, and --include-from-file.
* Add -T handling to -x (via include_from_file)

Hopefully, this will fix the openoffice port and a couple of
others that rely on -T and --null.

Fix "@-" which has apparently been broken for some time. <sigh>
While I'm here, add in a lot more error-checking around append_archive.

Don't abort immediately on directory change errors.
Instead, display a warning, clean up, and let main() return the error.
In particular, this means that chdir() problems won't leave broken
archives, though they will prompt an error exit value.

Document the -W convention for accessing long options.
Also correct an old error: there was no tar command in Sixth Edition.

Rename C=dir to -C dir, which is what people expect.
This requires some non-trivial surgery to the options parsing.

While here, let people who only have getopt() access long options
through the -W longopt=value convention.

Update some internal comments about the --no-same-permissions option.
Thanks to: Kris Kennaway for doing some gtar research for me.

As near as I can tell, --no-same-permissions is a no-op in
gtar, so that makes it easy to implement.

Required by: audio/timidity port
Thanks to: Kris Kennaway

Oops. bsdtar's old -X option didn't take an argument; the new
(gtar-compatible) one does require an argument.

Add gtar-compatible -X/--exclude-from

Fix build.

Rename -X to --one-file-system, as GNU tar uses -X for
something else. I would really like a short option for
this, but all of the obvious ones conflict with something else.

Clean up usage message(s):
* Usage goes to stderr, not stdout
* Use correct argument markup
* bsdtar --help no longer exits with an error return code
* ensure that the word "bsdtar" appears in the first
line output from "bsdtar --help" (even if the program is
invoked as "tar")

In particular, scripts can now test for the presence of bsdtar.
For example, in /bin/sh:

if (tar --help 2>&1 | grep bsdtar >/dev/null 2>&1) then \
echo bsdtar; else echo not bsdtar; fi

Oops: gtar's --dereference should be a synonym for bsdtar's -L, not -H.

Pointed out by: devel/nspr port
Thanks to: Kris Kennaway

Set program exit value to 1 if there are any of various errors when
creating an archive.

Pointed out by: Failure to complain when building certain broken packages
(Thanks again to Kris Kennaway for finding this!)

Fix the symlink-detection code. Don't squawk if we're just replacing
an existing symlink (as might happen if you extract an archive twice).
Also, if we remove the offending link, then we've removed the problem
and can safely go forward with the extraction.

Pointed out by: print/adobe-cmaps port (whose distfile has
duplicate entries for the same symlinks)
Thanks to: Kris Kennaway (for using ports as a testbed for bsdtar)

The --include='pattern' option is the natural counterpart to
--exclude='pattern'.

I should have added this a long time ago, since it's so useful for testing.
In particular, it allows me to select a few entries from a troublesome
archive so that I can easily focus my debugging efforts:
bsdtar -czf new.tgz --include='*foo*' @old.tgz

Ooops! Previous commit added an over-zealous error check.
It is, in fact, perfectly legal to not specify a compression when
writing an archive. <sigh>

If -b is specified, then force full padding for the last block.

Per style(9), don't use double spaces in expressions.

Pointed out by: njl, des.

Pointy hat: sticky bit 't' or 'T' is shown over IXOTH bit, not IXGRP. <sigh>

Improve help handling:
* --help produces long help message on systems with getopt_long
* -h with no other options also produces long help message
(If a mode is specified, -h has its usual meaning.)

Both "foo/" and "foo" should match "foo/bar", so strip
trailing '/' characters on include patterns for extraction.

Pointed out by: games/quakeforge port (thanks to Kris!)

Correction: -h should be a synonym for -L, not -H.
(as specified by Linux Standards Base, GNU tar, and 4.4BSD documentation)

Permit (but ignore) -Z with -x or -t, now that libarchive
recognizes and decompresses .Z format.

Minor corrections to error handling and user notifications:
* Move format/compression reporting to end of output, since
we don't always know the input format until then.
* Set bsdtar exit value to 1 if any file could not be restored.
* Generate gtar-style warning when stripping leading '/' characters.
* Warn when removing symlinks.

Provide framework for exiting with non-zero value on non-critical errors.

Add --no-same-owner as a synonym for -o.
Note that bsdtar's -o (which follows SUSv2) is not the same as GNU tar's -o.
In GNU tar, -o and --no-same-owner are not synonyms.

Pointed out by: Kris Kennaway (required by xpenguins port)

Add --dereference as a synonym for -H

Pointed out by: Kris Kennaway (unbreaks nspr port build)

Default is to restore timestamps, even when operating as root. <sigh>

Fix a segfault when looking up a non-existent group.

Accept (but ignore) GNU tar's -B option. bsdtar never
needs it, but it does no harm for people to specify it.

Set errno to 0 before calling getpwuid/getgrgid, so that error
messages will be displayed only when there is, in fact, an error.

Speed up safe_fprintf by copying to a temp buffer, then writing
blocks at a time, rather than using putc().

Propagate struct bsdtar down into a lot more places.
Move global progname variable into struct bsdtar.

Refactor name caching to use a common piece of code for uname_cache and
gname_cache. Cache negative lookups to dramatically improve performance
building archives containing nonexistent uid/gid.

Make symlink protection a bit more useful:
* Remove terminal symlinks so they can be replaced.
* If -U, remove intermediate symlinks as well
* Otherwise, refuse the extraction

Update manpage to track new security features.

Add --unlink and --unlink-first synonyms for -U
Make indentation of tar_longopts array more consistent.

Contrariwise, Linux sys/stat.h doesn't define mode_t and friends,
even though POSIX requires it. <sigh>

Include local copies of fts.c and fts.h to be used ONLY on non-FreeBSD
systems. The fts.h here is an exact copy of include/fts.h (except for
an initial explanatory comment and the revision tags, of course). The
fts.c here is slightly modified from lib/libc/gen/fts.c so it can
compile correctly on non-FreeBSD systems.

For portability, don't rely on libc's strmode(3). <sigh>

Correct copy/paste error in Linux nodump support.
Thanks to: Juergen Lock for his continuing patience
while I botch his patches.

Style: Use single tab after #define

More of Juergen Lock's patches for Linux.
(fflags support on Linux, nanosecond timestamp portability, enable
64-bit file offsets)

A security issue: An archive containing a symlink to another
directory, then a file with that symlink as a prefix can drop a file
outside of the current directory, which can be a security hole.

Plug this hole by refusing to extract files if a prefix of the
pathname is a symlink. The -P option disables this check.

-U is now implemented. (It's been recognized by bsdtar for a while;
now libarchive actually supports the underlying request.)

Refuse to extract entries with '..' in pathname.

Pointed out by: David Schultz

Remove information about specific formats; replace with a
reference to libarchive-formats(5).

Update write.c to follow recent API changes to libarchive.

Also, bump up the default copy buffer from 8k to 64k
to improve performance.

Fix username/groupname cache so it returns a name that
was just looked up by get{pw,gr}id(). Otherwise, it
returns a NULL name unless it's already in the cache.

Credit to Juergen Lock.

Eliminate a redundant call to archive_entry_set_pathname()

Allow brave souls to install bsdtar as the
default "tar" command by defining WITH_BSDTAR.

Note: "gtar" and "bsdtar" commands both exist regardless.

--help is only supported if you have getopt_long().
This allows bsdtar to compile on FreeBSD 4.x.

As suggested by Julian Elischer, use a self-sizing hash
table for the hardlink cache. This dramatically improves
performance when archiving millions of hardlinked files.

While I'm here, clean up some style bugs (per Bruce Evans)
and clarify some comments.

A simple cache of uid->uname lookups and gid->gname lookups eliminates
almost 1/2 of the CPU time required to create an uncompressed archive
and makes a noticable reduction in wallclock time.

When all the links to a file have been dumped, don't forget
to free() the memory used to store the filename.

Eliminate a lot of malloc/free calls by using
a stack-allocated buffer for safe_fprintf formatting.
Only if the result is too large do we resort to malloc.

Manpage changes suggested by Irina Liakh.

Also, add -h as a synonym for -H, for Linux Standards Base compliance.

Style fixes

Pointed out by: bde

Add DPADD to Makefile (thanks to Andrey for pointing this out).

Generally clean up Makefile, remove dmalloc references from source
(there are better ways to do this than pollute the FreeBSD src tree).

Allow arguments to appear in any order.

While I'm here, add a couple of extra sanity-checks to
the argument parsing (reject -j -z, for instance) and
update the docs a bit.

Requested by: most everyone ;-)

Document the newly-implemented -T option.

Add support for -T option on create.
(Required by pkg_create.)

Correct signature in alternative stub implementation of setup_acls()

Archive both access and default ACLs.

Initial commit for bsdtar.