MFH r324148:
Sync libarchive with vendor.

Relevant vendor changes:
PR #905: Support for Zstandard read and write filters
PR #922: Avoid overflow when reading corrupt cpio archive
Issue #935: heap-based buffer overflow in xml_data (CVE-2017-14166)
OSS-Fuzz 2936: Place a limit on the mtree line length
OSS-Fuzz 2394: Ensure that the ZIP AES extension header is large enough
OSS-Fuzz 573: Read off-by-one error in RAR archives (CVE-2017-14502)

Security: CVE-2017-14166, CVE-2017-14502

MFC r315636,315876,316095:
Sync libarchive with vendor

Vendor changes/bugfixes (FreeBSD-related):
r315636:
PR 867 (bsdcpio): show numeric uid/gid when names are not found
PR 870 (seekable zip): accept files with valid ZIP64 EOCD headers
PR 880 (pax): Fix handling of "size" pax header keyword
PR 887 (crypto): Discard 3072 bytes instead of 1024 of first keystream
OSS-Fuzz issue 806 (mtree): rework mtree_atol10 integer parser
Break ACL read/write code into platform-specific source files

r315876:
Store extended attributes with extattr_set_link() if no fd is provided
Add extended attribute tests to libarchive and bsdtar
Fix tar's test_option_acls
Support the UF_HIDDEN file flag

r316095:
Constify variables in several places
Unify platform ACL code in a single source file
Fix unused variable if compiling on FreeBSD without NFSv4 ACL support

MFC r314571:
Update libarchive to version 3.3.1 (and sync with latest vendor dist)

Notable vendor changes:
PR #501: improvements in ACL path handling
PR #724: fix hang when reading malformed cpio files
PR #864: fix out of bounds read with malformed GNU tar archives
Documentation, style, test suite improvements and typo fixes.

New options to bsdtar that enable or disable reading and/or writing of:
Access Control Lists (--acls, --no-acls)
Extended file flags (--fflags, --no-fflags)
Extended attributes (--xattrs, --no-xattrs)
Mac OS X metadata (Mac OS X only) (--mac-metadata, --no-mac-metadata)

MFC r302075:

Update libarchive to 3.2.1 (bugfix and security fix release)

List of vendor fixes:
- fix exploitable heap overflow vulnerability in Rar decompression
(vendor issue 719, CVE-2016-4302, TALOS-2016-0154)
- fix exploitable stack based buffer overflow vulnebarility in mtree
parse_device functionality (vendor PR 715, CVE-2016-4301, TALOS-2016-0153)
- fix exploitable heap overflow vulnerability in 7-zip read_SubStreamsInfo
(vendor issue 718, CVE-2016-4300, TALOS-2016-152)
- fix integer overflow when computing location of volume descriptor
(vendor issue 717)
- fix buffer overflow when reading a crafred rar archive (vendor issue 521)
- fix possible buffer overflow when reading ISO9660 archives on machines
where sizeof(int) < sizeof(size_t) (vendor issue 711)
- tar and cpio should fail if an input file named on the command line is
missing (vendor issue 708)
- fix incorrect writing of gnutar filenames that are exactly 512 bytes
long (vendor issue 682)
- allow tests to be run from paths that are equal or longer than 128
characters (vendor issue 657)
- add memory allocation errors in archive_entry_xattr.c (vendor PR 603)
- remove dead code in archive_entry_xattr_add_entry() (vendor PR 716)
- fix broken decryption of ZIP files (vendor issue 553)
- manpage style, typo and description fixes

Post-3.2.1 vendor fixes:
- fix typo in cpio version reporting (Vendor PR 725, 726)
- fix argument range of ctype functions in libarchive_fe/passphrase.c
- fix ctype use and avoid empty loop bodies in WARC reader

Security: CVE-2016-4300, CVE-2016-4301, CVE-2016-4302

MFC r299529,r299540,r299576,r299896:

r299529,r299540:
Update libarchive to 3.2.0

New features:
- new bsdcat command-line utility
- LZ4 compression (in src only via external utility from ports)
- Warc format support
- 'Raw' format writer
- Zip: Support archives >4GB, entries >4GB
- Zip: Support encrypting and decrypting entries
- Zip: Support experimental streaming extension
- Identify encrypted entries in several formats
- New --clear-nochange-flags option to bsdtar tries to remove noschg and
similar flags before deleting files
- New --ignore-zeros option to bsdtar to handle concatenated tar archives
- Use multi-threaded LZMA decompression if liblzma supports it
- Expose version info for libraries used by libarchive

r299576,r299896:
Fix broken cpio behavior.

Relnotes: yes

MFC r291329:

Remove redundant DPSRCS which were already in SRCS.

MFC r289195:

Integrate the tests from lib/libarchive, usr.bin/cpio, and usr.bin/tar in to
the FreeBSD test suite

functional_test.sh was ported from bin/sh/tests/functional_test.sh, as a
small wrapper around libarchive_test, bsdcpio_test, and bsdtar_test provided
by upstream.

A handful of testcases in lib/libarchive/tests have been disabled as they
were failing when run with kyua test (see BROKEN_TESTS in
lib/libarchive/tests/Makefile)

As a sidenote: this removes the check/test targets from the Makefiles as they
don't match the pattern used in the rest of the FreeBSD test suite.

Sponsored by: EMC / Isilon Storage Division

Conflicts:
lib/libarchive/test
usr.bin/cpio/test

Integrate the tests from lib/libarchive, usr.bin/cpio, and usr.bin/tar in to
the FreeBSD test suite

functional_test.sh was ported from bin/sh/tests/functional_test.sh, as a
small wrapper around libarchive_test, bsdcpio_test, and bsdtar_test provided
by upstream.

A handful of testcases in lib/libarchive/tests have been disabled as they
were failing when run with kyua test (see BROKEN_TESTS in
lib/libarchive/tests/Makefile)

As a sidenote: this removes the check/test targets from the Makefiles as they
don't match the pattern used in the rest of the FreeBSD test suite.

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Revert r289133; retry the merge

Integrate the rest of the pieces from libarchive into the
FreeBSD test suite (cpio, tar)

Re-branch because apparently resyncing from head has svn issues with missing
revisions from ^/user/ngie/more-tests...

#idontknowwhatevenanymore #howilearnedtogiveupsvnandacceptmydvcsoverlords

Convert to usr.bin/ to LIBADD
Reduce overlinking

NO_MAN= has been deprecated in favor of MAN= for some time, go ahead
and finish the job. ncurses is now the only Makefile in the tree that
uses it since it wasn't a simple mechanical change, and will be
addressed in a future commit.

MFV r248590,248594:
Update libarchive to 3.1.2

Some of new features:
- support for lrzip and grzip compression
- support for writing tar v7 format
- b64encode and uuencode filters
- support for __MACOSX directory in Zip archives
- support for lzop compresion (external utility)

Update libarchive to 3.0.4

Update libarchive to 3.0.3

Some of new features:
- New readers: RAR, LHA/LZH, CAB reader, 7-Zip
- New writers: ISO9660, XAR
- Improvements to many formats, especially including ISO9660 and Zip
- Stackable write filters to write, e.g., tar.gz.uu in a single pass
- Exploit seekable input; new "seekable" Zip reader can exploit the Zip
Central Directory when it's available; the old "streamable" Zip reader
is still fully supported for cases where seeking is not possible.

Full release notes available at:
https://github.com/libarchive/libarchive/wiki/ReleaseNotes

Use contrib sources for building libarchive, tar and cpio.
Make "make test" fully operational.

MFC after: 2 weeks

Sync bsdcpio with vendor branch release/2.8:

Revision 3770:
Merge r3768 from trunk: Fix typo in dev/ino verification for cpio formats.

Obtained from: http://code.google.com/p/libarchive
MFC after: 2 weeks

Append to CPIO_SRCS instead of using TAR_SRCS,
the variable name is misleading

MFC after: 2 weeks

Update bsdcpio to 2.8.4
Use common code from lib/libarchive/libarchive_fe

Approved by: kientzle
MFC after: 2 weeks

Enable liblzma support in libarchive
Adjust dependencies for programs using libarchive
Add xz and linkage against liblzma to rescue system

Approved by: kientzle, delphij (mentor)
MFC after: 2 weeks

Build usr.bin/ with WARNS=6 by default.

Also add some missing $FreeBSD$ to keep svn happy.

This addresses some issues with my earlier -R fix that
were pointed out by Brooks Davis and Alexey Dokuchaev:
* It now tries to lookup arguments as names first, then tries
to parse them as numbers. In particular, this makes the
behavior consistent with POSIX conventions when usernames
consist entirely of digits.
* It now uses strtoul() for the numeric parsing.

Finally, I've included an update to the test harness
to exercise the new numeric cases for -R.

Approved by: re (kib)

Merge from libarchive.googlecode.com:
* Lots of new tests.
* New -n / --numeric-uid-gid option
* More sanity-checking of arguments
* Various Windows portability improvements
* Sync up version number to 2.7.0

Format the output of -itv for real. In particular:
* Lookup uname/gname if not provided by the archive (I copied the
uname/gname lookup cache from bsdtar)
* Format device number instead of size for device nodes
* Format date.

There's still a few improvements that I could copy from
bsdtar, especially the locale-aware safe_fprintf() code
and the locale-aware setup for day_first date formatting.
(And, of course, I need to think through a clean way to
push this stuff down into libarchive.)

Thanks to Peter Wemm for reminding me of this overlooked TODO item.

MfP4: Verify correct interaction with umask: Add another
file with different permissions and set a non-zero umask
during the actual copy tests. The extra entry increases
the size of the test archives of course, so adjust the
expected sizes.

Update the total archive byte counters when writing entries to disk using
archive_write_disk.
Update cpio to use this to emit block counts in -p mode.
Update cpio tests to verify these block counts.

straighten out the "clean" target

svn:ignore built files

cpio -v emits a line for every item copied.

Update the passthrough_dotdot test to reproduce a
problem reported by Kris Kennaway.

PR: bin/124924

Test for proper handling of "cpio -p .."

PR: bin/124924

Test for a bug reported by Bernd Walter: In passthrough mode,
copying "dir/file" and then copying "dir" results in
"File on disk is not older; skipping" for the "dir" because
it was implicitly created by "dir/file." Among other sins,
this means that "dir" ends up with the wrong permissions
and ownership.

This is actually a libarchive bug; fix is forthcoming.

Comment a couple of places where bsdcpio and gcpio 2.9 disagree.
The number of blocks read from ustar archives is just an implementation
difference. The failure of bsdcpio to emit a block count to stderr
in -p mode is a real bug in bsdcpio.

The newc-format verification is now a little smarter about
following the archive structure. In particular, it no longer
crashes if you run it against GNU cpio 2.9 (although it does
still complain a lot more than it should).

MfP4: test improvements, mostly for portability.

Initial commit of bsdcpio 0.9.11b.

A new implementation of cpio that uses libarchive as it's back-end
archiving/dearchiving infrastructure. Includes test harness;
"make check" in the bsdcpio directory to build and run the test
harness.