MFC r323683:

MFV r323678: file 5.32

MFC r323278: Fix an incorrectly used conditional causing buffer overflow.

Reported by: Thomas Jarosch of Intra2net AG
Reviewed by: emaste, jhb
Approved by: re (marius)
Security: CVE-2017-1000249

MFC r308420: MFV r308392: file 5.29.

MFC r302221,302222:

MFV r302218: file 5.28.

Relnotes: yes

MFC r298192,299234,299238,299736:

file 5.27.

Revert r298920 until we have addressed all known regressions on head/ after
settle. file is now downgraded to 5.25.

PR: bin/209211
Requested by: Tomoaki AOKI
Relnotes: yes

MFC r298192: MFV r298178:

Update file to 5.26.

Relnotes: yes

MFC r287453,287454,288143:

file 5.25.

Relnotes: yes

MFC r284237,284277:

file 5.23.

Relnotes: yes

MFC r276577: MFV r276568:

Update file to 5.22.

MFC r275680:

Add fstyp(8). This utility, named after its SVR4 counterpart, detects
filesystems. It differs from file(1) in that it gives machine-parseable
output, it outputs filesystem labels, doesn't get confused by other
formats metadata, and runs in Capsicum sandbox.

Differential Revision: https://reviews.freebsd.org/D1255
Relnotes: yes
Sponsored by: The FreeBSD Foundation

MFC r275698: MFV r275696: file 5.21.

MFC r275666:

Fix multiple vulnerabilities in file(1) and libmagic(3).

Security: FreeBSD-SA-14:28.file
Security: CVE-2014-3710, CVE-2014-8116, CVE-2014-8117

MFC r267897: MFV r267843:

Update file/libmagic to 5.19.

Fix multiple vulnerabilities in file(1) and libmagic(3).

Security: FreeBSD-SA-14:16.file
Approved by: so

MFC r263289: Update NetBSD Foundation copyrights to 2-clause BSD

The NetBSD Foundation states "Third parties are encouraged to change the
license on any files which have a 4-clause license contributed to the
NetBSD Foundation to a 2-clause license."

This change removes clauses 3 and 4 from copyright / license blocks that
list The NetBSD Foundation as the only copyright holder.

Sponsored by: The FreeBSD Foundation

MFC r259724:

Apply upstream commit 33312fd828e59c78ae4ee30fd70d0ca109748cf0 (by
zoulasc) to contrib/file:

don't write a char to a pointer.

Hoist all the mergeinfo up to the root in preparation for enforcing merges
to the root only. All MFC's were rerecorded to the root.

Going forward, if an MFC includes mergeinfo, it will need to be made to
the root and committed from the root. Merges with --ignore-ancestry
or diff | patch can go anywhere.

The mergeinfo in HEAD is in a bad state from years of neglect and manual
tampering and this was branched into 10.x. This confuses the coalescing
code and prevents it from doing its job.

Approved by: re (gjb, implicit)

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Add support for Lua 5.2.

Submitted by: skreuzer

Correct a regression introduced during the import of file(1) 5.11.

Magic tests containing "search" or "regex" directives were incorrectly
compiled by "mkmagic" and were effectively ignored. This caused troff
files (for example) to be detected as simply "ASCII text" instead of
as "troff or preprocessor input, ASCII text".

PR: bin/170415
Approved by: consensus on developers@
MFC after: 3 days

Update file(1) to version 5.11.

In contrib/file/softmagic.c, fix a potential format string security
problem. (This fix has already been applied upstream, but we do not
have the latest version of file in the tree at the moment.)

MFC after: 1 week

With retirement of cpumask_t and usage of cpuset_t for representing a
mask of CPUs, pc_other_cpus and pc_cpumask become highly inefficient.

Remove them and replace their usage with custom pc_cpuid magic (as,
atm, pc_cpumask can be easilly represented by (1 << pc_cpuid) and
pc_other_cpus by (all_cpus & ~(1 << pc_cpuid))).

This change is not targeted for MFC because of struct pcpu members
removal and dependency by cpumask_t retirement.

MD review by: marcel, marius, alc
Tested by: pluknet
MD testing by: marcel, marius, gonzo, andreast

etire the cpumask_t type and replace it with cpuset_t usage.

This is intended to fix the bug where cpu mask objects are
capped to 32. MAXCPU, then, can now arbitrarely bumped to whatever
value. Anyway, as long as several structures in the kernel are
statically allocated and sized as MAXCPU, it is suggested to keep it
as low as possible for the time being.

Technical notes on this commit itself:
- More functions to handle with cpuset_t objects are introduced.
The most notable are cpusetobj_ffs() (which calculates a ffs(3)
for a cpuset_t object), cpusetobj_strprint() (which prepares a string
representing a cpuset_t object) and cpusetobj_strscan() (which
creates a valid cpuset_t starting from a string representation).
- pc_cpumask and pc_other_cpus are target to be removed soon.
With the moving from cpumask_t to cpuset_t they are now inefficient
and not really useful. Anyway, for the time being, please note that
access to pcpu datas is protected by sched_pin() in order to avoid
migrating the CPU while reading more than one (possible) word
- Please note that size of cpuset_t objects may differ between kernel
and userland. While this is not directly related to the patch itself,
it is good to understand that concept and possibly use the patch
as a reference on how to deal with cpuset_t objects in userland, when
accessing kernland members.
- KTR_CPUMASK is changed and now is represented through a string, to be
set as the example reported in NOTES.

Please additively note that no MAXCPU is bumped in this patch, but
private testing has been done until to MAXCPU=128 on a real 8x8x2(htt)
machine (amd64).

Please note that the FreeBSD version is not yet bumped because of
the upcoming pcpu changes. However, note that this patch is not
targeted for MFC.

People to thank for the time spent on this patch:
- sbruno, pluknet and Nicholas Esborn (nick AT desert DOT net) tested
several revision of the patches and really helped in improving
stability of this work.
- marius fixed several bugs in the sparc64 implementation and reviewed
patches related to ktr.
- jeff and jhb discussed the basic approach followed.
- kib and marcel made targeted review on some specific part of the
patch.
- marius, art, nwhitehorn and andreast reviewed MD specific part of
the patch.
- marius, andreast, gonzo, nwhitehorn and jceel tested MD specific
implementations of the patch.
- Other people have made contributions on other patches that have been
already committed and have been listed separately.

Companies that should be mentioned for having participated at several
degrees:
- Yahoo! for having offered the machines used for testing on big
count of CPUs.
- The FreeBSD Foundation for having sponsored my devsummit attendance,
which has been instrumental.
- Sandvine for having offered offices and infrastructure during
development.

(I really hope I didn't forget anyone, if it happened I apologize in
advance).

Add the ability for GDB to printout the thread name along with other
thread specific informations.

In order to do that, and in order to avoid KBI breakage with existing
infrastructure the following semantic is implemented:
- For live programs, a new member to the PT_LWPINFO is added (pl_tdname)
- For cores, a new ELF note is added (NT_THRMISC) that can be used for
storing thread specific, miscellaneous, informations. Right now it is
just popluated with a thread name.

GDB, then, retrieves the correct informations from the corefile via the
BFD interface, as it groks the ELF notes and create appropriate
pseudo-sections.

Sponsored by: Sandvine Incorporated
Tested by: gianni
Discussed with: dim, kan, kib
MFC after: 2 weeks

Properly warp around more code under COMPILE_ONLY to recover from build
errors.

Merger of the quota64 project into head.

This joint work of Dag-Erling Smørgrav and myself updates the
FFS quota system to support both traditional 32-bit and new 64-bit
quotas (for those of you who want to put 2+Tb quotas on your users).

By default quotas are not compiled into the kernel. To include them
in your kernel configuration you need to specify:

options QUOTA # Enable FFS quotas

If you are already running with the current 32-bit quotas, they
should continue to work just as they have in the past. If you
wish to convert to using 64-bit quotas, use `quotacheck -c 64';
if you wish to revert from 64-bit quotas back to 32-bit quotas,
use `quotacheck -c 32'.

There is a new library of functions to simplify the use of the
quota system, do `man quotafile' for details. If your application
is currently using the quotactl(2), it is highly recommended that
you convert your application to use the quotafile interface.
Note that existing binaries will continue to work.

Special thanks to John Kozubik of rsync.net for getting me
interested in pursuing 64-bit quota support and for funding
part of my development time on this project.

Merge vendor/file/dist@192348, bringing FILE 5.03 to 8-CURRENT.

Security: CVE-2009-1515

This belongs in //svn.freebsd.org/base/vendor/file/dist now.

Merge vendor/file/dist@191739, bringing FILE 5.00 to 8-CURRENT.

Add an additional COMPILE_ONLY check.

Record that base/vendor/file/dist@186675 was merged.
Merge base/vendor/file/dist@186675@186690, bringing FILE 4.26 to 8-CURRENT.

This commit was generated by cvs2svn to compensate for changes in r175898,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r175882,
which included commits to RCS files with non-trunk default branches.

Update for the 4.23 import.

This commit was generated by cvs2svn to compensate for changes in r175296,
which included commits to RCS files with non-trunk default branches.

Update for the 'file' 4.21 import.

This commit was generated by cvs2svn to compensate for changes in r169962,
which included commits to RCS files with non-trunk default branches.

Update for the 'file' 4.19 import.

This commit was generated by cvs2svn to compensate for changes in r169942,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r159825,
which included commits to RCS files with non-trunk default branches.

Update for the 'file' 4.17 import.

This commit was generated by cvs2svn to compensate for changes in r159764,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r139368,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r133359,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r111658,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r110949,
which included commits to RCS files with non-trunk default branches.

Make current.

This commit was generated by cvs2svn to compensate for changes in r103373,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r84685,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r80588,
which included commits to RCS files with non-trunk default branches.

Update for file 3.35.

This commit was generated by cvs2svn to compensate for changes in r75937,
which included commits to RCS files with non-trunk default branches.

Update for file 3.34.

This commit was generated by cvs2svn to compensate for changes in r74784,
which included commits to RCS files with non-trunk default branches.

Information on how I did the import and upgrade instructions.

This commit was generated by cvs2svn to compensate for changes in r69216,
which included commits to RCS files with non-trunk default branches.

This commit was generated by cvs2svn to compensate for changes in r68349,
which included commits to RCS files with non-trunk default branches.