libmagic: Use HOST_CC when compiling hostprog used by build

The "mkmagic" program should be built with the host compiler.

Only use BTOOLSPATH if not building for host

Obtained from: Juniper Networks, Inc.
Reviewed by: sjg
Differential Revision: https://reviews.freebsd.org/D44539

MFV: file 5.45.

MFC after: 3 days

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

Install working pkgconfig .pc files for compat libraries

The default ones are install them to /usr/libdata/pkgconfig, and we can't
use this path for compat libraries, so we use /usr/lib<suffix>/pkgconfigi here.

Test Plan: grep -rn libdir= ./usr/lib32/pkgconfig/*.pc
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D34939

Convert libs with pc files to use PCFILES

Now the .pc ends up in the correct package (-dev)

Reviewed by: bapt, emaste
Differential Revision: https://reviews.freebsd.org/D29172
MFC after: 2 weeks

Fix installation of magic file.

Reported by: lwhsu
MFC after: 2 weeks
X-MFC-with: r362258

MFV r362254: file 5.39.

MFC after: 2 weeks

MFV r357712: file 5.38.

MFC after: 2 weeks

Tweaks for DIRDEPS_BUILD

libmagic only depend on mkmagic if not DIRDEPS_BUILD

libpmc fix -I for libpmcstat

local.dirdeps.mk be even more careful about adding gnu/lib/csu to DIRDEPS

Reviewed by: bdrewery
Differential Revision: https://reviews.freebsd.org/D22872

Update dirdeps.mk and gendirdeps.mk

The env space consumed by exporting all libc's .meta files
left little room for command line,
so unexport when done.

Update dirdeps.mk to latest and add
dirdeps-targets.mk to simplify/update targets/Makefile

Makefile changes to go with Makefile.depend changes in D22494

Reviewed by: bdrewery
MFC after: 1 week
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D22495

MFV r354582: file 5.37.

MFC after: 3 days

Fix .depend files to work for build tools.

This is somewhat of a follow-up to r335746.

MFC after: 2 weeks
Sponsored by: DellEMC

Don't use CCACHE for linking.

MFC after: 2 weeks
Sponsored by: Dell EMC

Style fixup:

A non-functional commit to make adjustment to an aesthetically
unpleasing long line.

Fix build post r333919.

This commit results in an aesthetically unpleasing long line
which will be fixed next commit.

LIB32: Avoid linking in unneeded (and invalid lib32) libz for libmagic build tool.

Usually this is just ignored:
/usr/bin/ld: skipping incompatible /scratch/obj/root/git/freebsd/amd64.amd64/obj-lib32/lib/libz/libz.so when searching for -lz
/usr/bin/ld: skipping incompatible /scratch/obj/root/git/freebsd/amd64.amd64/obj-lib32/lib/libz/libz.a when searching for -lz

However some combination of newer toolchains simply fail here instead:
/usr/bin/ld: error: /home/dstolfa/obj/home/dstolfa/cadets/amd64.amd64/obj-lib32/lib/libz/libz.so is incompatible with /usr/lib/crt1.o

Libz is not needed for mkmagic so just exclude it.

Reported by: Domagoj Stolfa <domagoj.stolfa@gmail.com>
MFC after: 2 weeks
Sponsored by: Dell EMC

Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones

This simplifies pathing in make/displayed output

MFC after: 3 weeks
Sponsored by: Dell EMC Isilon

Fix build: explicitly reference the generated magic.h as dependency
of mkmagic and point its location.

Reported by: jenkins
Pointy hat to: delphij
MFC after: 2 weeks
X-MFC-With: 302221
Approved by: re (kib)

MFV r302218: file 5.28.

Relnotes: yes
MFC after: 2 weeks
Approved by: re (gjb)

DIRDEPS_BUILD: Run the staged bootstrap-tools version of build-tools.

This avoids running target binaries.

Sponsored by: EMC / Isilon Storage Division

MFV r298178:

Update file to 5.26.

MFC after: 2 weeks
Relnotes: yes

MFH

Sponsored by: The FreeBSD Foundation

META_MODE: Don't rebuild build-tools targets during normal build.

This avoids 'build command changed' due to CFLAGS/CC changes during the
normal build. Without this the build-tools targets end up rebuilding
for the *target* rather than keeping the native versions built in
build-tools.

Sponsored by: EMC / Isilon Storage Division

First pass through library packaging.

Sponsored by: The FreeBSD Foundation

Convert libraries to use LIBADD
While here reduce a bit overlinking

MFV r267843: update file/libmagic to 5.19.

MFC after: 2 weeks

Sort .ALLSRC before concatenating files together. This makes sure that the
file are always built the same.

(Note that Header and Localstuff must appear first and in that order, the
sorting does not affect as a coincident effect).

Submitted by: sjg
MFC after: 3 days

r235638 is not the clean way to add support for building on ancient FreeBSD
versions. Instead use Imp's good work on "legacy" and follow the outcome
of the previous TRB discussions on this topic.

Now use the libc getline() if it exists, and only where it doesn't
create a bootstraping version.

Correct a regression introduced during the import of file(1) 5.11.

Magic tests containing "search" or "regex" directives were incorrectly
compiled by "mkmagic" and were effectively ignored. This caused troff
files (for example) to be detected as simply "ASCII text" instead of
as "troff or preprocessor input, ASCII text".

PR: bin/170415
Approved by: consensus on developers@
MFC after: 3 days

Don't depend on getline being on the build machine. That's
not the case for FreeBSD 7.x machines.

Update file(1) to version 5.11.

Use CFLAGS and LDFLAGS when compiling mkmagic. This ensures that all the
correct flags are being used (esp. NO_WERROR wrt clang).

Submitted by: Dimitry Andric <dimitry at andric.com>
Reviewed by: ru

With r208341 in place WARNS can be increased.

Build lib/ with WARNS=6 by default.

Similar to libexec/, do the same with lib/. Make WARNS=6 the norm and
lower it when needed.

I'm setting WARNS?=0 for secure/. It seems secure/ includes the
Makefile.inc provided by lib/. I'm not going to touch that directory.
Most of the code there is contributed anyway.

Bump the version of all non-symbol-versioned shared libraries in
preparation for 8.0-RELEASE. Add the previous version of those
libraries to ObsoleteFiles.inc and bump __FreeBSD_Version.

Reviewed by: kib
Approved by: re (rwatson)

Merge vendor/file/dist@191739, bringing FILE 5.00 to 8-CURRENT.

Don't make the MIME magic any longer.

Bump library versions in preparation for 7.0.

Ok'd by: kan

Update for the 'file' 4.17 import.

Add missing shared library interdependencies.

Bump the shared library version number of all libraries that have not
been bumped since RELENG_5.

Reviewed by: ru
Approved by: re (not needed for commit check but in principle...)

Clean up makefile.

- Remove all:. It's redundant, and ${LIB} in it is just a bug.
- Remove .ORDER:. *.mgc files can safely be built in parallel.
- Remove PITA. The mkmagic tool is smart to put the binary file
into the current directory (${.OBJDIR}) even if the source file
lives somewhere else, which is just what we need.

Install the header for libmagic.

PR: 73647
Submitted by: Uranus <uranus@it.muds.net>

uniq(1) is not an install tool, and using it was causing
"uniq: not found" during the installworld.

Spotted by: Roman Neuhauser

Don't try to programatically support running file(1) w/in /usr/obj or other
non-standard install path.

Agreed with: ru

Uncomment the real paths and remove the local debugging paths.

Noticed by: ru (who's faster than 'make world' on my stock src test machine)

Bmake the library containing and processing the magic.

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

Install working pkgconfig .pc files for compat libraries

The default ones are install them to /usr/libdata/pkgconfig, and we can't
use this path for compat libraries, so we use /usr/lib<suffix>/pkgconfigi here.

Test Plan: grep -rn libdir= ./usr/lib32/pkgconfig/*.pc
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D34939

Convert libs with pc files to use PCFILES

Now the .pc ends up in the correct package (-dev)

Reviewed by: bapt, emaste
Differential Revision: https://reviews.freebsd.org/D29172
MFC after: 2 weeks

Fix installation of magic file.

Reported by: lwhsu
MFC after: 2 weeks
X-MFC-with: r362258

Tweaks for DIRDEPS_BUILD

libmagic only depend on mkmagic if not DIRDEPS_BUILD

libpmc fix -I for libpmcstat

local.dirdeps.mk be even more careful about adding gnu/lib/csu to DIRDEPS

Reviewed by: bdrewery
Differential Revision: https://reviews.freebsd.org/D22872

Update dirdeps.mk and gendirdeps.mk

The env space consumed by exporting all libc's .meta files
left little room for command line,
so unexport when done.

Update dirdeps.mk to latest and add
dirdeps-targets.mk to simplify/update targets/Makefile

Makefile changes to go with Makefile.depend changes in D22494

Reviewed by: bdrewery
MFC after: 1 week
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D22495

Fix .depend files to work for build tools.

This is somewhat of a follow-up to r335746.

MFC after: 2 weeks
Sponsored by: DellEMC

Don't use CCACHE for linking.

MFC after: 2 weeks
Sponsored by: Dell EMC

Style fixup:

A non-functional commit to make adjustment to an aesthetically
unpleasing long line.

Fix build post r333919.

This commit results in an aesthetically unpleasing long line
which will be fixed next commit.

LIB32: Avoid linking in unneeded (and invalid lib32) libz for libmagic build tool.

Usually this is just ignored:
/usr/bin/ld: skipping incompatible /scratch/obj/root/git/freebsd/amd64.amd64/obj-lib32/lib/libz/libz.so when searching for -lz
/usr/bin/ld: skipping incompatible /scratch/obj/root/git/freebsd/amd64.amd64/obj-lib32/lib/libz/libz.a when searching for -lz

However some combination of newer toolchains simply fail here instead:
/usr/bin/ld: error: /home/dstolfa/obj/home/dstolfa/cadets/amd64.amd64/obj-lib32/lib/libz/libz.so is incompatible with /usr/lib/crt1.o

Libz is not needed for mkmagic so just exclude it.

Reported by: Domagoj Stolfa <domagoj.stolfa@gmail.com>
MFC after: 2 weeks
Sponsored by: Dell EMC

Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones

This simplifies pathing in make/displayed output

MFC after: 3 weeks
Sponsored by: Dell EMC Isilon

Fix build: explicitly reference the generated magic.h as dependency
of mkmagic and point its location.

Reported by: jenkins
Pointy hat to: delphij
MFC after: 2 weeks
X-MFC-With: 302221
Approved by: re (kib)

DIRDEPS_BUILD: Run the staged bootstrap-tools version of build-tools.

This avoids running target binaries.

Sponsored by: EMC / Isilon Storage Division

META_MODE: Don't rebuild build-tools targets during normal build.

This avoids 'build command changed' due to CFLAGS/CC changes during the
normal build. Without this the build-tools targets end up rebuilding
for the *target* rather than keeping the native versions built in
build-tools.

Sponsored by: EMC / Isilon Storage Division

First pass through library packaging.

Sponsored by: The FreeBSD Foundation

Convert libraries to use LIBADD
While here reduce a bit overlinking

Sort .ALLSRC before concatenating files together. This makes sure that the
file are always built the same.

(Note that Header and Localstuff must appear first and in that order, the
sorting does not affect as a coincident effect).

Submitted by: sjg
MFC after: 3 days

r235638 is not the clean way to add support for building on ancient FreeBSD
versions. Instead use Imp's good work on "legacy" and follow the outcome
of the previous TRB discussions on this topic.

Now use the libc getline() if it exists, and only where it doesn't
create a bootstraping version.

Correct a regression introduced during the import of file(1) 5.11.

Magic tests containing "search" or "regex" directives were incorrectly
compiled by "mkmagic" and were effectively ignored. This caused troff
files (for example) to be detected as simply "ASCII text" instead of
as "troff or preprocessor input, ASCII text".

PR: bin/170415
Approved by: consensus on developers@
MFC after: 3 days

Don't depend on getline being on the build machine. That's
not the case for FreeBSD 7.x machines.

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Use CFLAGS and LDFLAGS when compiling mkmagic. This ensures that all the
correct flags are being used (esp. NO_WERROR wrt clang).

Submitted by: Dimitry Andric <dimitry at andric.com>
Reviewed by: ru

With r208341 in place WARNS can be increased.

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

Build lib/ with WARNS=6 by default.

Similar to libexec/, do the same with lib/. Make WARNS=6 the norm and
lower it when needed.

I'm setting WARNS?=0 for secure/. It seems secure/ includes the
Makefile.inc provided by lib/. I'm not going to touch that directory.
Most of the code there is contributed anyway.

Bump the version of all non-symbol-versioned shared libraries in
preparation for 8.0-RELEASE. Add the previous version of those
libraries to ObsoleteFiles.inc and bump __FreeBSD_Version.

Reviewed by: kib
Approved by: re (rwatson)

Don't make the MIME magic any longer.

- Import the HEAD csup code which is the basis for the cvsmode work.

Bump library versions in preparation for 7.0.

Ok'd by: kan

Update for the 'file' 4.17 import.

Add missing shared library interdependencies.

Bump the shared library version number of all libraries that have not
been bumped since RELENG_5.

Reviewed by: ru
Approved by: re (not needed for commit check but in principle...)

Clean up makefile.

- Remove all:. It's redundant, and ${LIB} in it is just a bug.
- Remove .ORDER:. *.mgc files can safely be built in parallel.
- Remove PITA. The mkmagic tool is smart to put the binary file
into the current directory (${.OBJDIR}) even if the source file
lives somewhere else, which is just what we need.

Install the header for libmagic.

PR: 73647
Submitted by: Uranus <uranus@it.muds.net>

uniq(1) is not an install tool, and using it was causing
"uniq: not found" during the installworld.

Spotted by: Roman Neuhauser

Don't try to programatically support running file(1) w/in /usr/obj or other
non-standard install path.

Agreed with: ru

Uncomment the real paths and remove the local debugging paths.

Noticed by: ru (who's faster than 'make world' on my stock src test machine)

Bmake the library containing and processing the magic.