Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

pkg(7): use libmd for sha256 instead of openssl

OpenSSL 3.0 has deprecated the sha256 api, let's use libmd which has the
same API instead.

In order to avoid the collision in definitions (sha256.h cannot be
included in the same file as a file where openssl headers has been
included) let's move the sha256 related code in its own file

PR: 270023
Reported by: ngie

pkg-bootstrap: use latest package set on arm64 stable branches

As with i386 and amd64, "latest" packages are available on stable
branches for arm64/aarch64.

Reviewed by: manu
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35445

pkgbase: Add a FreeBSD-pkg-bootstrap package

And put pkg and its keys in it.
It's easier for small image to depend on this package rather than the
larger utilities one.

MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D33458

pkg: use specific CONFSNAME_${file} for FreeBSD.conf

Setting CONFSNAME directly is a little more complicated for downstream
consumers, as any additional CONFS that are added here will inherit the
group name by default. This is perhaps arguably a design flaw in CONFS
because inheriting NAME will never give a good result when additional
files are added, but this is a low-effort change.

While we're here, pull FreeBSD.conf.${branch} out into a PKGCONF
variable so one can just drop a new repo config in entirely with a new
naming scheme. CONFSNAME gets set based on chopping anything off after
".conf", so that, e.g.:

- FooBSD.conf => FooBSD.conf
- FooBSD.conf.internal => FooBSD.conf

Reviewed by: bapt, manu
Differential Revision: https://reviews.freebsd.org/D28767

pkg(7): replace usage of sbuf(9) with open_memstream(3)

open_memstream(3) is a standard way to obtain the same feature we do get
by using sbuf(9) (aka dynamic size buffer), switching to using it makes
pkg(7) more portable, and reduces its number of dependencies.

Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D30005

Make use of the getlocalbase() function for run-time adjustment of the
local software base directory, as committed in SVN rev. 367813.

The pkg and mailwrapper programs used the LOCALBASE environment variable
for this purpose and this functionality is preserved by getlocalbase().

After this change, the value of the user.localbase sysctl variable is used
if present (and not overridden in the environment).

The nvmecontrol program gains support of a dynamic path to its plugin
directory with this update.

Differential Revision: https://reviews.freebsd.org/D27237

Revert the whole getlocalbase() set of changes while a different design is
hashed out.

Replace hardcoded references to _PATH_LOCALBASE with calls to getlocalbase.3

Reviewed by: imp, se

Rework the logic for installing the pkg(8) configuration.

'quarterly' package sets do not exist for head, so explicitly
install the 'latest' configuration file there. Otherwise,
fall back to the original conditional evaluation to determine
if the 'latest' or 'quarterly' configuration file should be
installed.

Reported by: manu
Reviewed by: manu
MFC after: 3 days
Sponsored by: Rubicon Communications, LLC (Netgate)

Apply r346792 (cperciva) from stable/12 to head. The original commit
message:

On non-x86 systems, use "quarterly" packages.

x86 architectures have "latest" package builds on stable/*, so keep using
those (they'll get switched over to "quarterly" during releases).

The original commit was a direct commit to stable/12, as at the time it
was presumed it would not be necessary for head. However, when it is time
to create a releng branch or switch from PRERELEASE/STABLE to BETA/RC, the
pkg(7) Makefile needs further adjusting. This commit includes those
further adjustments, evaluating the BRANCH variable from release/Makefile
to determine the pkg(7) repository to use.

MFC after: immediate (if possible)
Sponsored by: Rubicon Communications, LLC (Netgate)

Keep two versions of the FreeBSD.conf pkg configuration file; one which
points at the "latest" branch and one which points at the "quarterly"
branch. Install the "latest" version unless overridden via the newly
added PKGCONFBRANCH variable.

This does not change user-visible behaviour (assuming said vairable is
not set) but will make it easier to change the defaults in the future --
on stable branches we will want "latest" on x86 but "quarterly" elsewhere.

Discussed with: gjb
MFC after: 3 days
X-MFC: After MFCing this I'll make a direct commit to stable/* to
switch non-x86 architectures to "quarterly".

Move pkg/FreeBSD.conf to usr.sbin/pkg/

Approved by: bapt (mentor)
Differential Revision: https://reviews.freebsd.org/D16491

usr.sbin: normalize paths using SRCTOP-relative paths or :H when possible

This simplifies make logic/output

MFC after: 1 month
Sponsored by: Dell EMC Isilon

Implement pubkey support for the bootstrap

Note that to not interfer with finger print it expects a signature on pkg itself
which is named pkg.txz.pubkeysign

To genrate it:
echo -n "$(sha256 -q pkg.txz)" | openssl dgst -sha256 -sign /thekey \
-binary -out ./pkg.txz.pubkeysig

Note the "echo -n" which prevent signing the '\n' one would get otherwise

PR: 202622
MFC after: 1 week

Convert usr.sbin to LIBADD
Reduce overlinking

Remove spurious libelf dependency.

Submitted by: pluknet

Use pkg-1.4-style platform identifiers based on MACHINE_ARCH (e.g.
FreeBSD:11:amd64 instead of freebsd:11:x86:64) when bootstrapping pkg.
Thanks to portmgr for providing symlinks so both styles work.

Reviewed by: bapt
MFC after: 3 weeks

Rework privatelib/internallib

Make sure everything linking to a privatelib and/or an internallib does it directly
from the OBJDIR rather than DESTDIR.
Add src.libnames.mk so bsd.libnames.mk is not polluted by libraries not existsing
in final installation
Introduce the LD* variable which is what ld(1) is expecting (via LDADD) to link to
internal/privatelib
Directly link to the .so in case of private library to avoid having to complexify
LDFLAGS.

Phabric: https://phabric.freebsd.org/D553
Reviewed by: imp, emaste

Add missing LIBM to DPADD from r263650

Reported by: jilles

Fix build with libucl 20140321

Switch pkg(7) from libyaml to libucl

Add a pkg(7) manpage for bootstrap

Approved by: bapt
MFC after: 2 days

Link in libcrypto as well to fix build in some cases.

Approved by: bapt
MFC after: 2 days
Reported by: many

Support checking signature for pkg bootstrap.

If the pkg.conf is configured with SIGNATURE_TYPE: FINGERPRINTS,
and FINGERPRINTS: /etc/keys/pkg then a pkg.sig file is fetched along
with pkg.txz. The signature contains the signature provided by the
signing server, and the public key. The .sig is the exact output
from the signing server in the following format:

SIGNATURE
<openssl signed>
CERT
<rsa public key>
END

The signature is verified with the following logic:

- If the .sig file is missing, it fails.
- If the .sig doesn't validate, it fails.
- If the public key in the .sig is not in the known trusted fingerprints,
it fails.
- If the public key is in the revoked key list, it fails.

Approved by: bapt
MFC after: 2 days
Discussed by: bapt with des, jonathan, gavin

Rename libbsdyml to libyaml, make private, and bump
SHLIB_MAJOR to 1.0

Suggested by: des
Approved by: bapt
MFC after: 1 week

Fix a typo in DPADD.

Add the ability to correctly read pkg.conf is exists.

Only look for boostrap useful options:
- PACKAGESITE
- ABI
- MIRROR_TYPE
- ASSUME_ALWAYS_YES

While here makes PACKAGESITE expand the ${ABI} variable.
Allow to deactivate any SRV record look up (MIRROR_TYPE=none)
Use the same mechanism as for pkgng itself: first get configuration out of
environment variable and fallback on pkg.conf if exists.

Reviewed by: bdrewery

if PACKAGESITE url scheme is not 'file://', always try to revolv SRV records and
use them if any. It allows the bootstrap to use directly pkg.FreeBSD.org instead
of pkgbeta.FreeBSD.org

MFC after: 1 month

- Use _PATH_TMP instead of hardcoded /tmp
- more style(9) fixes
- some refactoring
- better error detection
- Add the DPADD to Makefile

Submitted by: Garrett Cooper <yanegomi@gmail.com>
Approved by: des (mentor)

add usr.sbin/pkg which is a bootstrap tool for pkgng.
it respects PACKAGESITE, PACKAGEROOT, and a new environment variable ABI (if a user want to use a different API from the base one for its packages)
it has no man page on purpose to avoid hidding the pkg(8) man page from the pkgng package.
for now uses pkgbeta.FreeBSD.org as default mirror to find its package
it respects MK_PKGTOOLS

Approved by: des (mentor)