certctl: Revert to symlinks.

Unfortunately tar will not be able to extract base.txz to a system where
/etc and /usr are not on the same filesystem if the certificates are
hard links.

PR: 277828
Reviewed by: mp
Differential Revision: https://reviews.freebsd.org/D44496

certctl: Convert line endings before inspecting files.

This ensures that certificate files or bundles with DOS or Mac line
endings are recognized as such and handled identically to those with
Unix line endings.

PR: 274952
Reviewed by: allanjude
Differential Revision: https://reviews.freebsd.org/D42490

certctl: Fix recent regressions.

- If an untrusted certificate is also found in the list of trusted
certificate, issue a warning and skip it, but don't fail.
- Split on -+BEGIN CERTIFICATE-+ instead of "Certificate:" since
that's what we're really looking for.

Also fix a long-standing bug: .crl files are not certificates, so we
should not include them when searching for certificates.

Reported by: madpilot, netchild, tijl
Reviewed by: netchild, allanjude
Differential Revision: https://reviews.freebsd.org/D42276

certctl: Split certificate bundles before processing.

This allows 'certctl rehash' to do the right thing when ca_root_nss is
installed, instead of linking the entire bundle to the hash of the
first certificate it contains.

MFC after: 3 days
Reviewed by: allanjude
Differential Revision: https://reviews.freebsd.org/D42087

certctl: Clean up.

MFC after: 3 days
Reviewed by: allanjude
Differential Revision: https://reviews.freebsd.org/D42086

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

Strip trailing slashes from DESTDIR

Solves duplicate slashes in paths

Scanning //usr/share/certs/trusted for certificates...
Scanning //usr/local/share/certs for certificates...

Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/595

certctl: Introduce a new -d <distbase> option

This will be used by Makefile.inc1 to fix -DNO_ROOT distributeworld,
which needs to split out DESTDIR from DISTBASE so the METALOG file
includes the base/ prefix.

Reviewed by: kevans
Obtained from: CheriBSD
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D35808

secure/caroot, certctl: Rename secure/caroot/blacklisted

Old certctl commands still work for compatability, but are deprecated.

Approved by: secteam (gordon)
Differential Revision: https://reviews.freebsd.org/D30807

certctl: factor out certname resolution

create_blacklisted() will identify a cert whether it's provided a path to
a cert or the hash.serial format that is shown by `certctl list`.

Factor this logic out into a resolve_certname() so that it may be reused
elsewhere.

certctl: replace hardcoded uses of /usr/local

Use the new user.localbase sysctl here as well, to reduce the number of
hardcoded localbase by one (1).

MFC after: 3 days (note: just use a literal /usr/local default)

certctl: fix unprivileged mode

The first issue was lack of quoting around INSTALLFLAGS, which set it
incorrectly and produced an error on -M.

The second issue was that we weren't actually doing the install in
unprivileged mode, making it effectively useless. This was designed to pass
through the proper metalog/unpriv flags to install(1), so just let it
happen.

MFC after: 3 days

certctl: fix hashed link generation with duplicate subjects

Currently, certctl rehash will just keep clobbering .0 rather than
incrementing the suffix upon encountering a duplicate. Do this, and do it
for blacklisted certs as well.

This also improves the situation with the blacklist to be a little less
flakey, comparing cert fingerprints for all certs with a matching subject
hash in the blacklist to determine if the cert we're looking at can be
installed.

Future work needs to completely revamp the blacklist to align more with how
it's described in PR 246614. In particular, /etc/ssl/blacklisted should go
away to avoid potential confusion -- OpenSSL will not read it, it's
basically certctl internal.

PR: 246614
Reviewed by: Michael Osipov <michael.osipov siemens com>
Tested by: Michael Osipov
With suggestions from: Michael Osipov
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D26167

certctl: fix test syntax

test doesn't understand &&, but it does understand -a.

MFC after: 1 week

Add an unprivileged mode where calls to install are passed appropriate
flags. For ease of integration, use the same flags as install:

-U unprivileged mode
-D <destdir> Specify DESTDIR (overrides the environment)
-M <metalog> Full path to METALOG file

Reviewed by: kevans
Obtained from: CheriBSD
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D24932

certctl: don't fall over flat with relative DESTDIR

Up until now, all of our DESTDIR use has been with absolute paths. It turned
out that the cd in/out dance we do here breaks us down later on, as the
relative path no longer resolves.

Convert EXTENSIONS to an ERE that we'll use to grep ls -1 of the dir we're
inspecting, rather than cd'ing into it and globbing it up.

MFC after: 3 days

certctl: follow-up to r361022, prune blacklist as well

Otherwise, removals from the blacklist may not get processed as they should.

While we're here, restructure these to not bother with mkdir(1) if we've
already tested them to exist.

MFC after: 3 days

certctl(8): don't completely nuke $CERTDESTDIR

It's been reported/noted that a well-timed `certctl rehash` will completely
obliterate $CERTDESTDIR, which may get used by ports or system
administrators. While we can't guarantee the certctl semantics when other
non-certctl-controlled bits live here, we should make some amount of effort
to play nice.

Pruning all existing links, which we'll subsequently rebuild as needed, is
sufficient for our needs. This can still be destructive, but it's perhaps
less likely to cause issues.

I also note that we should probably be pruning /etc/ssl/blacklisted upon
rehash as well.

Reported by: cem's dovecot server
MFC after: 3 days

certctl(8): switch to install(1) to fix DESTDIR support

"Oops" - ln(1) is fine and dandy, but when you're using DESTDIR...it's not-
the path will almost certainly be invalid once the root you've just
installed to is relocated, perhaps to /.

Switch to install(1) using `-l rs` to calculate the relative symlink between
the two, which should work just fine in all cases.

MFC after: 1 week

certctl(8): let one blacklist based on hashed filenames

It seems reasonable to allow, for instance:

$ certctl list
# reviews output -- ah, yeah, I don't trust that one
$ certctl blacklist ce5e74ef.0
$ certctl rehash

We can unambiguously determine what cert "ce5e74ef.0" refers to, and we've
described it to them in `certctl list` output -- I see little sense in
forcing another level of filesystem inspection to determien what cert file
this physically corresponds to.

certctl(8): realpath the file before creating the symlink

Otherwise we end up creating broken relative symlinks in
/etc/ssl/blacklisted.

[2/3] Add certctl(8)

This is a simple utility to hash all trusted on the system into
/etc/ssl/certs. It also allows the user to blacklist certificates they do
not trust.

This work was done primarily by allanjude@, with minor contributions by
myself.

No objection from: secteam
Differential Revision: https://reviews.freebsd.org/D16857