Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

various: general adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.

Switch hastctl and hastd to libmd instead of openssl for sha256

Reviewed by: bapt, brnrd, pjd
Approved by: pjd (maintainer)
Sponsored by: ScaleEngine Inc., FOSDEM DevSummit
Differential Revision: https://reviews.freebsd.org/D9423

- Add support for 'memsync' mode. This is the fastest replication mode that's
why it will now be the default.
- Bump protocol version to 2 and add backward compatibility for version 1.
- Allow to specify hosts by kern.hostid as well (in addition to hostname and
kern.hostuuid) in configuration file.

Sponsored by: Panzura
Tested by: trociny

For functions that return -1 on failure check exactly for -1 and not for
any negative number.

MFC after: 3 days

Use PJDLOG_ASSERT() and PJDLOG_ABORT() everywhere instead of assert().

MFC after: 3 days

Add missing ifdef. This fixes build with NO_OPENSSL.

Reported by: Pawel Tyll <ptyll@nitronet.pl>
Approved by: pjd (mentor)
MFC after: 1 week

Remove hast_proto_recv(). It was used only in one place, where
hast_proto_recv_hdr() may be used. This also fixes the issue
(introduced by r220523) with hastctl, which crashed on assert in
hast_proto_recv_data().

Suggested and approved by: pjd (mentor)

In hast_proto_recv() remove unnecessary check. The size is checked
later in hast_proto_recv_data().

Approved by: pjd (mentor)
MFC after: 1 week

In hast_proto_recv_data() check that the size of the data to be
received does not exceed the buffer size.

Approved by: pjd (mentor)
MFC after: 1 week

Allow to compress on-the-wire data using two algorithms:
- HOLE - it simply turns all-zero blocks into few bytes header;
it is extremely fast, so it is turned on by default;
it is mostly intended to speed up initial synchronization
where we expect many zeros;
- LZF - very fast algorithm by Marc Alexander Lehmann, which shows
very decent compression ratio and has BSD license.

MFC after: 2 weeks

Allow to checksum on-the-wire data using either CRC32 or SHA256.

MFC after: 2 weeks

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Constify arguments we can constify.

MFC after: 2 weeks
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com

MFC r209263:

r209175:

Eliminate dead code.

Found by: Coverity Prevent
CID: 5158

r209177:

Remove macros that are not really needed. The idea was to have them in case
we grow more descriptors, but I'll reconsider readding them once we get there.

Passing (a = b) expression to FD_ISSET() is bad idea, as FD_ISSET() evaluates
its argument twice.

Found by: Coverity Prevent
CID: 5243

r209179:

Plug memory leaks.

Found by: Coverity Prevent
CID: 7052, 7053, 7054, 7055

r209180:

Plug memory leak.

Found by: Coverity Prevent
CID: 7051

r209181:

Plug memory leak.

Found by: Coverity Prevent
CID: 7056

r209182:

Plug memory leak.

Found by: Coverity Prevent
CID: 7057

r209183:

Initialize gctl_seq for synchronization requests.

Reported by: hiroshi@soupacific.com
Analysed by: Mikolaj Golub <to.my.trociny@gmail.com>
Tested by: hiroshi@soupacific.com, Mikolaj Golub <to.my.trociny@gmail.com>

r209184:

Fix typos.

r209185:

Correct various log messages.

Submitted by: Mikolaj Golub <to.my.trociny@gmail.com>

Note that without some of these changes hastd won't work on 8.x properly.

Approved by: re (kensmith)

Eliminate dead code.

Found by: Coverity Prevent
CID: 5158
MFC after: 3 days

MFC r207070,r207343,r207345,r207347,r207348,r207371,r207372,r207390:

r207070:

Fix compilation with WITHOUT_CRYPT or WITHOUT_OPENSSL options.

Reported by: Andrei V. Lavreniyuk <andy.lavr@reactor-xg.kiev.ua>

r207343:

Don't assume that "resource" property is in metadata.

Reported by: Mikolaj Golub <to.my.trociny@gmail.com>

r207345:

Use WEXITSTATUS() to obtain real exit code.

r207347:

Mark temporary issues as such.

r207348:

Restart worker thread only if the problem was temporary.
In case of persistent problem we don't want to loop forever.

r207371:

Fix a problem where hastd will stuck in recv(2) after sending request to
secondary, which died between send(2) and recv(2). Do it by adding timeout
to recv(2) for primary incoming and outgoing sockets and secondary outgoing
socket.

Reported by: Mikolaj Golub <to.my.trociny@gmail.com>
Tested by: Mikolaj Golub <to.my.trociny@gmail.com>

r207372:

- Check if the worker process was killed by signal and restart it.
- Improve logging.

Pointed out by: Garrett Cooper <yanefbsd@gmail.com>

r207390:

Default connection timeout is way too long. To make it shorter we have to
make socket non-blocking, connect() and if we get EINPROGRESS, we have to
wait using select(). Very complex, but I know no other way to define
connection timeout for a given socket.

Reported by: hiroshi@soupacific.com

Fix compilation with WITHOUT_CRYPT or WITHOUT_OPENSSL options.

Reported by: Andrei V. Lavreniyuk <andy.lavr@reactor-xg.kiev.ua>
MFC after: 3 days

MFC r204076,r204077,r204083,r205279:

r204076:

Please welcome HAST - Highly Avalable Storage.

HAST allows to transparently store data on two physically separated machines
connected over the TCP/IP network. HAST works in Primary-Secondary
(Master-Backup, Master-Slave) configuration, which means that only one of the
cluster nodes can be active at any given time. Only Primary node is able to
handle I/O requests to HAST-managed devices. Currently HAST is limited to two
cluster nodes in total.

HAST operates on block level - it provides disk-like devices in /dev/hast/
directory for use by file systems and/or applications. Working on block level
makes it transparent for file systems and applications. There in no difference
between using HAST-provided device and raw disk, partition, etc. All of them
are just regular GEOM providers in FreeBSD.

For more information please consult hastd(8), hastctl(8) and hast.conf(5)
manual pages, as well as http://wiki.FreeBSD.org/HAST.

Sponsored by: FreeBSD Foundation
Sponsored by: OMCnet Internet Service GmbH
Sponsored by: TransIP BV

r204077:

Remove some lines left over by accident.

r204083:

Add missing KEYWORD line.

Pointed out by: dougb

r205279 sys:

Simplify loops.

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

Please welcome HAST - Highly Avalable Storage.

HAST allows to transparently store data on two physically separated machines
connected over the TCP/IP network. HAST works in Primary-Secondary
(Master-Backup, Master-Slave) configuration, which means that only one of the
cluster nodes can be active at any given time. Only Primary node is able to
handle I/O requests to HAST-managed devices. Currently HAST is limited to two
cluster nodes in total.

HAST operates on block level - it provides disk-like devices in /dev/hast/
directory for use by file systems and/or applications. Working on block level
makes it transparent for file systems and applications. There in no difference
between using HAST-provided device and raw disk, partition, etc. All of them
are just regular GEOM providers in FreeBSD.

For more information please consult hastd(8), hastctl(8) and hast.conf(5)
manual pages, as well as http://wiki.FreeBSD.org/HAST.

Sponsored by: FreeBSD Foundation
Sponsored by: OMCnet Internet Service GmbH
Sponsored by: TransIP BV