sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

teken: color #3 is yellow not brown - use TC_YELLOW as the name

The console escape code standard (ECMA-48) specifies color #3 (escape
code 33) as yellow. A brown console color is an artifact of the VGA
palette, which replaces dim (but not bright) yellow with brown.

Reviewed by: adrian, imp
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34531

Adjust teken to allow build as part of loader

Building for loader needs specific headers.

My recent fix for programmable function keys in syscons only worked
when TEKEN_CONS25 is configured. Fix this by adding a function to
set the flag that enables the fix and always calling this function
for syscons.

Expand the man page for teken_set_cons25(). This function is not
very useful since it can only set but not clear 1 flag. In practice,
it is only used when TEKEN_CONS25 is configured and all that does is
choose the the default emulation for syscons at compile time.

Fix function keys for syscons in cons25 mode (vidcontrol -T cons25).

kbd(4) (but only documented in atkbd(4)) maintains a table of strings
for 96 function keys. Using teken broke this 9+ years ago for the
most usable first 12 function keys and for 10 cursor keys, by supplying
its own non-programmable strings so that the keyboard driver's strings
are not used.

Fix this by supplying NULL in the teken layer for syscons in cons25 mode
so that the the strings are found in the kbd(4) layer.

vt needs more changes to use kbd(4)'s tables. Teken's cons25 table is
still needed to supply nonempty strings for vt in cons25 mode.

Keep using teken's xterm tables for both syscons and vt in xterm mode.
Function keys should at least default to xterm values in xterm mode,
and kbd(4) doesn't support this.

teken_set_cons25() sets a sticky flag to ask for the fix, and space is
reserved for another new flag. vt should set this flag when it uses
kbd(4)'s tables.

PR: 226553 (for vt)

teken: Unbreak syscons' use of teken

Only vt(4) initializes these callbacks non-NULL at this time, so invoke the
function pointers conditionally.

Broken in r333669.

Submitted by: bde@

teken, vt(4): New callbacks to lock the terminal once

... to process input, instead of inside each smaller operations such as
appending a character or moving the cursor forward.

In other words, before we were doing (oversimplified):

teken_input()
<for each input character>
vtterm_putchar()
VTBUF_LOCK()
VTBUF_UNLOCK()
vtterm_cursor_position()
VTBUF_LOCK()
VTBUF_UNLOCK()

Now, we are doing:

vtterm_pre_input()
VTBUF_LOCK()
teken_input()
<for each input character>
vtterm_putchar()
vtterm_cursor_position()
vtterm_post_input()
VTBUF_UNLOCK()

The situation was even worse when the vtterm_copy() and vtterm_fill()
callbacks were involved.

The new callbacks are:
* struct terminal_class->tc_pre_input()
* struct terminal_class->tc_post_input()

They are called in teken_input(), surrounding the while() loop.

The goal is to improve input processing speed of vt(4). As a benchmark,
here is the time taken to write a text file of 360 000 lines (26 MiB) on
`ttyv0`:

* vt(4), unmodified: 1500 ms
* vt(4), with this patch: 1200 ms
* syscons(4): 700 ms

This is on a Haswell laptop with a GENERIC-NODEBUG kernel.

At the same time, the locking is changed in the vt_flush() function
which is responsible to draw the text on screen. So instead of
(indirectly) using VTBUF_LOCK() just to read and reset the dirty area
of the internal buffer, the lock is held for about the entire function,
including the drawing part.

The change is mostly visible while content is scrolling fast: before,
lines could appear garbled while scrolling because the internal buffer
was accessed without locks (once the scrolling was finished, the output
was correct). Now, the scrolling appears correct.

In the end, the locking model is closer to what syscons(4) does.

Differential Revision: https://reviews.freebsd.org/D15302

Pedantic polishing of code to please FlexeLint.

Approved by: ed

sys: general adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.

Oops, my fix for bright colors broke bright black some more (in cases
that used to work via the bold hack).

Fix the table entry for bright black. Fix spelling of plain black in
nearby table entries (use the macro for black everywhere everywhere).
Fix the currently-unused non-bright color table to not have bright
colors in entries 9-15.

Improve nearby comments. Start converting to the xterm terminology
and default rendering of "bright" instead of "light" for bright
colors.

Syscons wasn't affected by the bug since I optimized it a little by
converting colors 0-15 directly. This also fixes the layering of
the conversion for these colors.

Apply the same optimization to vt (actually the layer above it). This
also moves the conversion 1 closer to the correct layer for colors
0-15.

The optimization of just avoiding 2 calls to a trivial function is worth
about 10% for simple output to the virtual buffer with occasional
rendering. The optimization is so large because the 2 calls are done
on every character, so although there are too many other calls and
other instructions per character, there are only about 10 times as
many. Old versions of syscons were about 10 times faster for simple
output, by using a fast path with about 12 instructions per character.
Rendering to even slow hardware takes relatively little time provided
it is rarely actually done.

Add teken_256to16() to convert xterm-256 256-color codes to xterm 16-color
codes. This will be used to fix bright colors.

Improve teken_256to8(). Use a lookup table instead of calculations. The
calculations were inaccurate since they used indexes into the xterm-256
6x6x6 color map instead of actual xterm colors. Also, change the threshold
for converting to a primary color: require the primary's component to be
2 or more higher instead of just higher. This affects about 1/5 of the
table entries and gives uniformly distributed colors in the 6x6x6 submap
except for greys (35 entries each for red, green, blue, cyan, brown and
magenta, instead of approx. only 15 each for the mixed colors). Even
more mixed colors would be better for matching colors, but uniform
distribution is best for preserving contrast.

For teken_256to16(), bright colors are just the ones with luminosity >=
60%. These are actually light colors (more white instead of more
saturation), while xterm bright colors except for white itself are
actually saturated with no white, so have luminosity only 50%.

These functions are layering violations. teken cannot do correct
conversions since it shouldn't know the color maps of anything except
xterm. Translating through xterm-16 colors loses information. This
gives bugs like xterm-256 near-brown -> xterm-16 red -> VGA red.

Pick UINT_MAX / 100 as an upperbound.

The fix that I applied in r286798 is already good, but it assumes that
sizeof(int) > sizeof(short). Express the upperbound in terms of
UINT_MAX. By dividing that by 100, we're sure that the resulting value
is never larger than approximately UINT_MAX / 10, which is safe.

PR: 202326
Discussed with: kcwu csie org
MFC after: 1 month

Stop parsing digits if the value already exceeds USHRT_MAX.

There is no need for us to support parsing values that are larger than
the maximum terminal window size. In this case that would be the maximum
of unsigned short.

The problem with parsing larger values is that they can cause integer
overflows when adjusting the cursor position, leading to all sorts of
failing assertions.

PR: 202326
Reported by: kcwu csie org
MFC after: 1 month

Fix typo.

Pointed by: Ronald Klop
Pointy hat: ray

Sponsored by: The FreeBSD Foundation

Fix crash on load of bigger font. It reduce width and height of terminal, but
current cursor position stay bigger that terminal window size, so next input
triggers assert.

Reported by: emaste

Sponsored by: The FreeBSD Foundation

Merge VT(9) project (a.k.a. newcons).

Reviewed by: nwhitehorn
MFC_to_10_after: re approval

Sponsored by: The FreeBSD Foundation

Fix various whitespace inconsistencies in sys/teken.

Add proper build infrastructure for teken.

I'm not sure whether we should install teken as a library on any stock
FreeBSD installation, but I can imagine people want to tinker around
with it now and then. Create a /sys/teken/libteken, which holds a
Makefile to install a shared library version of the terminal emulator,
complete with a manpage.

Also add Makefiles for the demo/stress applications, to build it against
the shared library.

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Just use <stdint.h> instead of <inttypes.h>. We don't need it here.

Place home and end before insert and delete.

These keys have different sequences when using cursorkeys, while insert
and delete stay the same. If they are placed like this, libteken will
return NULL instead of a proper sequence for these characters.

Allow Syscons terminal emulators to provide function key strings.

xterm and cons25 have some incompatibilities when it comes to escape
sequences for special keys, such as F1 to F12, home, end, etc. Add a new
te_fkeystr() that can be used to override the strings.

scterm-sck won't do anything with this, but scterm-teken will use
teken_get_sequences() to obtain the proper sequence.

Discard Device Control Strings and Operating System Commands.

These strings often contain things like:

- Window titles.
- Extended key map functionality.
- Color palette switching.

We could look at these features in the future (if people consider them
to be important enough), but we'd better discard them now. This fixes
some artifacts people reported when using TERM=xterm.

Reported by: des@, Paul B. Mahol

Add 256 color support.

It is quite inconvenient that if an application for xterm uses 256 color
mode, text suddenly starts to blink (because of ;5; in the middle).
We'd better just implement 256 color mode and add a conversion routine
from 256 to 8 color mode, which doesn't seem to be too bad in practice.

Remapping colors is done quite simple. If one of the channels is most
actively represented, primary colors are used. If two channels are most
actively represented, secondary colors are used. If all three channels
are equal (gray), it picks between black and white.

Reported by: Paul B. Mahol <onemda gmail com>

Get rid of now deprecated SCS wrappers.

We always build SCS, even when processing 8-bit data. There is no reason
why we should be able to disable it now.

Make SCS work in 8-bit mode.

This means we can finally do things like VT100 box drawing when using
Syscons (8-bit characters). As far as I know, the only remaining issue
is the absense of proper escape sequences for special keyboard
characters (cursor, F1 to F12, etc) and xterm emulation should be ready
for general use.

Enabling xterm would have the following advantages:

- Easier possible migration to Unicode. cons25 termcap entries are very
8-bit centric. They use things like CP437 characters for box drawing,
etc.

- Better support for SSH'ing to other operating systems/devices. Most
switches use VT100-style admin interfaces.

- Reduced bandwidth, because applications can now use things like
scrolling regions.

- You can finally use applications like dtach(1) on both the console and
inside an xterm.

Commit all local modifications I have to libteken:

- Make xterm/cons25 support runtime configurable. This allows me to
share libteken between syscons and my new vt driver.
- Add a fix to print blanks after printing a double width character to
prevent rendering artifacts.
- Add some more utility functions that I use in the vt driver.

Small style(9) bug introduced in the previous commit.

Make 8-bit support run-time configurable.

Now to do the same for xterm support. This means people can eventually
toy around with xterm+UTF-8 without recompiling their kernel.

Make resizing of teken terminals a bit more safe.

Just perform a full reset when resizing the terminal. This means the
cursor, scrolling region, etc. are never positioned outside the
terminal.

Expose the TF_REVERSE flag to the console driver.

Right now libteken processes TF_REVERSE internally and returns the
toggled colors to the console driver. This isn't entirely correct. This
means that the bold flag is always processed by the foreground color,
while reversing should be done after the foreground color has been set
to a brighter version by the bold flag.

This is no problem with the syscons driver, because with VGA it only
supports 16 foreground and 8 background colors. My WIP console driver
reconfigures the graphics hardware to disable the blink functionality
and uses 16 foreground and 16 background colors. This means that this
driver will handle the TF_REVERSE flag a little different from what
syscons does right now.

Move libteken out of the syscons directory.

I initially committed libteken to sys/dev/syscons/teken, but now that
I'm working on a console driver myself, I noticed this was not a good
decision. Move it to sys/teken to make it easier for other drivers to
use a terminal emulator.

Also list teken.c in sys/conf/files, instead of listing it in all the
files.arch files separately.