man(1): ignore absolute path for .so include

We want only a relative include, as ".so man1/foobar.1"

MFC after: 1 week

man(1) needs to check for .so files not only in the first line
PR: 279182

Some manual pages have a copyright notice or commit id before including
other files with the .so macro. We need to skip comments and empty lines
at the beginning of the manpage while checking for the first .so macro.

MFC after: 1 week

man: the exists function needs to validate the first parameter

This fixes an issue with the ".so " macro for FreeBSD
ports manual pages.

PR: 275978
Reported by: Jamie Landeg-Jones <jamie@catflap.org>
MFC after: 1 week

Differential Revision: https://reviews.freebsd.org/D45231 (discussion)

man: do not ignore the exit status of roff tools

PR: 223516
Approved by: emaste, bapt
Differential Revision: https://reviews.freebsd.org/D44798

man: fix redirect for locales and file arguments

Use the locale-specific path for localized pages and the appropriate parent
directory for file arguments when handling redirects.

Signed-off-by: Mohamed Akram <mohd.akram@outlook.com>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/1019

man: support special characters in filenames

man.sh needs to handle double quotes and sub shell character
as '`' '$' etc.

PR: 275967
Reviewed by: bapt

MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D43453

man: avoid unportable use of utilities

echo -e is not portable. It can be replaced by printf %b (it works
only with the /bin/sh built-in echo, not /bin/echo anyway).
head -# is not portable, but head -n # is.

Replace these two things in three places total.

Signed-off-by: Mohamed Akram <mohd.akram@outlook.com>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/1062

man(1): support spaces in filenames

The globbing function in exists() needs to handle white spaces.

PR: 275978
Reviewed by: kevans, bapt, emaste
MFC after: 1 week

fix using man(1) with multiple pages

PR: 275978
Reported by: Mohamed Akram
Fixes: 789480702e490818244af11279868ba4f3dabe6b
MFC after: 1 week

support spaces in filenames

PR: 275978
MFC after: 1 week

improve handling of shell metacharacters in filenames

PR: 275967
MFC after: 1 week

switch groff parameter -man to -mandoc

groff 1.23.0 changed the semantics of the -man parameter, and many
manual pages are not rendered. The -mandoc parameter brings back
the old behavior, as in groff 1.22.4 and earlier.

PR: 273565, 273245

Reviewed by: emaste, bapt
MFC after: 1 week for all supported branches (stable/12, 13, 14)
Differential Revision: https://reviews.freebsd.org/D41737

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

man: fix `man -K` search

Quote re arg to grep in case it has spaces, and quote [:blank:] tr arg
to avoid the shell interpreting [].

PR: 272729
Reviewed by: Mina Galić <freebsd@igalic.co>
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D41282

man(1): use gzcat for .gz files

POSIX zcat appends the .Z suffix to file arguments causing the command to fail
with .gz files.

Signed-off-by: Mohamed Akram <mohd.akram@outlook.com>

Reviewed by: markj
MFC after: 2 weeks
Pull Request: https://github.com/freebsd/freebsd-src/pull/789

man(1): Support custom sections

PR: 271830
Signed-off-by: Mohamed Akram <mohd.akram@outlook.com>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/764

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

man(1): Support zstd compressed manpages using zstdcat

Reviewed by: emaste, markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D32296

do not expect mandoc(1) to terminate in finite time

PR: 266868

man(1): Add full search (-K) flag

This flag allows a full text search on man pages. Although this is a last resort
option, it can be useful to pin point a certain man page.

It can be used with -S to narrow the search.

Unlike the Linux version, the search takes place in the rendered text so it
avoids false-positives when the text is found in comments in the source files.

It relies on `grep(1)` and `mandoc(1)` to do its job.

Add flag documentation and EXAMPLES to the manual page (bump .Dd).

Usage example:

man -w -K '\<arm\>' -S 1:8

Reviewed By: ceri, emaste, pauamma_gundo.com
Approved by: manpages (bcr@), debdrup@
Differential Revision: https://reviews.freebsd.org/D30984

man: reset OPTIND before parsing args

From jilles: POSIX requires that a script set `OPTIND=1` before using
different sets of parameters with `getopts`, or the results will be
unspecified.

The specific problem observed here is that we would execute `man -f` or
`man -k` without cleaning up state from man_parse_args()' `getopts`
loop. FreeBSD's /bin/sh seems to reset OPTIND to 1 after we hit the
second getopts loop, rendering the following shift harmless; other
/bin/sh implementations will leave it at what we came into the loop at
(e.g., bash as /bin/sh), shifting off any keywords that we had.

Input from: jilles
Reviewed by: allanjude, bapt, imp
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D32063

flua: Add a libjail module

libjail is pretty small, so it makes for a good proof of concept demonstrating
how a system library can be wrapped to create a loadable Lua module for flua.

* Introduce 3lua section for man pages
* Add libjail module

Reviewed by: kevans, manpages
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D26080

Teach man(1) about C.UTF-8.

While here, use LANG as the proper source to select man pages language/encoding,
falling back to LC_CTYPE.

Reviewed by: bapt
Approved by: kib (mentor, implicit)
Differential Revision: https://reviews.freebsd.org/D17835

Switch the default pager for most commands to less

Finally, a pager for the nineties.

MFC after: Never
Relnotes: Yes
Differential Revision: https://reviews.freebsd.org/D13465
Poll: https://reviews.freebsd.org/V7

Only skip looking for manpages if both man directory and cat directory
are not existing.

This allows man(1) to read catpages when no man directories are available at all

PR: 223559
Reported by: wosch
MFC after: 3 days

In case man(1) found a catpage to display skip looking ".so" which is manpage
only.

In case we are trying to read a catpage, the manpage variable is not defined.
It results in the "cattool" having no arguments.

In case the catpage is compressed, the cattool used is "zcat" which dies if the
standard input is a terminal, meaning the function calling it is exiting as if
there were no ".so"
In case the catpage is uncompressed, the cattool used is "zcat -f" which waits
reading standard input, making the man(1) command hang.

PR: 223560
Reported by: wosch
MFC after: 3 days

various: general adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.

man(1): silent the output of mandoc when testing

This reduce the spam a user may face when mandoc tries to
figure out if it can renders a manpage or fallback on groff(1)

Reported by: bdrewery
MFC after: 3 days

- Remove separate handling of /bin and /usr/bin in manpath. They are no
longer a special case.
- Prefer PREFIX/share/man over PREFIX/man.
- Add /usr/local/share/man to man_default_path.
- Update manpath man page.

Reviewed by: bapt

Remove the warning when MANPATH is set in the environment

The MANPATH environment variable behaviour is documented properly in the manpage
and it now has extended to new feature that allows to make MANPATH env variable
extending the default search path rather than overwriting it making the warning
painful

Reported by: kargl
MFC after: 1 week

Extend functionality MANPATH in man(1) to followup with apropos(1) from
mandoc.

If MANPATH begins with a colon, it is appended to the default list; if it ends
with a colon, it is prepended to the default list; or if it contains two
adjacent colons, the standard search path is inserted between the colons. If
none of these conditions are met, it overrides the standard search path.

Import the MANPATH description from mandoc into the man(1) man page

Reported by: kargl
MFC after: 1 week

Add share/man if it exists to the MANPATH

localbase is not consistent with base for manpages:
/usr/local/man vs /usr/share/man adding share/man allows to fix that
inconsistency and would permit to remove tons of patches/modifications in the
ports tree

Restore man -t for manpages rendered by mandoc

Reported by: swills

inode should be different to actually mean mandocdb is in used

Sponsored by: gandi.net

Fix man -k with mandocdb

If apropos(1) and whatis(1) are not hardlinks to man(1) that means the system is
using mandocdb, then man -k should spawn apropos(1) and/or whatis(1) directly

Reported by: kevlo
Tested by: kevlo
Sponsored by: gandi.net

Allow MANWIDTH to work with mandoc(1)

Reported by: bdrewery

Only manpages with fatal errors should be rendered by groff(1)

mandoc -Tlocale is now the default, no need to enforce it anymore

Fix comments

Fix typo

Change man(1) to use mandoc to render manpages

man(1) now first test the manpage to run with mandoc to make sure it can be
rendered.
In case groff cannot be found (because base has been built WITHOUT_GROFF) it
recommands to install groff from the packages

Remove default support for 1aout section manpages. There haven't been
any since at least July 2002.

Spelling fixes for usr.bin/

When MANCOLOR environment variable is set, enable ANSI color escapes
in grotty(1). This makes it possible to view colorized manpages in
color.

When MANPAGER environment variable is set, use it instead of PAGER.

Why another environment variable, one might ask? With color output
enabled, both a terminal and a pager should support the ANSI color
escapes. On a supporting terminal, less(1) with option -R would be
such a pager, while "more -s" (the current default pager for man(1))
will show garbage. It means a different default pager is needed when
color output is enabled, but many people have PAGER set customary,
and it's unlikely to support ANSI color escapes, so introducing yet
another variable (MANPAGER) seemed like a good option to me:

- if MANPAGER is set, use that unconditionally;

- if you disable color support (it is by default), and don't set
MANPAGER, you get an old behavior: -P pager, $PAGER, "more -s",
in that order;

- if you enable color support (by setting MANCOLOR), and don't set
MANPAGER, we ignore PAGER which is unlikely to support ANSI color
escapes, and you get: -P pager, "less -Rs", in that order;

- you might have good reasons for different man(1) and general
purpose pagers;

- later versions of GNU man(1) support MANPAGER.

Don't use col(1) since grotty(1) never outputs reverse line feeds,
and because col(1) mangles ANSI color escapes if enabled. Spaces
to tabs compression is now done by passing -h to grotty(1).

Discussed with: uqs

Trim more when parsing MANCONFIG directive.

Added support for the MANWIDTH environment variable:

If set to a numeric value, used as the width manpages should be
displayed. Otherwise, if set to a special value ``tty'', and
output is to a terminal, the pages may be displayed over the
whole width of the screen.

Fix man -t by not passing grotty flags to groff when grotty is not
involved.

This fixes a regression introduced with r221303.

Noticed by: jilles

Let users' PATH decide which groff suite to pick up.

Let groff pass the -c flag to grotty, which will turn off ANSI
sequences. While these are not a problem for our more/less, they get
mangled by col(1) and this will result in garbage output.

This makes man(1) work together with textproc/groff, in case the
user decided to delete the old groff from base (-DWITHOUT_GROFF).

Overhaul locale handling.

Use locale(1) to determine the locale instead of trying to hand roll it.
Correctly construct groff call based on charset and locale independently,
not the mix between the two.

Submitted by: uqs@

Use test(1) operators and test for the catpage not being older than the
manpage.

Identical mtimes (as generated by buildworld for these files) precluded
catpages from working.

Approved by: gordon

Move sysctl invocation to using a variable that's fully pathed.
This prevents errors for users that don't have /sbin in their PATH.

Submitted by: Max Boyarov
Approved by: mentor (wes@ implicit)

Add ability to decompress different format manual pages.
Add support for .so directive in manual pages.

Approved by: mentor (wes@)

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Add the ability to display specific manual pages if passed on the
commandline. This mirrors the old (undocumented) GNU man functionality.
Also document this feature in the implementation notes section of
the manpage.

Submitted by: arundel
Approved by: wes (mentor implicit)

If LANG/LC_CTYPE/LC_ALL is set and the localized man page contains a page
also in the base set, the man utility when invoked with -a would display
pages for each locale in the same tree:

$ LANG=en_GB.ISO8859-15 man -wa man
/usr/share/man/en.ISO8859-15/man1/man.1.gz
/usr/share/man/man1/man.1.gz
/usr/share/man/en.ISO8859-15/man7/man.7.gz
/usr/share/man/man7/man.7.gz

Use continue to break out of the loop for the current locale. This results
in behavior more closely matching the old GNU man implementation:

$ LANG=en_GB.ISO8859-15 man -wa man
/usr/share/man/en.ISO8859-15/man1/man.1.gz
/usr/share/man/en.ISO8859-15/man7/man.7.gz

This will still search for a copy of the file in other manual path
locations. If there was a /usr/local/man/man1/man.1.gz file, it would still
be displayed. This is also consistent with the GNU man implementation.

Submitted by: arundel
Approved by: wes (mentor implicit)

Fix up whatis/apropos issue displaying all output on a single due to
forgotten quotes.

Submitted by: Brandon Gooch
Approved by: wes (mentor, implicit)

Implementaiton of man, manpath, whatis, and apropos written entirely in sh.

Features of this new version in favor of the old one:
BSD licensed -- old one is GPL.
Imports configuration from /etc/man.conf and LOCALBASE/etc/man.d/*.conf
allowing ports to extend the base functionality. The pluggable
configuration can supplement the manual search path (retiring use.perl),
add locales, and override language specific toolsets (attempt to merge
the japanese/man port into the base system as much as possible).

Much effort has been made to make this version mirror the functionality
of the existing implementation. For 99% of users, it should be a drop in
replacement.

PR: gnu/143271, gnu/4419
Reviewed by: dougb (previous versions)
Approved by: wes (mentor)