bin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

bin: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

ls: Improve POSIX compatibility for -g and -n.

- Change -g (ignored for BSD 4.3 compatibility since BSD 4.4)
to use POSIX semantics of implying -l but omitting the owner's
name.

- Change -n to imply -l.

The -o option remains unchanged (POSIX defines -o as a complement to
-g that implies -l but omits group names whereas BSD defines -o to add
file flags to -l). This compromise is the same used by both NetBSD
and OpenBSD.

PR: 70813
Reviewed by: jhb, Pau Amma <pauamma@gundo.com>
Co-authored-by: John Baldwin <jhb@FreeBSD.org>
Differential Revision: https://reviews.freebsd.org/D34747

Handle NULL return from localtime(3) in ls(1) and find(1)

The ls(1) (with -l option) and find(1) (with -ls option) utilties
segment fault when operating on files with very large modification
times. A recent disk corruption set a spurious bit in the mtime
field of one of my files to 0x8000000630b0167 (576460753965089127)
which is in year 18,266,940,962. I discovered the problem when
running fsck_ffs(8) which uses ctime(3) to convert it to a readable
format. Ctime cannot fit the year into its four character field, so
returns ??? ??? ?? ??:??:?? ???? (typically Thu Nov 24 18:22:48 2021).

With the filesystem mounted, I used `ls -l' to see how it would
report the modification time and it segment faulted. The find(1)
program also segment faulted (see script below). Both these utilities
call the localtime(3) function to decode the modification time.
Localtime(3) returns a pointer to a struct tm (which breaks things
out into its component pieces: year, month, day, hour, minute,
second). The ls(1) and find(1) utilities then print out the date
based on the appropriate fields in the returned tm structure.

Although not documented in the localtime(3) manual page, localtime(3)
returns a NULL pointer if the passed in time translates to a year
that will not fit in an "int" (which if "int" is 32-bits cannot
hold the year 18,266,940,962). Since ls(1) and find(1) do not check
for a NULL struct tm * return from localtime(3), they segment fault
when they try to dereference it.

When localtime(3) returns NULL, the attached patches produce a date
string of "bad date val". This string is chosen because it has the
same number of characters (12) and white spaces (2) as the usual
date string, for example "Sep 3 22:06" or "May 15 2017".

The most recent ANSI standard for localtime(3) does say that localtime(3)
can return NULL (see https://pubs.opengroup.org/onlinepubs/9699919799/
and enter localtime in the search box). Our localtime(3) man page should
be updated to indicate that NULL is a possible return. More importantly,
there are over 100 uses of localtime(3) in the FreeBSD source tree (see
Differential Revision D36474 for the list). Most do not check for a NULL
return from localtime(3).

Reported by: Peter Holm
Reviewed by: kib, Chuck Silvers, Warner Losh
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D36474

ls(1): Allow LSCOLORS to specify an underline

Allows capitalizing the background color character to enable an
underline instead of bold, capitalizing the foreground color char will
still do bold.

Differential Revision: https://reviews.freebsd.org/D30547

ls(1): Add --color=when

--color may be set to one of: 'auto', 'always', and 'never'.

'auto' is the default behavior- output colors only if -G or COLORTERM are
set, and only if stdout is a tty.

'always' is a new behavior- output colors always. termcap(5) will be
consulted unless TERM is unset or not a recognized terminal, in which case
ls(1) will fall back to explicitly outputting ANSI escape sequences.

'never' to turn off any environment variable and -G usage.

Reviewed by: cem, 0mp (both modulo last-minute manpage changes
Differential Revision: https://reviews.freebsd.org/D16741

Convert ls(1) to not use libxo(3)

libxo imposes a large burden on system utilities. In the case of ls, that
burden is difficult to justify -- any language that can interact with json
output can use readdir(3) and stat(2).

Logically, this reverts r291607, r285857, r285803, r285734, r285425,
r284494, r284489, r284252, and r284198.

Kyua tests continue to pass (libxo integration was entirely untested).

Reported by: many
Reviewed by: imp
Discussed with: manu, bdrewery
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D13959

General further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

Renumber copyright clause 4

Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96

Use uintmax_t to print st_nlink.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Fix ls -l alignement with new locales

Latest update of locales introduced abbreviated month that follows the regionale
rules meaning that they can be of variable length instead of being arbitrary
truncated to top 3 characters.

To fix alignement, ls now computes the visible length of the abbreviated month,
pads the shorter month with spaces in order to make sure everything is properly
aligned

Reviewed by: ache, ed, jilles
Differential Revision: https://reviews.freebsd.org/D4239

Cast uid and gid to the correct type for display to solve segfault in ls(1) on 32bit arches

Correctly escape literal % for display

This fixes segfaults in 32bit arches caused by r285734

Reviewed by: ngie
Approved by: dim
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D3191

Remove an excess space accidently introduced in the output in ls(1) by r285734

Spotted by: dim
Approved by: eadler (mentor)
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D3152

Fix some issues with the application of libxo to ls(1)

* Add whitespace trimming to some fields (username, group, size, inode, blocks) to avoid whitespace in JSON strings
* fix -m mode, was invalid JSON (repeated keys), and was missing outer array container
* in -n mode, numeric uids and gids were returned as strings

Approved by: eadler (mentor)
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D2854

Continue ubreaking ``env LANG=ru_RU.KOI8-R ls -l''.

File names are in current locale as well.

Unbreak ``env LANG=ru_RU.KOI8-R ls -l''.
Time strings are in the current locale.

r284198 seems to have left a null format string printf that gcc does *not*
like breaking mips builds.

Submitted by: Shawn Webb <shawn.webb@hardenedbsd.org>
Obtained from: HardenedBSD

Convert ls(1) to use libxo(3).

Obtained from: Phil Shafer <phil@juniper.net>
Sponsored by: Juniper Networks, Inc.

Undo over-aggressive conversion of spaces to tabs. ie: those within
format strings, "period, space, space" in comment text, etc.

Replace spaces with tabs where appropriate.

Reminded by: jh@

Add y flag and environment variable LS_SAMESORT to specify the same
sorting order for time and name with the -t option. IEEE Std 1003.2
(POSIX.2) mandates that the -t option sort in descending order, and
that if two files have the same timestamp, they should be sorted in
ascending order of their names. The -r flag reverses both of these
sort orders, so they're never the same. This creates significant
problems for sequentially named files stored on FAT file systems,
where it can be impossible to list them in the order in which they
were created.

Add , (comma) option to print file sizes grouped and separated by
thousands using the non-monetary separator returned by localeconv(3),
typically a comma or period.

MFC after: 14 days

Fix bin/ build with a 64-bit ino_t.

Original code by: Gleb Kurtsou

Get rid of major/minor number distinction.

As of FreeBSD 6, devices can only be opened through devfs. These device
nodes don't have major and minor numbers anymore. The st_rdev field in
struct stat is simply based a copy of st_ino.

Simply display device numbers as hexadecimal, using "%#jx". This is
allowed by POSIX, since it explicitly states things like the following
(example taken from ls(1)):

"If the file is a character special or block special file, the
size of the file may be replaced with implementation-defined
information associated with the device in question."

This makes the output of these commands more compact. For example, ls(1)
now uses approximately four columns less. While there, simplify the
column length calculation from ls(1) by calling snprintf() with a NULL
buffer.

Don't be afraid; if needed one can still obtain individual major/minor
numbers using stat(1).

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

MFC r202945:

Fixes for ls(1) long format (-l) output:

- Allow -h option to work if the listing contains at least one device
file.
- Align major and minor device numbers correctly to the size field.

PR: bin/125678

MFC r196712:

Add NFSv4 ACL support to ls(1).

MFC r196773:

Fix regression introduced in r196712 - the 'name' string needs
to be rewritten for each file we want to check ACL on. Without
this change, ls(1) would check only the ACL on the first file
to list.

Fixes for ls(1) long format (-l) output:

- Allow -h option to work if the listing contains at least one device
file.
- Align major and minor device numbers correctly to the size field.

PR: bin/125678
Approved by: trasz (mentor)
MFC after: 1 month

Fix regression introduced in r196712 - the 'name' string needs
to be rewritten for each file we want to check ACL on. Without
this change, ls(1) would check only the ACL on the first file
to list.

Add NFSv4 ACL support to ls(1).

- Import the HEAD csup code which is the basis for the cvsmode work.

Turn a tab into a space. This fixes a misalignment for ls -l.

Tabs Noticed by: Antoine Brodin

Add -D option to specify exact format of date and time output with ls -l.

Avoid a spurious warning for each whiteout found during "ls -lW".

# ls -lW
total 2
-rw-r--r-- 1 root wheel 6 Oct 18 14:46 file1
ls: ./file2: No such file or directory
w--------- 0 root wheel 0 Jan 1 1970 file2

Add a new -U flag to instruct ls to use the birthtime for printing or
sorting.

Submitted by: Andrzej Tobola ato at iem dot pw dot edu dot pl
MFC after: 1 week

/*- or .\"- or #- to begin license clauses.

Remove from the printfns the assumption that dp->list != NULL. Even
if there are no entries, these functions may be called to print the
total number of blocks (0) for consistency's sake.

Use humanize_number(3).

Reminded by: jhb

Treat filenames as multibyte character strings (according to the current
LC_CTYPE setting) when determining which characters are printable.
This is an often-requested feature.

Use wcwidth() to determine the number of column positions a character
takes up, although there are still a few places left where we assume
1 byte = 1 column position, e.g. line-wrapping when handling the -m option.

The error handling here is somewhat more complicated than usual: we do
our best to show what we can of a filename in the presence of conversion
errors, instead of simply aborting.

Remove clause 3 from the UCB licenses.

OK'ed by: imp, core

Stop iterating over ACLs if we've already determined we
will print them (i.e., number of successful calls to acl_get_entry()
exceeds 3). This makes O(1) what was O(num_TYPE_ACCESS_ACLs).

This is a slightly modified version of submitter's patch.

PR: bin/65042
Submitted by: Christian S.J. Peron <maneo@bsdpro.com>

Fixes to output of `ls -lh` for certain file sizes:
1. Sizes in the range 1000 -- 1023 units require four characters width
for the integer; increase the field width to accomodate this.
2. Sizes in the range 9.95 -- 10 units were being displayed as "10.0"
units; adjust the logic to fix this, and now that we've got an extra
character of field width, print fractional units if the size is less
than 99.95 units.
3. Don't display sub-byte precision.

This should mean that the following sizes are displayed:
0B .. 1023B
1.0U .. 9.9U
10.0U .. 99.9U
100U .. 1023U
for values of U in "KMGTPE".

PR: bin/63547
Pointy hat to: cperciva
Approved by: rwatson (mentor)

Fix alignment of size field in `ls -lh` -- the width was being computed
from log[10](largest file size), but when outputting in human-friendly
format the width is always at most 4. (eg. "123K", " 12K", "1.2K".)

PR: bin/59320
Approved by: rwatson (mentor)

Handle realloc() failure correctly.

Fix a bazillion warnings. This makes almost the whole of src/bin/*
WARNS=6, std=c99 clean.

Tested on: i386, alpha

pathconf() and acl_get_file() follow links so they cannot be used to
determine whether a symlink has an ACL. Instead, assume that symbolic
links don't have ACLs and don't bother checking. Avoids spurious
ENOENT warnings when listing directories containing broken symlinks
on filesystems with ACLs enabled.

Pointed out by: rwatson, bde

Use warn() instead of perror() or fprintf() where appropriate.

Print a `+' character after the standard UNIX permission fields in long
listings if the file has an extended ACL (more than the required 3 entries).
This is what Solaris and IRIX do, and what the withdrawn POSIX.2c standard
required.

Reviewed by: rwatson (an earlier version of the patch)

Teach "ls -Z" to use the policy-agnostic MAC label interfaces rather
than the LOMAC-specific interfaces for listing MAC labels. This permits
ls to view MAC labels in a manner similar to getfmac, when ls is used
with the -l argument. Next generation LOMAC will use the MAC Framework
so should "just" work with this and other policies. Not the prettiest
code in the world, but then, neither is ls(1).

Obtained from: TrustedBSD Project
Sponsored by: DARPA, Network Associates Laboratories

Constify, staticify, rationalise types and fix other related warnings.

Print non-printing characters in directory names, as well as file names,
as `?' or `\ooo', depending on whether the -b or -B flags were used.

PR: 43995
MFC after: 1 month

Output "human-readable" values with a non-0 precision where
appropriate. Before this, a 2.9 GB file was misleadingly reported as
"2G". This mostly brings unit_adjust() in line with what is in du(1).

Reviewed by: jmallett
Approved by: nik

Fix a warning of "possibly used before initialisation".

Reviewed by: tjr

Consistently use __FBSDID

Add missing options required by SUSv3:
-m List files across the page, separated by commas.
-p Print a slash after directory names
-x Same as -C but sort across the columns rather than down

Submitted by: Kyle Martin <mkm@ieee.org>

restore missing default case removed in ls.c:r1.57

add break statements to default cases where missing.

Submitted by: bde

#include <time.h> for the definition of time functions instead of
depending on namespace pollution 2 layers deep in <sys/stat.h>.

Removed unused includes.

Use __FBSDID() and clean up the vendor tags.

WARNS=4 fixes, plus a healthy dose of fixes inspired by lint.

o __P has been reoved
o Old-style K&R declarations have been converted to new C89 style
o register has been removed
o prototype for main() has been removed (gcc3 makes it an error)
o int main(int argc, char *argv[]) is the preferred main definition.
o Attempt to not break style(9) conformance for declarations more than
they already are.

LSCOLOURS should be spelt LSCOLORS.

Submitted by: Jordan DeLong <fracture@allusion.net

Restore these files to shiny KNF.

Convert some spaces into tabs that I missed first time around.

Revert most of rev 1.42, to restore KNF style.

Requested by: bde

Add a new flag, -h which when combined with the -l option causes
file sizes to be displayed with unit suffixes; Byte, Kilobyte,
Megabyte, Gigabyte, Terabyte and Petabyte in order to reduce the
number of digits to three or less.

Submitted by: nik

Make it clearer what changed to a user using the old format of LSCOLORS.

Mop up some warnings.

Make the style more consistent throughout the file.

Use the default colour instead if a bad colour specification is given.

Revamp the colour support to allow for bold characters. Colours
are now defined using the characters a-h and A-H for the bold
variants. The old way using 0-7 for the colours still works, but
prints a message asking the user to switch.

PR: bin/27374

Add LOMAC options (the "Z" flag in both cases) to display extra information
in ls(1) and ps(1).

Sponsored by: DARPA, NAI Labs

Silence minor cc warning

Use nl_langinfo instead of %Ef

make sure we do not write out non-printable characters in file names
and symbolic links (by default)

PR: bin/19354
Reviewed by: silence on -current

I broke locale sensitive ordering of date and month in
the long -l output format with the last commit. Fix it
by replacing the "%b %e" strftime format with "%Ef".

Make a note in the manual page that the LANG environment
variable affects the running of ls.

Reviewed by: ache

Don't assume that the output of strftime for "%c" ("national
representation of time and date") won't change in time. Instead
of hard coding the locations of the time elements and hoping that
they don't move use strftime to generate the desired formats in
the first place.

PR: bin/7826

Greately simplify oxtabs fix by simple setting f_notabs for f_color

Honor f_notabs flag even in COLORLS

Back out spaces to TABs replace removing for COLORLS
Rewrite corresponding comment to say what happens in reality with oxtabs
and current terminal column.

Don't replace TABs with spaces for COLORLS because "some terminals get confused"
as comment says. I know no terminal confused by this. If even such terminal
exists, it must be termcap flag for this, not hardcoded in ls.

Staticize more functions

Make signal handler safe - don't use stdio (pointed by bde)
Staticize some color functions
Add yet one tolower() call which is forgotten after check
Don't check for OOPS - not really needed

Get rid of curses completely - use tgoto instead of tparm

Kill yourself in ^C handler, not exit(1) to allow script to sense signal

Cosmetique of prev. optimization - don't use global variable

Don't use curses includes, include termcap.h instead
Don't use curses functions, use tputs instead
Add ^C reaction - reset colors
Optimization - don't turn off colors after EACH file printed.
Fix wrong ctype macro arg type in LSCOLORS parsing

Don't look up the ANSI sequences each time a colour is changed,
this is extremely inefficient, instead write them all down at the
beginning.

The correct sequence to switch colours off is to first use 'op' if
it exists, otherwise use 'oc'. If neither of these exist then we
shouldn't be doing colour with this terminal.

Reviewed by: ache

* Re-implement colour support using termcap's AF and AB capabilities
to manage the ANSI colour sequences. Colour support is disabled
unless the TERM environment variable references a valid termcap.

* Allow optional compilation of the colour support in the Makefile,
defaulting to yes. This allows us to switch it off for fixit
floppies and other mediums where space is an issue and the extra
bloat of statically linking with ncurses isn't acceptable.

* Display a warning if colour is requested with '-G' but support
for it isn't compiled in.

Add colour support to /bin/ls (at a cost of 1056 bytes on my system).

It is not switched on by default and must be enabled with the -G
flag. When using ls -G the output behaviour is modified with ANSI
colour sequences wrapped around filenames to help distinguish file
types. (Colours can be redefined in the LSCOLORS environment
variable as described in the manual page.)

Colour support is silently disabled (if switched on) if stdout
isn't a tty.

Based on: asami's colorls port.
PR: bin/18900 && ports/18616.

Print negative minor numbers in hex. Negative minor numbers are
essentially large unsigned ones, and we already print minor
numbers > 255 in hex.

$Id$ -> $FreeBSD$

Allow env. variable LS_COLWIDTHS to specify minimum column widths,
effectively overriding the dynamically-sized-column feature. This
is mostly useful for non-interactive use, where it may be necessary
to ensure that listings taken at different times have columns that
line-up correctly. I have been assured that at least one large,
well-known program will soon be taking advantage of this. :-)

PR: bin/7011
Submitted by: Joel Ray Holveck <joelh@gnu.org>

o Renamed '-b' (show unprintables in octal) to '-B'

o Added a new '-b' which behaves as in AT&T Unices (show unprintables in
octal, using C escape codes when possible)

o Added '?' to the getopt() string, since the code in the switch considers
it as a valid option.

Added -b option to display unprintables in octal.
PR: 1315

Observe precedence set by Phillippe Charnier in adding an
rcsid.

Remove #if(n)def BSD_4_4_LITE cruft and sccsid -> rcsid.

Import Lite2's src/bin. All files in src/bin are off the vendor branch,
so this doesn't change the active versions.

Revert $FreeBSD$ to $Id$

Make the long-awaited change from $Id$ to $FreeBSD$

This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.

Boy, I'm glad we're not using sup anymore. This update would have been
insane otherwise.

Make ls include the year when displaying times that are more than
6 months into the future. Closes PR# 1657.

Submitted by: Sakari Jalowaara <sja.home.tekla.fi>

Merge Lite2 mods, and -Wall cleaning. undelete(2) cruft
not yet implemented is protected by a define (BSD4_4_LITE)
that should be removed when this call is supported by the
kernel.

Remove unused include of utmp.h from print.c. Closes PR# 486.

Submitted by: Tom Samplonius <tom@haven.uniserve.com>

Print dev minor #'s > 255 in hex.

Reviewed by: jkh, kuku, phk

Change ctime to strftime %c to use national date/time representation

Delete bogus referneces to timezone code internal header file `tzfile.h',
which is no longer bogusly installed in /usr/include.

Remove trailing whitespace.

Reviewed by: phk

You will find enclosed some changes to make gcc -Wall more happy in
/usr/src/bin. Note that some patches are still needed in that directory.

I (Joerg) finished most of Philippe's cleanup. /bin/sh will still
need *allot* of work, however.

Submitted by: charnier@lirmm.fr (Philippe Charnier)

Added $Id$

BSD 4.4 Lite bin Sources