xargs: use getline() instead of fgetln()

This replaces fgetln() with getline(). The main reason for this is
portability, making things easier for people who want to compile these
tools on non-FreeBSD systems.

I appreciate that's probably not the top concern for FreeBSD base tools,
but fgetln() is impossible to port to most platforms, as concurrent
access is essentially impossible to implement fully correct without the
line buffer on the FILE struct. Other than this, many generic FreeBSD
tools compile fairly cleanly on Linux with a few small changes.

Most uses of fgetln() pre-date getline() support (added in 2009 with
69099ba2ec8b), and there's been some previous patches (ee3ca711a898
8c98e6b1a7f3 1a2a4fc8ce1b) for other tools.

Obtained from: https://github.com/dcantrell/bsdutils and
https://github.com/chimera-linux/chimerautils
Signed-off-by: Martin Tournoij <martin@arp242.net>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/893

Remove copyright strings ifdef'd out

We've ifdef'd out the copyright strings for some time now. Go ahead and
remove the ifdefs. Plus whatever other detritis was left over from other
recent removals. These copyright strings are present in the comments and
are largely from CSRG's attempt at adding their copyright to every
binary file (which modern interpretations of the license doesn't
require).

Sponsored by: Netflix

usr.bin: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

xargs: Prevent overflow in linelen calculation if nargs is large.

MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D41023

xargs: disallow -R0 and -L0

Both cases were interpreted as these flags are unset. This meant that
-R0 got converted to -R5 and that -L0 didn't have any effect at all.
Since make at most 0 replacements isn't useful and since call utility
for every 0 lines read doesn't make sense, throw an error for these
two cases.

MFC after: 1 week
Reviewed by: des, kevans
Differential Revision: https://reviews.freebsd.org/D41022

xargs: fix -R so that it accepts negative numbers again

fbc445addf9 converted the parsing of arguments to strtonum but made
the accepted range for -R too restrictive. As documented in the man
page, it should accept negative numbers.

Added a test for this, which was provided by Jose Luis Duran.

Fixes: fbc445addf9
MFC after: 1 week
Reviewed by: des, kevans
Differential Revision: https://reviews.freebsd.org/D41021

xargs: Consistently use strtonum() to parse arguments.

MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D40425

xargs: Fix typo in error message.

MFC after: 1 week
Sponsored by: Klara, Inc.

xargs: improve foundeof check for -E

4aeb63826e0f got it almost correct (we can't use strcmp() here as
current argument isn't guaranteed to be NUL-terminated), but we also
need to check that current argument length is equal to that of eofstr.

PR: 270867
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D39583

xargs: fix exit code when using -P

currently when xargs runs in parallel mode (e.g. -P2), it somtimes
incorrectly returns zero exit code. this commit fix it and also adds
tests.

Reviewed by: mjg
PR: 267110

xargs: terminate if line replacement cannot be constructed

If the line with replacement cannot be constructed xargs should
terminate as documented in the man page

We encounter this error, but gnu/xargs doesn't because they have a much
larger limit for created outputs (~10000 lines).

Reviewed by: oshogbo
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D35574

xargs: compile yesexpr as ERE

yesexpr is an extended regular expression for quite some time now,
use appropriate flag when compiling it.

PR: 238762
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D27509

xargs: add some long options for GNU compatibility

These are low-effort to add, so let's just do it.

Reported by: "LukeShu" on Hacker News (-r / --no-run-if-empty)
MFC after: 1 week

xargs: Fix exit status expression when a child process fails to exec.

PR: 244327
Submitted by: thomas.duffy.99@alumni.brown.edu
MFC after: 1 week

General further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

Renumber copyright clause 4

Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96

Fix type mismatches for malloc(3) and Co.

This is rather pedantic, as for most architectures it holds that
sizeof(type *) == sizeof(type **)

Found by: clang static analyzer
Reviewed by: ed
Differential Revision: https://reviews.freebsd.org/D4722

fix regression in xargs -Px (introduced in r286289) and add regression tests

PR: 202152
Submitted by: jbeich (original), Nikolai Lifanov (final)
Reviewed by: jbeich
Approved by: bapt (mentor)
Differential Revision: https://reviews.freebsd.org/D3330

xargs now takes -P0, creating as many concurrent processes as possible

PR: 199976
Submitted by: Nikolai Lifanov <lifanov@mail.lifanov.com>
Reviewed by: mjg, bjk
Approved by: bapt (mentor)
MFC after: 1 month
Relnotes: yes
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D2616

Convert atoi(3) to stronum(3) which allows to arguments and report proper errors
to the users

Obtained from: OpenBSD

Style(9) changes fo xargs.c

xargs: Before exiting, wait for all invocations of the utility.

This only has an effect with -P, otherwise errors are only detected when the
utility is not running.

Submitted by: Matthew Story

xargs: Remove an unclear comment that only tried to repeat what the code did

Reported by: bde

xargs: Fix comma splice in error message.

Reported by: bde

xargs: If a utility exits with 255 or a signal, write an error message.

If a utility called by xargs exits with status 255 or because of a signal,
POSIX requires writing an error message.

PR: 165155
Submitted by: Matthew Story matthewstory gmail com

Remove the advertising clause from UCB copyrighted files in usr.bin. This
is in accordance with the information provided at
ftp://ftp.cs.berkeley.edu/pub/4bsd/README.Impt.License.Change

Also add $FreeBSD$ to a few files to keep svn happy.

Discussed with: imp, rwatson

Various syntactic tweaks to satisfy style(9). No change to execution.

Submitted by: gcooper@

xargs can be fooled by exiting children that it did not start, causing
it to kick off a new command before the previous has finished, resulting
in corrupted (interleaved) output. It is also fooled by non-exiting
children it did not start, failing to exit until all extraneous children
have exited.

This patch makes xargs keep track of children it starts, ignoring
pre-existing ones.

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

- Import the HEAD csup code which is the basis for the cvsmode work.

NUL terminate buffer from fgetln(3). From fgsch@OpenBSD.

Whitespace after switch.

Inspired by: OpenBSD

Check the entire length of the current argument for the EOF string with -E,
but don't expect a proper ASCII string to exist right here right now, don't
use strcmp(3) which checks for a NUL. As we're still building the argument
up, the next character might be garbage. It would probably be just as safe to
temporarily write a NUL there, but if we've reached the end of argument memory
that might not be the best idea, I think. It's unclear.

Doing it this way seems to meet the most with the original intent.

PR: 85696
Prodded by: stefanf

Add a FreeBSD-specific -S flag which controls the maximum size of an argument
having replacements done in it via -I.

Add -r option for GNU compatibility.

MFC after: 2 weeks

MFC 1.22: Fix -0 vs -L/-I processing, mainly so that 'xargs -0 -I []' will
do something sensible (namely: treat then '\0' as the EOL character, when
deciding what "a line" is for -N). Note that -I implies -N.

MFC after: 3 days

Call setlocale() with category LC_ALL instead of LC_MESSAGES. We need
LC_CTYPE and LC_COLLATE to correctly interpret regular expressions
returned by nl_langinfo(YESEXPR), and it doesn't hurt to include the
rest.

- Use _PATH_TTY and _PATH_DEVNULL macros.
- Don't fail if we can't open /dev/null since this can happen if
xargs is jail'ed or chroot'ed.

These fixes were submitted by Todd Miller from the OpenBSD project.
There was one problem in those fixes that broke -o, which is corrected
here and should be committed to the OpenBSD repo by Todd soon.

MFC in: 3 days

Use waitpid, instead of wait3, which is more portable.

Submitted by: "Todd C. Miller" <Todd.Miller@courtesan.com>
Obtained from: OpenBSD
Reviewed by: tjr

Remove redundant return; from void function.

Submitted by: "Todd C. Miller" <Todd.Miller@courtesan.com>
Obtained from: OpenBSD

Don't shaddow exp(3).

Migrate to a new way of dealing with building from old revisions of
FreeBSD. This method attempts to centralize all the necessary hacks
or work arounds in one of two places in the tree (src/Makefile.inc1
and src/tools/build). We build a small compatibility library
(libbuild.a) as well as selectively installing necessary include
files. We then include this directory when building host binaries.

This removes all the past release compatibilty hacks from various
places in the tree. We still build on tip of stable and current. I
will work with those that want to support more, although I anticipate
it will just work.

Many thanks to ru@, obrien@ and jhb@ for providing valuable input at
various stage of implementation, as well as for working together to
positively effect a change for the better.

Simplify compatibility ifdef.

When -o isn't specified, open /dev/null as stdin.

Suggested by: jhb

Add a new -o option to tell xargs(1) to reopen /dev/tty as stdin in
the child process, before executing the command. This is very useful
when you do stuff like ``find ... | xargs interactive_application''.
Without -o, the application would inherit the pipe as its stdin, and
you thus lose any control over it.

This flag has been carefully chosen to not conflit with other options
of other xargs utilities like GNU xargs.

Reviewed by: jmallett

Extend our -R extension which sets the number of arguments in which -I will
replace to support magic values less than zero, which mean to just go nuts
and expand as many as we want.

MFC after: 2 weeks

De-typo usage string.

Add the -P option which executes multiple copies of the specified utility
in parallel. Idea from GNU xargs.

Take __FreeBSD_version into account when BOOTSTRAPPING.

Fix typos; each file has at least one s/seperat/separat/
(I skipped those in contrib/, gnu/ and crypto/)
While I was at it, fixed a lot more found by ispell that I
could identify with certainty to be errors. All of these
were in comments or text, not in actual code.

Suggested by: bde
MFC after: 3 days

Declare environ as char **environ like in environ(7), not char *environ[].
This corrects a problem whereby xargs could not walk the environment table
to count the amount of space it used, and treated it as if it were empty.
This problem was introduced in rev 1.15.

MFC after: 2 days

Stylistic nit:

main()'s argv argument is char*[], and functions that may inherit that arg
should use char*[] as well.

Err, duh, free(3) doesn't set its argument to NULL... Fix realloc of a freed
chunk.

Restructure sanity checks so that -I and -J just override eachother, rather
than triggering a usage(). Allow -R and -I to be specified in any order, and
thus change how -R checks for -I not being given and triggering a usage().

Partially requested by: gad

Allow the input line to be NULL, and teach strnsubst() that NULL means to use
a nil-string.

Simplify prerun() in the case where there are no arguments. Can't do -I or -J
substitution if `utility' is not specified and we're using the buildin echo;
It has no arguments for us to abuse.

xargs.c,v 1.33 broke the upgrade path from old versions of 4.x and 5.0.

-I and -J both set the same variable, replstr, to the string to replace with
a line of input, and both work differently, so prevent them from both being
passed to xargs(1).

Modify the -p implementation to use a user's locale, so they can respond to
the prompt in their native language.

Also make the prompt fit what POSIX asks for (?...).

This should not affect use of -p with yes(1) [as every locale I know of matches
'y' as YESEXPR as well], but that's what -t is for anyway. -p is meant to be
really used interactively.

Submitted by: tjr, jmallett

Fix a typo.

Submitted by: Carl Schmidt <cschmidt@slackerbsd.org>

Wrap an obscenely long line while I'm here.

In an effort to make this utility easier to work with at a source level, move
out the parse loop to a seperate function, and move local variables around as
is needed. To keep the state of some variables and keep from having huge arg
lists to functions, make a bunch global.

Change use of err(3) in situations where malloc(3) will be setting errno to
errx(3) since assuming malloc(3) sets errno is non-portable, and provides no
more useful information in the context of FreeBSD.

Submitted by: bde (err/errx), Carl Schmidt <cschmidt@slackerbsd.org> (some
of the movement of the input loop to a function)
Approved by: src/tools/regression/usr.bin/xargs

Grouping changes.

Submitted by: Carl Schmidt <cschmidt@slackerbsd.org>

Don't use pointers as booleans.

Replace calloc(3) with malloc(3), and set the tail of the
argument list to NULL, to terminate the arguments passed
to execvp(2).

Thanks to: bde (for pointing out some missing parens)

And with apologies to Ozzy Osbourne:

On his way to dinner
It took him by surprise
When an email from bde
Said his code was full of lies

Style (indentation)
And his bracing were all wrong
He casted to size_t
When he should have cast to long

I don't mind

Single letter identifiers
Unwrapped Line
Over 80 chars
Far over 80 chars

Who can we get to send diffs
We need Bruce D. Evans
Shows you comparason to his
We need Bruce D. Evans again

Move substitution out to a function, prerun(), and replace use of run() with
it. It handles everything (right now) that needs done before run(), except
the -J case, because while that would be easy, I don't quite understand -J.

Reviewed by: src/tools/regression/usr.bin/xargs

Get rid of bogus holder for realloc(3).

Rewrite the loop that handles substitution in the -I case, and try to make
it easier to understand. Making it easy to understand isn't all that easy,
so litter the code with comments so some other poor soul can come along some
day and work on this if they see fit to do so. Avoid calling strlen(3) to
check for a nil-string, when we can just check for *str=='\0'.

Approved by: src/tools/regression/usr.bin/xargs/

Rename `lflag' to `Lflag' to match the option's case.

Revert revision 1.22. I incorrectly modified the Berkeley identifiers.

Requested by: bde, mike

__COPYRIGHT() and __SCCSID().

An explicit cast to size_t for an inline integer, since the two are not the
same on Alpha and lint(1) pointed that out.

lint(1) on the same architecture pointed out how silly a cast to (u_int) to
malloc(3)'s argument was. Change that to size_t.

Make -J a proper flag internal to the code (rather than just check for
use of replstr and lack of Iflag), and add -R, which when given with
-I controls the number of arguments on which replacement will be done.

Some people happen to think it's idiotic to limit to 5 arguments, so
let the user override it if they like.

Rework some of the -I support so I can't find a way to make xargs(1) core or
corrupt memory. Simplifies the code in one or two places, also removes some
code that looks like it was bogus or incomplete.

Update strnsubst to have one or two extra conditions which maybe would make
it more efficient, or at least more versatile. This is likely a no-op.

In the case where we can't open /dev/tty, fall back to -t behaviour.

Submitted by: fenner

Compound two fprintf(3)s into 1 using string concatenation.

After 3 months...

Merge xargs(1) with that of xMach.

Bring in xargs(1) changes to add -L and -I as per the Single Unix Specification
version 3. Proper exit status numbers are implemented, and the manual page has
been updated to reflect reality.

The code has been ANSIfied, and a new file has been added to xargs(1) to do the
substring substitution as SUSv3 requires.

Traditional behaviour should not be affected, use of -J should be deprecated
in favor of the more portable -I (though -J has been left, for now).

Submitted by: me, tjr (the exit status stuff)
Obtained from: xMach

ANSIfy: Function declarations and prototypes, use of environ(7).

Reviewed by: mike
Approved by: mike

Change back to using vfork() now that execvp() is vfork()-safe. If execvp()
fails, errno is saved to a volatile variable that the parent later inspects.

PR: bin/34898
Submitted by: Tim J. Robbins <tim@robbins.dropbear.id.au>
MFC after: 1 week

Leave the `copyright' variable exposed, since the main purpose of it
is to put a copyright in the object file.

Submitted by: bde

Fix vendor ID (mostly obtained from Rev 1.1).

WARNS=2 fixes, remove register keyword, use __FBSDID()

Add a -J replstr option that allows the user to tell xargs to insert
the data read from standard input at a specific point in the command
line arguments rather than at the end.

Submitted by: dd, gad
Reviewed by: gad, brian

$Id$ -> $FreeBSD$

xargs was spliting the input in a very different way from everyone else,
and was also a bit inconsistent: leading blanks, or any double blanks
generated empty arguments, but a trailing blank did not.

PR: bin/2630, bin/10914
Submitted by: Arne Henrik Juul <arnej@imf.unit.no>

Calls one or more of malloc(), warn(), err(), syslog(), execlp() or
execvp() in the child branch of a vfork(). Changed to use fork()
instead.

Some of these (mv, find, apply, xargs) might benefit greatly from
being rewritten to use vfork() properly.

PR: Loosely related to bin/8252
Approved by: jkh and bde

Remove compile time dependency on ARG_MAX.

This fix only removes the dependency on compile time constants. The code
has other (old) problems that need to be addressed.

PR: 1791
Reviewed-by: bde, tegge

Use err(3) instead of local redefinition.

compare return value from getopt against -1 rather than EOF, per the final
posix standard on the topic.

Reviewed by: Warner Losh <imp@village.org>
Submitted by: Marc Slemko <marcs@znep.com>
Obtained from: OpenBSD

Add -0 for reading the results of find -0.

Make xargs take into account the bytes occupied by the environment.

Original by: peter

BSD 4.4 Lite Usr.bin Sources