usr.sbin: Remove repeated words

Signed-off-by: Elyes Haouas <ehaouas@noos.fr>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/887

newsyslog(8): Add support of specifying compression method in configuration file.

Administrators can now specify a global compression method directly
at the beginning of the newsyslog.conf file, for example:

<compress> none

Relnotes: yes
Reviewed by: dvl
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D43174

newsyslog(8): Add option to globally override compression method.

Historically, newsyslog compressed rotated log files to save disk space.
This was useful in the early days. However, with modern file systems like
ZFS offering native compression, and with the availability of larger hard
drives, the benefits of additional compression have become less significant.
This is particularly true considering the inconvenience of decompressing
log files when searching for specific patterns.

Additionally, the original implementation of compression methods was not
future-proof. As a result, we have redefined the J, X, Y, Z flags to
signify "treat the file as compressible" rather than "compress the file
with that specific method."

A new command-line option, -c, has been introduced to allow overriding
these settings in a more future-proof way. The available choices are:

* none - do not compress, regardless of flag.
* legacy - historical behavior: J=bzip2, X=xz, Y=zstd, Z=gzip.
* bzip2, xz, zstd, gzip - apply the specified compression method.

Currently, the default is set to 'legacy' to preserve historical behavior.
However, our intention is to change this default to 'none' in FreeBSD 15.0.

Additionally, this update changes the default settings for zstd to use
multithreading and long-range options, better aligning with its intended
use.

Inspired by D42961 .

Reviewed by: debdrup (earlier version, mdoc(7))
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D43165

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

Mark usage function as __dead2 in programs where it does not return

In most cases, usage does not return, so mark them as __dead2. For the
cases where they do return, they have not been marked __dead2.

Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/735

newsyslog(8): Implement a new 'E' flag to not rotate empty log files

Based on an idea from dvl's coworker, László DANIELISZ, implement
a new flag, 'E', that prevents newsyslog(8) from rotating the empty
log files. This 'E' flag ist mostly usable in conjunction with 'B'
flag that instructs newsyslog(8) to not insert an informational
message into the log file after rotation, keeping it still empty.

Reviewed by: markj, ian, manpages (rpokala)
Approved by: markj, ian, manpages (rpokala)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D28940

newsyslog(8): make configuration parser more robust.

Currently newsyslog supports <include> directive that is used
in our default /etc/newsyslog.conf in the following form:

<include> /usr/local/etc/newsyslog.conf.d/*

While this is suitable for ports installing their own rules
for logs rotation, this also makes newsyslog break entire
processing of all files if it encounters single line it cannot parse.
This includes lines referring to nonexistent username/group for log
ownership, so newsyslog stops calling errx() function in the parser.

With this fix, newsyslog uses warnx() instead of errx() in such cases
to print a warning, recover gracefully and continue with execution.

Among other cases, this unbreaks initial creation of log files
having flag "C" at boot time (newsyslog -CN). This is most important
for systems having RAM-based /var file system like nanobsd(8)-based
that rely on newsyslog to bring system log files into existence.

MFC after: 1 month

newsyslog: Fix stack corruption when initializing a zipwork structure.

This happens when compressing a previously uncompressed already-rotated
file, as happens when handling the 'p' flag in newsyslog.conf. The file
name is stored in a flexible array member, so these structures cannot be
stack allocated.

Also make sure that we call change_attrs() and do_zipwork() in dry-run
mode; they handle this properly, contrary to the commit log message for
r327451.

CID: 1008168
Github PR: https://github.com/freebsd/freebsd/pull/427
MFC after: 2 weeks
Submitted by: Radek Brich (original version)

Fix several Coverity-detected issues in newsyslog.

- CID 1394815, CID 1305673: Dereference before null check - memory was
allocated and the allocation checked for NULL with a call to errx()
if it failed. Code below that was guaranteed that the pointer was
non-NULL, but there was another check for NULL at the exit of the
function (after the memory had already been referenced). Eliminate
the useless NULL check.

- CID 1007452: Resource leak - Storage intended to be allocated and
returned to the caller was never freed. This was the result of a
regression in the function signature introduced in r208648 (2010)
(thanks for that find, @cem!). Fixed by altering the function
signature and passing the allocated memory to the caller as
intended. This also fixes PR158794.

- CID 1008620: Logically dead code in newsyslog.c - This was a direct
result of CID 1007452. Since the memory allocated as described there
was not returned to the caller, a subsequent check for the memory
having been allocated was dead code. Returning the memory
re-animates the code that is the subject of this CID.

- CID 1006131: Unused value - in parsing a configuration file, a
pointer to the end of the last field was saved, but not used after
that. Rewrite to use the pointer value. This could have been fixed
by avoiding the assignment altogether, but this solutions more
closely follows the pattern used in the preceding code.

PR: 158794
Reported by: Coverity, Ken-ichi EZURA <k.ezura@gmail.com> (PR158794)
Reviewed by: cem, markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D19105

Revert r344389 & r343906 - Fix Coverity errors in newsyslog

It was pointed out that a couple of the "memory leak" CIDs that I
fixed were arguably Coverity errors rather than errors in the
newsyslog code and the cure was worse than the disease. Revert both
changes. The first change, which included fixes for other Coverity
errors, will be re-worked to omit the troublesome changes and then
re-committed with the remaining fixes.

Reported by: bde
Sponsored by: Dell EMC Isilon

Complete fix for CID 1007454, CID 1007453: Resource leak in newsyslog

The result of a strdup() was stored in a global variable and not freed
before program exit. This is a follow-up to r343906. That change
attempted to plug these resource leaks but managed to miss a code path
on which the leak still occurs. Plug the leak on that path, too.

MFC after: 3 days
Sponsored by: Dell EMC Isilon

Fix several Coverity-detected issues in newsyslog.

- CID 1394815, CID 1305673: Dereference before null check - memory was
allocated and the allocation checked for NULL with a call to errx()
if it failed. Code below that was guaranteed that the pointer was
non-NULL, but there was another check for NULL at the exit of the
function (after the memory had already been referenced). Eliminate
the useless NULL check.

- CID 1007454, CID 1007453: Resource leak - The result of a strdup()
was stored in a global variable and not freed before program exit.

- CID 1007452: Resource leak - Storage intended to be allocated and
returned to the caller was never freed. This was the result of a
regression in the function signature introduced in r208648 (2010)
(thanks for that find, @cem!). Fixed by altering the function
signature and passing the allocated memory to the caller as
intended. This also fixes PR158794.

- CID 1008620: Logically dead code in newsyslog.c - This was a direct
result of CID 1007452. Since the memory allocated as described there
was not returned to the caller, a subsequent check for the memory
having been allocated was dead code. Returning the memory
re-animates the code that is the subject of this CID.

- CID 1006131: Unused value - in parsing a configuration file, a
pointer to the end of the last field was saved, but not used after
that. Rewrite to use the pointer value. This could have been fixed
by avoiding the assignment altogether, but this solutions more
closely follows the pattern used in the preceding code.

PR: 158794
Reported by: Coverity, Ken-ichi EZURA <k.ezura@gmail.com> (PR158794)
Reviewed by: cem, markj
MFC after: 1 week
Sponsored by: Dell EMC Isilon

Fix age_old_log() after r337468.

We can no longer use sizeof() to get the path buffer's size. Apply
a straightforward fix for now with the aim of MFCing soon.

PR: 233633
Submitted by: Katsuyuki Miyoshi <katsu@miyoshi.matsuyama.ehime.jp>
MFC after: 3 days

newsyslog(8): Reject configurations that specify setuid or executable logs

Prevent some classes of foot-shooting that may result in permissions
problems.

Reviewed by: dab, delphij, vangyzen (earlier version)
Relnotes: yes (behavior change)
Sponsored by: Dell EMC Isilon
Differential Revision: D16831

Simplify compression code.

- Remove the compression suffix macros and move them directly into the
compress_type array.
- Remove the hardcoded sizes on the suffix and compression args arrays.
- Simplify the compression args arrays at the expense of a __DECONST
when calling execv().
- Rewrite do_zipwork. The COMPRESS_* macros can directly index the
compress_types array, so the outer loop is not needed. Convert
fixed-length strings into asprintf or sbuf calls.

Submitted by: Dan Nelson <dnelson_1901@yahoo.com>
Reviewed by: gad
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D16518

Fix a flag collision introduced in r327451.

PR: 230350
MFC after: 3 days

Remove ARGS_NUM, accidentally left over after r337050.

Don't hard-code the number of compression utility arguments.

The zstd invocation constructed by newsyslog contains one more parameter
than invocations for the other supported compression utilities. However,
the maximum number of arguments was hard-coded, leading to an
out-of-bounds array access when using zstd compression.

newsyslog: fix compression arglist construction

Reuse of the index variable in two nested loops resulted in only the first
argument in the list being used (fine for gzip, not fine for zstd). Also
add tests for xz and zstd, and fix the COMPRESS_SUFFIX_MAXLEN macro.

Submitted by: dnelson_1901_yahoo.com
Differential Revision: https://reviews.freebsd.org/D16509

Use the FQDN in the newsyslog log message when RFC 5424 is enabled.

The RFC 5424 spec mentions that logging FQDNs over short hostnames is
preferred. Alter this code, so that the hostname doesn't get truncated
on startup. Keep track of the length of the short hostname, so that
fprintf() can do the truncation where necessary.

MFC after: 1 month

newsyslog: implement 'p' flag

Implement the 'p' flag for newsyslog from NetBSD. This flag results in
the first log file for a given file to not be compressed.

While here, don't change file attributes during a no-op run

PR: 162798
Submitted by: heas@shrubbery.net
MFC After: 1 month

newsyslog: Fix issues after r326616

When building the command to execute for compression, newsyslog was modifying
the generic arguments array instead of its own copy.
Meaning on the second file to compress with the same arguments, the command line
was not the one expected.
Fix it by creating one copy of the arguments per execution and modifying that
copy.

While here, print the command line executed in verbose mode.

Reported by: many

Really fix typo and improve wording of the comment

Fix typo

Reported by: danfe

Allow newsyslog to execute compression commands which
have a semantic different than the traditional gzip(1)

This is done to allow to use zstd(1) as a compression tool without
having to patch it to change its default behavior.

Add newsyslog capability to write RFC5424 compliant rotation message.

This modification adds the capability to newsyslog to write the
rotation message in a format that is compliant with RFC5424. This
capability is enabled on a per-log file basis through a new value
("T") in the flags field in newsyslog.conf. This is useful on systems
that use the RFC5424 format for log files so that the rotation message
format matches that of the other log messages. There has been recent
mention of adding an RFC5424 compliant mode to syslogd and at least
one alternative system log daemon (rsyslogd) that already has the
capability to use that format.

Reviewed by: vangyzen, ngie
Approved by: vangyzen (mentor)
MFC after: 2 months
Relnotes: yes
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D10253

Add a new Y flag to newsyslog.conf

This makes newsyslog use zstandard to compress log files.

Given Z is already taken for gzip and zstandard compression level stands in
between gzip and xz (which has the X flag) chosing Y sounds ok :)

Fix improper use of "its".

Sponsored by: Dell EMC Isilon

Properly patch up dirname()/basename() calls to not clobber ent->log.

It turns out that we had a couple of more calls to dirname()/basename()
in newsyslog(8) that assume the input isn't clobbered. This is bad,
because it apparently breaks log rotation now that the new dirname()
implementation has been merged.

Fix this by first copying the input and then calling
dirname()/basename(). While there, improve the naming of variables in
this function a bit.

Reported by: Ryan Steinmetz, gjb
Reviewed by: bdrewery, allanjude
Differential Revision: https://reviews.freebsd.org/D7838

Clean up use of basename() and dirname().

Pull copies of the input pathname string before calling basename() and
dirname() to make this comply to POSIX. Free these copies at the end of
this function. While there, remove the duplication of the 's' ->
'logfname' string. There is no need for this.

newsyslog: Eliminate unnecessary sleep(10) when -R and -s are specified

After going through the signal work list, during which do_sigwork()
is called and essentially does nothing because -s and -R were
specified on the command line, newsyslog will sleep for 10 seconds
as the (verbose) code says: "Pause 10 seconds to allow daemon(s)
to close log file(s)".

However, the man page verbiage for -R (and -s) seems quite clear
that this sleep() is unnecessary because the daemon was expected
to have already closed the log file before calling newsyslog.

PR: 210020
Submitted by: David A. Bright <david_a_bright@dell.com>
MFC after: 1 week
Sponsored by: Dell Inc.
Differential Revision: https://reviews.freebsd.org/D6727

newsyslog: treat 'c' flag in the config as 'C'

When -C was introduced in r114137 the plan was to have -C and -c being used for
"create" due to a typo in FreeBSD <= 4.8 a temporary compatibility hack has been
added to make -c being like -G aka GLOB and a warning was issued for the user to
be aware of the futur change for -c.

12 years later it is more than time to remove that hack and finish the what was
intent in r114137

Submitted by: Alexandre Perrin <alex@kaworu.ch>
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D4000

newsyslog.conf: allow to configure the signal using the signal name.

Submitted by: Alexandre Perrin <alex@kaworu.ch>
MFC after: 1 week
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D3961

Fix a ton of speelling errors

arc lint is helpful

Reviewed By: allanjude, wblock, #manpages, chris@bsdjunk.com
Differential Revision: https://reviews.freebsd.org/D3337

If we fail to send a signal after rotation, print the pidfile from which
the corresponding PID was obtained.

PR: 194143
Submitted by: Lyndon Nerenberg <lyndon@orthanc.ca>
MFC after: 1 week

Fix newsyslog(8) to use the size of the file instead of the blocks it
takes on disk, as advertised in newsyslog.conf(5).

This fixes newsyslog(8) on ZFS with compression enabled to not have
large files compared to the expected rotation size.

# grep remotes/messages /etc/newsyslog.conf
/var/log/remote/messages root:info 640 5 500 * JC
# ls -alh /var/log/remote/messages
-rw-r----- 1 root info 3.2M Jan 31 20:02 /var/log/remote/messages
# newsyslog -vN|grep remote/messages
/var/log/remote/messages <5J>: size (Kb): 464 [500] --> skipping
# stat -f "st_size: %z st_blocks: %b" /var/log/remote/messages
st_size: 3372627 st_blocks: 928
# zfs get -H compressratio zroot/syslogs
zroot/syslogs compressratio 3.77x -

With fix:

# newsyslog -v | grep remote/messages
/var/log/remote/messages <5J>: size (Kb): 3338 [500] --> trimming log....

Approved by: bapt (mentor)
PR: docs/150877
Reported by: Joshua Isom <jrisom@gmail.com>
MFC after: 2 weeks

Initialize the struct tm before handing it to strptime(3).

PR: 183550
MFC after: 2 weeks

Fix -Wunsequenced

Submitted by: dt71@gmx.com

We want to stat the archived log file rather than the logfile itself.

PR: bin/179122
Submitted by: Oliver Fromme <olli@secnetix.de>
MFC after: 3 days

Some filesystems (NFS in particular) do not fill out the d_type field when
returning directory entries through readdir(3). In this case we need to
obtain the file type ourselves; otherwise newsyslog -t will not be able to
find archived log files and will fail to both delete old log files and to
do interval-based rotations properly.

Reported by: jilles
Reviewed by: jilles
MFC after: 2 weeks

Fix interval-based rotations when the -t flag is used. In this case, find
the most-recently archived logfile and use its mtime to determine whether
or not to rotate, as in the non-timestamped case.

Previously we would just try to use the mtime of <logfile>.0, which always
results in a rotation since it generally doesn't exist in the -t case.

PR: bin/166448
Approved by: emaste (co-mentor)
Tested by: Marco Steinbach <coco executive-computing.de>
MFC after: 2 weeks

Rename the run_cmd field to sw_runcmd to make it consistent with the
other fields in struct sigwork_entry.

Approved by: rstone (co-mentor)
MFC after: 1 week

Ensure that newsyslog -n prints the correct message for a rotation rule
that uses the 'R' flag.

Reviewed by: gad
Approved by: rstone (co-mentor)
MFC after: 1 week

When the 'R' flag is used with a newsyslog.conf entry, some fields of
the corresponding struct sigwork_entry were left uninitialized,
potentially causing an early return from do_sigwork(). Ensure that these
fields are initialized, and handle the 'R' flag properly in
do_sigwork().

PR: bin/175330
Reviewed by: gad
Approved by: rstone (co-mentor)
MFC after: 1 week

Make sure to update the mtime of a logfile after archiving it. This
ensures that the next rotation happens at the correct time when using
interval-based rotations.

PR: bin/174438
Reviewed by: gad
Approved by: rstone (co-mentor)
MFC after: 1 week

Have -n imply -r, since dry-run mode obviously doesn't require root
privileges.

Approved by: rstone (co-mentor)
MFC after: 1 week

Fix a typo in an error message.

Approved by: rstone (co-mentor)
MFC after: 1 week

More -Wmissing-variable-declarations fixes.

In addition to adding missing `static' keywords:
- bin/dd: Pull in `extern.h' to guarantee consistency with source file.
- libexec/rpc.rusersd: Move shared globals into an extern.h.
- libexec/talkd: Move `debug' and `hostname' into extern.h.
- usr.bin/cksum: Put counters in extern.h, as they are used by ckdist/mtree.
- usr.bin/m4: Move `end_result' into extern.h.
- usr.sbin/services_mkdb: Move shared globals into an extern.h.

We don't need to check the result of sending signal when -R option is
specified.

Submitted by: Ilya A. Arkhipov
MFC after: 1 week

Hide DIR definition by making it an opaque struct typedef.

Introduce dirfd() libc exported symbol replacing macro with same name,
preserve _dirfd() macro for internal use.

Replace dirp->dd_fd with dirfd() call. Avoid using dirfd as variable
name to prevent shadowing global symbol.

Sponsored by: Google Summer Of Code 2011

Don't run through time checks when entry is definitely oversized. This
leads to newsyslog rotating on (size OR time) if both are specified.

PR: 100018, 160432

Further fix a typo and spelling classic correctly in function names,
too.

Submitted by: Ben Kaduk <minimarmot@gmail.com>

Replace index() and rindex() calls with strchr() and strrchr().

The index() and rindex() functions were marked LEGACY in the 2001
revision of POSIX and were subsequently removed from the 2008 revision.
The strchr() and strrchr() functions are part of the C standard.

This makes the source code a lot more consistent, as most of these C
files also call into other str*() routines. In fact, about a dozen
already perform strchr() calls.

Spelling fixes for usr.sbin/

- Remove extraneous null ptr deref checks
- Fix memory leak

Submitted by: Slono Slono <slonoman2011@yandex.ru>
Approved by: jhb
MFC after: 1 week

Add new modifier - "R", when it is specified the path to pid file
will be considered as a path to a binary or a shell script to be executed
after rotation has been completed instead of sending signal to the process
id in that file.

Sponsored by: Sippy Software, Inc.
From the: FreeBSD hacking lounge at BSDCan

Fix an old bug in newsyslog where we kept one log file more than was
requested in newsyslog.conf. This was only the case using the non-time
based filenames (.0, .1, .2 etc.).

The change also makes newsyslog clean clean up the old extra logfile so
users don't end up with a single stale logfile which won't be rotated
out.

This change also cleans up some code a bit to avoid more copy / paste
code and removes some old copy / paste code in the process.

PR: bin/76697
MFC after: 2 weeks

Sync manpage's SYNOPSIS with program's usage.

Make code more friendly to the non-C99 compilers - don't allocate
local variables in the `for' loop declaration. This allows trunk
newsyslog.c to be compiled on 7.x. This change should be no-op from
the functional POV.

Add xz(1) support to newsyslog.
Rewrite and simplify logfile compression code.

Approved by: gad
MFC after: 3 weeks

Make -S functional.

MFC after: 1 week

Fix a typo in a comment.

MFC after: 3 days

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Add a -S switch to override the default syslog pid file. This can be useful
if syslogd's -P switch or a syslogd alternative is being used.

MFC after: 3 weeks

Add support for creating the archived log filenames using a time-stamp
instead of the traditional simple counter.

Using the time-stamp based file-names, once a log file is archived, it
will not change name until it is deleted. This means that many backup
systems will only perform one backup of the archived log file, instead
for performing a new backup of the logfile upon each logfile rotation.

This implementation is separate from the patches in the mentioned PR,
as I wasn't aware of the existence of the PR until after I had
implemented the same functionality as the patches in the PR provide.
Unlike the PR, this new code does honor the 'log count' in
newsyslog.conf so old logfiles are deleted. This new code does not
currently support never deleting the archived logfiles.

PR: bin/29363
MFC after: 3 weeks

Add file include processing for newsyslog.

Format for the include line in /etc/newsyslog.conf is:
<include> /etc/defaults/newsyslog.conf

Other notes of interest:
Globbing is supported in <include> statements.
Properly detect circular include loop dependencies.

Reviewed by: gad@
Approved by: wes@ (mentor)
MFC after: 2 months

Convert newsyslog to using queue(3) macros instead of a home rolled version.

Reviewed by: gad@
Approved by: wes@ (mentor)
MFC after: 2 months

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

MFC r202668+r200806:

Don't consider non-existence of a PID file an error, we should be able
to proceed anyway as this most likely mean that the process has been
terminated. [1]

Add a new option, -P, which reverts newsyslog(8) to the old behavior,
which stops to proceed further, as it is possible that processes which
fails to create PID file get screwed by rotation. [2]

PR: bin/140397
Submitted by: Dan Lukes [1]
Requested by: stas [2]

Add a new option, -P, which reverts newsyslog(8) to the old behavior,
which stops to proceed further, as it is possible that processes which
fails to create PID file get screwed by rotation.

Requested by: stas
MFC after: 2 weeks
X-MFC with: r200806

Don't consider non-existence of a PID file an error, we should be able
to proceed anyway as this most likely mean that the process has been
terminated.

PR: bin/140397
Submitted by: Dan Lukes <dan obluda cz>
MFC after: 1 month

- Import the HEAD csup code which is the basis for the cvsmode work.

Fix two typos.

Use same signedness for i and matched_c. This should make
newsyslog(8) WARNS=6 clean.

Remove mention of the `W' flag, which has been turned to no-op by the
neworder change. Keep the option in a config file parser, to not
violate POLA.

MFC after: 2 weeks

Remove the last three calls to strncpy(), two of which would have
been bugs if the source had ever been too big for the destination.

MFC after: 3 weeks

Remove the "oldorder" processing. The "neworder" processing has
been the default for two years now, without any problems reported.

MFC after: 3 weeks

Improve error-handling related to the fork() done to compress files after
they have been rotated. Among other things, use warnx() instead of warn()
for some messages where the value if errno is irrelevant to the problem
being reported.

MFC after: 5 days

Fix logic error which causes <null> to be printed instead of the
actual file name in error message.

MFC After: 2 weeks

Add the -N option to not rotate any files. This is to be used in
cojunction with -C and is used by /etc/rc.d/newsyslog.

I forgot that this was in my perforce tree and not my running system and
thus committed a non-working newsyslog script.

Reported by: des
Pointy hat: brooks

Because the `permission' field in conf_entry is intended to be used as
parameter 2 in chmod(2), which is a mode_t (and in turn a __uint_16_t),
it's more likely that it should be defined as an unsigned variable.

This commit should make newsyslog WARNS=6 clean, but don't bump the knob
until I have a universe build.

MFC After: 1 month

Mention "-d directory" in usage().

Pointy hat to: brooks

Don't prepend the directory specified by -d when the file is a relative
path. Doing so makes no sense. I'm not sure allowing relative paths
makes sense either, but I'm not going to break that now.

Add a new -d argument which is used to specify an alternate root for log
files similar to DESTDIR in the BSD make process. This only affects log
file paths not config file (-f) or archive directory (-a) paths.

Nothing says that /var/log can't be not a directory but a symbolic link
to a directory. Therefore, use stat(2) instead of lstat(2) to check if
/var/log exists.

MFC after: 7 days

Fix the format-string in a call to err(). It was causing a warning if
compiled on 4.x-stable.

Fix the check for a "duplicate filename to compress", so that we're checking
the *filename* and not the pid_file(!). Stupid brain-fault on my part.
This could cause a segfault under -neworder if newsyslog had to rotate
multiple files, and later ones had specifed the 'N' flag.

Bug first reported by: le
MFC after: 3 days

Switch to using the "neworder" for rotating log files, by default. The
main advantage of this is that daemon's are only signalled once per run,
instead of once for each file that is rotated.

MFC after: 2 weeks

Add an "oldorder" option, so that when the default changes to "neworder",
people have a way to drop back to the previous logic.

MFC after: 13 days

In "neworder" processing, reduce the delay between signals to separate
processes, and balance that by adding a 10-second delay after all the
processes have been signaled. Also improvement a few messages printed
with `-n' or `-v' processing (mostly signal-related messages).

MFC after: 13 days

Major re-ordering of the steps that newsyslog will use when processing
files to rotate. The new order will first rotate all files that need
to be rotated, and then send a single signal to each process which
needs to be signaled, and finally it will compress all the files which
were rotated.

This means daemons will be signaled once per run of newsyslog, instead
of once per file rotated. Also, files will be compressed in order of
file-size (smallest to largest). Also, it waits for each file to be
completely compressed before starting the next one (effectively as if
the 'w' flag is specified for all entries in newsyslog.conf). This
avoids the situation of having 10 gzip's going at the same time (each
with a log.0 and a log.0.gz file active), and it also means that file
attributes can be reliably set on files after they are compressed.

NOTE: This commit does define NEWORDER (which you could get rid of if
you really don't trust this), but it does not flip the "-D neworder"
switch. So, at the moment none of these changes happen unless you
request them (perhaps by adding '<debug> neworder' in newsyslog.conf).

PR: bin/25070 inspired some parts of this
Submitted by: parts from bin/25070 done by Helge Oldach
MFC after: 14 days

A variety of minor changes. Allow users to set a debugging option via
the newsyslog.conf file. Rename one size-related variable, and move
another one from the stack into conf_entry. Add a routine to change
file-attributes (chown, chmod, chflags), instead of having several
places doing the same sequence of system-calls. A few cosmetic/style
changes.

These should not effect any users. Most of these probably look
pointless, but they are the "insignificant parts" of a much larger
update that I'll be committing soon. Doing these as a separate update
should make that update easier to read.

MFC after: 14 days

Style-istic fix to a number of #define's that were not followed by a tab...

MFC after: 16 days

Add a 'D' flag that can be specified on entries in newsyslog.conf.
If specified, the matching log files will have the NODUMP flag set
on them after they are created.

Submitted by: Sean Eric Fagan
MFC after: 16 days

Change standard processing to use the newer createlog() routine
that had been written some months ago for other processing. This
should get rid of a few subtle situations where an existing log
file would not exist (for a short time) while it is being rotated.

MFC after: 16 days

Improved versions of the is*ch() and tolowerch() macros that I like to use.

When rotating some "blah.log" file, make sure that a chmod and
(if requested) a chown is done on the "blah.log.0" file.

PR: bin/67137
Submitted by: jeh
MFC after: 10 days

Do not :-terminate err(3) string, one will be added anyway.

Restructure the time processing routines, mainly to fix up the
"will trim at" message printed when the user requests '-v'. The
previous code would often print the wrong time, such as:
On Sept 22, run: newsyslog -nv /var/log/wtmp
And see: will trim at Mon Sep 1 05:00:00 2003
correct msg: will trim at Wed Oct 1 05:00:00 2003

MFC after: 20 days

Move the parse8601 and parseDWM routines into a new ptime.c file. The
only code-change is to add a "next_time" parameter to both routines (and
that is not used yet). A later update will make "next_time" more useful.

MFC after: 20 days

Switch dotrim() to take advantage of the 'struct conf_entry' that
is already passed in, instead of having the caller copy values from
that struct into additional parameters.

MFC after: 22 days

Reduce the annoying compiler warnings that pop up when compiling with
gcc 3.3.x and -Wshadow. Just renames 'log' variables to be 'logname'.

MFC after: 22 days

Correct the comment about which timezone-change loses an hour...

MFC after: 23 days

Add a '-D <something>' command line arg, which can be used to set
debugging options. Initial option is '-D TN=<time>', which can be
used to see how newsyslog would work if run at the specified time.
(time format is ISO 8601, since that is already supported).

MFC after: 23 days

Use strtol() instead of strtoul() in parse8601, so we can detect
negative values. Mainly done to sync this routine with OpenBSD.

Obtained from: OpenBSD
MFC after: 23 days

Fix typo in the previous commit. Was checking wrong variable...

MFC after: 23 days

Change parse8601 and parseDWM so they return an alternate error value
for invalid times, and have the caller print the error message.

MFC after: 23 days

Cosmetic change to move parse8601 right next to parseDWM. No code
is changed. (that will come in later updates).

MFC after: 23 days

When checking the 'user:group' field in newsyslog.conf, freebsd's source
was mistakenly calling the standard isnumber() function to find out if
the given 'user' or 'group' were all numeric. This meant that only the
first character of the fields were actually checked, so a username of
(say) '3com' would look like a number, and thus get mapped to uid=3 (bin)
instead of username=3com.

This bug was introduced back in freebsd's v1.1. That initial import
almost matches netbsd's v1.9, except that an internal isnumber()
routine was removed in favor of the standard library version. The thing
is, that internal routine was checking the entire string, and not just
the first digit. In OpenBSD, isnumber() was eventually renamed to
isnumberstr() to make the distinction more obvious, and I'm going to
follow that lead.

I believe this also happens to remove the last references to isnumber()
in the entire freebsd base system.

Obtained from: OpenBSD, by a long circuitous route
MFC after: 5 days

Fix an 'bad file descriptor' error which would come up when using
the 'C' flag on an entry that also specifies 'user:group' info.

Submitted by: Riccardo Torrini <riccardo@torrini.org> in -current
MFC after: 4 days

Fix age_old_log checking so that it will notice log files which were
rotated and then compressed with bzip2 instead of gzip. Otherwise,
any file which had a time-interval specified for 'when' and also
specified the 'J' flag would be rotated every time newsyslog was run.
(this is a quick-fix, trying to beat the code-freeze for 5.1-release)

PR: bin/51519
MFC after: 1 week

Slightly improve the checking of the 'size' field, so people will get
a warning message if they switch the values for 'size' and 'when'.

MFC after: 1 week

Use __FBSDID over rcsid[]. Protect copyright[] where needed.

Add a -C (create) option for newsyslog, and a 'C' flag for entries in the
config file. If the -C option is specified once, then newsyslog will create
any entries which specify the 'C' option. If -C is given twice, then
newsyslog will create all missing log files. Some of this code comes
from NetBSD, although this implementation does not exactly match theirs.

Reviewed by: freebsd-arch
MFC after: 10 days

Fix interactions between entries for a specific file vs entries for
a filename pattern, and also wrt filenames given on the command line.
Now if a file is listed as a specific entry, it will not *also* be
processed by an entry specifying a pattern. And filename-patterns
will now only match existing files (ignoring directories, etc).

MFC after: 3 weeks

The processing of files given on the command-line will work a little better
if we return the entire worklist, instead of just the last entry. A minor
fix to revision 1.57.

MFC after: 15 days

Add a config-file flag of 'U' or 'u' to indicate that the pid-file
will contain the pid for a process group. This means the file must
contain a negative value (as would be needed in the 'kill' commmand).
I still need to write man-page update before MFC-ing.

This started by rewriting the get_pid() routine. Later I looked at
what OpenBSD has, and included a few ideas from their send_signal()
routine. So, parts of this change are from OpenBSD, even though
OpenBSD does not actually have a 'U' flag.

PR: bin/28435
Reviewed by: no objections on freebsd-arch
MFC after: 3 weeks

Prefix all -n (noaction) "unix cmds" with tab, thus making it much easier
to sort out noaction messages from -v (verbose) messages. This also
improves a few noaction messages.

Obtained from: OpenBSD
MFC after: 3 weeks

Move the -s (nosignal) processing, as newsyslog needs to print out a
warning message if -s is specified and it rotates a file that expects
to be compressed. This warning message is not printed if -R is also
specified, because we assume a -sR request is coming from the process
which would have been signaled, and that it has already released the
logfile.

Indirectly noticed by: sheldonh

Add missing 'static' on two routines, so they match their declarations.

PR: 29363 (a small part of)
Submitted by: dinoex
MFC after: 3 weeks

A variety of cosmetic changes, mostly to (slightly) reduce diffs with
OpenBSD. Except for one added '\n', the object code is not changed.

MFC after: 3 weeks

Collapse two more parameters to dotrim() into the 'ent' parameter that is
already there. This should not effect anything.

MFC after: 3 weeks

Change to using uid_t and gid_t instead of int. Also change this won't
bother doing a chmod() if neither of these two fields are set.

Obtained from: OpenBSD
MFC after: 3 weeks

Fix the interactions between specific log files given on the command line,
and config-file entries which specify a filename-pattern (glob). It is
still not perfectly-right, but at least it isn't completely-wrong.

Reviewed by: no objections on freebsd-arch
MFC after: 3 weeks
MFC addendum: (or after the code-freeze of 4.x is lifted)

Add a command-line option of '-R somename', which indicates that newsyslog
should rotate all files given on the command, even if they don't seem to
need to be rotated. This would be used by some other command that decides
the given log file(s) should be rotated, but wants the "how" of that rotation
to be determined by entries to newsyslog. Wes expects to change syslogd to
take advantage of this. Man page will be updated after we're sure this is
all working the way we want it to.

Reviewed by: no objections on freebsd-arch
MFC after: 3 weeks
MFC addendum: (or after the code-freeze of 4.x is lifted)

Add a command-line option of '-s', which indicates that newsyslog should
not send a signal to any processes. Also add a config-file flag of 'N' or
'n', which indicates that the given logfile has no process which needs a
signal when it is rotated. Both of these are based on changes NetBSD
has made, although the implementation is somewhat different.

PR: bin/36553 (2nd half)
Reviewed by: no objections on freebsd-arch
Obtained from: NetBSD (in spirit, at least)
MFC after: 3 weeks

Fix an error which caused 'free(): error: chunk is already free'
if an entry in the config file included flag 'G'.

MFC after: 3 weeks

Check the value given for number-of-logs to keep in config-file entries,
and error-out if the value is negative (avoiding an infinite loop).

PR: bin/30654 (part of it)
MFC after: 3 weeks

1) Fix so -a honors the -n ("do nothing") option. Before, if the directory
given for -a did not exist, then newsyslog would always try to create
it, even if -n was specified.
2) When -a processing *does* create the directory, have it check the result
from mkdir(), and immediately error-out if that failed.

PR: bin/46974
MFC after: 3 weeks

Minor style fix. Add parenthesis around 'return' values.

Reviewed by: md5
MFC after: 3 weeks

Add the idea of a "default rotate action", which will be used if a file is
specified at runtime, but that filename is not listed in the newsyslog.conf
file. This default-action can be changed by having a line in newsyslog.conf
with the filename of "<default>". Before this change, the program would
quietly ignore the given file. An update to the man page will be written
after I finish some other updates to newsyslog.c.

Reviewed by: no objections from freebsd-arch
MFC after: 3 weeks

Make newsyslog(8) print the correct date when the logs are turned over.

PR: 46395
Submitted by: maxim
MFC: eventually

Add a new `W' flag, that when used in conjunction with the `Z' flag or the
`J' flag means that newsyslog should wait for previously started compression
jobs to complete before starting a new one for this entry. When it is used
along with the `G' flag, in the case when several logfiles match the pattern
and should be compressed, the newsyslog will compress logs one by one,
ensuring that only one compression job is running at a time.

This prevents newsyslog(8) from overloading system by starting several
compression jobs on big files simultaneously.

Sponsored by: Porta Software Ltd
MFC after: 2 weeks

- Allow comments to be placed at the end of configuration line.

Sponsored by: Porta Software Ltd
Approved by: re
MFC after: 2 weeks

Add a new newsyslog.conf flag - 'G', which if set tells newsyslog(8) that
the specified filename of the log to be rotated is in fact shell glob
pattern. In this case, all files matching this pattern will be rotated
using the same options. Useful in the case when there is no pre-defined
name for the logfiles (e.g. xtradius, samba etc).

Sponsored by: PortaOne Software Ltd
MFC after: 2 weeks

Fix getopt(3) optstring: remove a flag we does not handle.

PR: bin/44066
Submitted by: Stefan Farfeleder <e0026813@stud3.tuwien.ac.at>
MFC after: 1 week

Remove the last EOF spaces I missed in rev. 1.43.

MFC after: 1 week

style(9): sort includes, remove EOL spaces.

MFC after: 1 week

wait(2) for compress_log() processes to complete before exit.

Spotted by: Grigoriy Strokin <grg@isabase.philol.msu.ru>
Reviewed by: -current
MFC after: 1 month

Close a very small window during which new (empty) instances of rotated log
files are owned by the caller of newsyslog (usually root:wheel) even if
alternative ownerships were specified in newsyslog.conf.

Note that this is part of a wider problem which is fully addressed in
OpenBSD. Anyone with the time and inclination to incorporate the full
fix for the wider problem will receive no complaints from me and should
feel free to walk all over this delta.

PR: bin/36738
MFC after: 1 week

Produce a different error message for mktime(3) failure than for bad
newsyslog.conf syntax.

PR: bin/36641

Fix -F with logfiles rotated on time only.

Approved by: silence on -audit
MFC after: 1 month

Fix the type of the NULL arg to execl().

jhb says a 10 line change containing casts is more invasive than a 28 line
change.

Use the POSIX syntax for chown.

Submmited by: garrett

Simplify (and hopefully clarify) some date-parsing arithmetic.

style(9)

WARNS-2'ify.
(this included removing the funny "if (unsigned_var < 0)" test)

Add the ability to Bzip2 your logs.

PR: 27901
Submitted by: Anders Nordby <anders@fix.no> (stylistic changes by me)

Fix the type of the NULL arg to execl()

Idea from: Theo de Raadt <deraadt@openbsd.org>

- Correct MAXPATHLEN/MAXHOSTNAMELEN usage
- Check return values of malloc() and strdup()
- Replace strcpy()/strcat()/sprintf() usage with strlcpy()/snprintf

Reviewed by: -audit

When warning that a log file can't be compressed, tell the reader which
log file it was.

Allow user to specify which logs to rotate

- implement logfile archivation to a separate, user
configurable directory
- implement alternate and more flexible way to specify
logfile rotation time in addition to the ISO 8601
restricted format
- cleanup the source which was a mix of several styles
of persons who maintained it so far, ran through
knfom script got from bde.

Reviewed by: (in part) sheldonh and garyj

Run newsyslog source through knfom script obtained from Bruce Evans.
Joerg Wunsch suggested to do this to make the functional changes in
the next commit to newsyslog (which were run through the same script)
better visible.

$Id$ -> $FreeBSD$

Go back to allowing ``.'' as a username.group separator for backward
compatibility. : is still the documented non-ambiguous approach. The
algorithm used will correctly parse david.obrien.staff as strrchar() is
used, and in my mind more people would use a ``.'' in the username than
the group name.

Convinced by argument and patch by: sheldonh (with slight changes by me)

Syntax for user/group is changed from "user.group" to "user:group" to be
consistant with chown(8).

A slight bit of code and doco cleanup, but mostly:

Add the much-needed and long-desired ability to force the log rotation
to take place on a specific day or time, and document same. This works
by extending the syntax of the `when' field to allow a restricted
ISO 8601 time specification, and performing the log rotation if newsyslog
is run during the same hour as is specified.

The largest PID is now 99999
Reported by: Ben Stuyts <ben@stuyts.nl>

Add optional config file field: signal number to send (defaulted to HUP)
F.e. apache restarts less agressively on USR1 then on HUP

1) kill -HUP syslog, even if compressing supposedly "binary" files
PR: docs/6385

2) -n (noaction) does not imply -r (run as non-root), since as of
Rev. 1.12 (ache), -r changes actual behaviour.

3) missing \n from if(noaction) messages.

Arg! Someone fixed the synopsis in -stable's newsyslog.8, but not
in -current. However, they fixed both the ordering and the missing -F
flag. Merge from 22 and fix ordering here, too.

Fix the usage here, too (-F option).

Typo fix in comment ("Force the tim now matter what" s/tim/trim/)

Add "-F" (force) flag to newsyslog.
Submitted by: Terry Lambert <terry@freebsd.org>

Sleep 10secs after kill -HUP instead of 3secs, some daemons like Apache
may have log time to re-configure

Cosmetic in err() strings.

Allow a '-' flag as a placeholder so that path_to_pid_file
can be specified without the flags field.
Fix bogus "trail" left in parse routine where the code jumps
past the end of the line and wanders into oblivion.

Do not kill -HUP syslog if running in user mode (-r), it allows
private user logs rotation without error diagnostics

1) Add yet one optional field: pid file to send SIGHUP to if log rotated
(good thing for apache f.e.) This change is backward compatible with old
newsyslog.conf files.

2) Do not compress log if SIGHUP sending failed for some reason
(f.e. pid file deleted). Newcoming messages will be lost
otherwise.

3) Misc cleanup while I am here.

compare return value from getopt against -1 rather than EOF, per the final
posix standard on the topic.

Revert $FreeBSD$ to $Id$

Make the long-awaited change from $Id$ to $FreeBSD$

This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.

Boy, I'm glad we're not using sup anymore. This update would have been
insane otherwise.

Doh! Actually read the patch I applied and see what it was *trying* to do,
then make it do that.
Error-of-my-ways-pointed-out-by: Bob Willcox <bob@luke.pmr.com>

PR#2724 in NetBSD notes that newsyslog save log files even if the number
of copies to save is zero. Incorporate suggested fix with some stylistic
cleanup to make the resulting code more readable.
Submitted-By: Kenneth Stailey <kstailey@dol-esa.gov>

Added function prototypes and removed in-line prototypes of standard
functions such as malloc and time.

Fix uninitialized variable that caused a core dump if /etc/newsyslog.conf
was empty. Closes PR bin/1300.

Found by: Brian Tao <taob@io.org>

Submitted by: charnier@lirmm.fr (Philippe Charnier)
1) correct MAX_PID according to system definition.
2) replace fprintf(stderr,...)/perror/exit by 4.4 equivalence.
3) make strings enough larger so they can't overflow: MAXHOSTNAMELEN for
host and MAXPATHLEN for filenames.
4) call execl with correct parameters (compress -> gzip).

Obtained from: NetBSD (PR#1906)
removed date formatting glitch in newsyslog

Obtained from: NetBSD
imported the newsyslog command from NetBSD - it make the "rotation" of
the logfiles much simpler (it is currently done by "hand" in the
/etc/[daily,weekly,monthly] scripts) - now it will be done by invoking
newsyslog every hour which is very customizable via a
/etc/newsyslog.conf file