diff: honour -B flag with -q

PR: 278988
Reviewed by: bapt
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45220

diff: Integrate libdiff from OpenBSD GoT.

This adds support for two new diff algorithms, Myers diff and Patience
diff.

These algorithms perform a different form of search compared to the
classic Stone algorithm and support escapes when worst case scenarios
are encountered.

Add the -A flag to allow selection of the algorithm, but default to
using the new Myers diff implementation.

The libdiff implementation currently only supports a subset of input and
output options supported by diff. When these options are used, but the
algorithm is not selected, automatically fallback to the classic Stone
algorithm until support for these modes can be added.

Based on work originally done by thj@ with contributions from kevans@.

Sponsored by: Klara, Inc.
Reviewed by: thj
Differential Revision: https://reviews.freebsd.org/D44302

diff: Fix --expand-tabs and --side-by-side.

* Overhaul column width and padding calculation.
* Rewrite print_space() so it is now a) correct and b) understandable.
* Rewrite tab expansion in fetch() for the same reason.

This brings us in line with GNU diff for all cases I could think of.

Sponsored by: Klara, Inc.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D44014

usr.bin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

usr.bin: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

diff: Fully comment out the jackpot variable.

This fixes a set but unused warning.

diff: restyle loop a bit

This is a bit more readable, and this loop is probably unlikely to gain
any `continue` or `break`s.

Suggested by: pstef
Differential Revision: https://reviews.freebsd.org/D37676

diff: fix side-by-side output with tabbed input

The previous logic conflated some things... in this block:
- j: input characters rendered so far
- nc: number of characters in the line
- col: columns rendered so far
- hw: column width ((h)ard (w)idth?)

Comparing j to hw or col to nc are naturally wrong, as col and hw are
limits on their respective counters and nc is already brought down to hw
if the input line should be truncated to start with.

Right now, we end up easily truncating lines with tabs in them as we
count each tab for $tabwidth lines in the input line, but we really
should only be accounting for them in the column count. The problem is
most easily demonstrated by the two input files added for the tests,
the two tabbed lines lose at least a word or two even though there's
plenty of space left in the row for each side.

Reviewed by: bapt, pstef
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D37676

diff: Don't (ab)use sprintf() as a kind of strcat().

Previously print_header() used sprintf() of a buffer to itself as a
kind of string builder but without checking for overflows. This
raised -Wformat-truncation and -Wrestrict warnings in GCC. Instead,
just conditionally print the new timestamp fields after the initial
strftime()-formatted string. While here, use sizeof(buf) with
strftime() rather than a magic number.

Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D36814

diff: Don't treat null characters like carriage returns in readhash().

The implicit fall-through in the !D_FORCEASCII case caused null
characters to be treated as carriage returns honoring the D_STRIPCR,
D_FOLDBLANKS, and D_IGNOREBLANKS flags.

Reported by: GCC -Wimplicit-fallthrough
Reviewed by: bapt
Fixes: 3cbf98e2bee9 diff: read whole files to determine if they are ASCII text
Differential Revision: https://reviews.freebsd.org/D36813

diff: Fix a use after free as well as a memory leak in change().

When -B or -I are used, change() evaluates the lines in a hunk to
determine if it is a hunk that should be ignored. It does this by
reading each candidate line into a mallocated buffer via preadline()
and then calling ignoreline(). Previously the buffer was freed as a
side effect of ignoreline_pattern() called from ignoreline().
However, if only -B was specified, then ignoreline_pattern() was not
called and the lines were leaked. If both options were specified,
then ignoreline_pattern() was called before checking for a blank line
so that the second check was a use after free.

To fix, pull the free() out of ignoreline_pattern() and instead do it
up in change() so that is paired with preadline().

While here, simplify ignoreline() by checking for the -B and -I cases
individually without a separate clause for when both are set. Also,
do the cheaper check (-B) first, and remove a false comment (this
function is only called if at least one of -I or -B are specified).

Reviewed by: emaste
Reported by: GCC 12 -Wuse-after-free
Differential Revision: https://reviews.freebsd.org/D36822

diff: Use start of change when searching for function

Use the start of change when searching for a function rather than the
start of the context. In short functions if this could result in search
for the function name starting from before the function definition.

PR: 262086
Reviewed by: bapt, mckusick, mhorne
Sponsored by: Klara Inc.
Differential Revision: https://reviews.freebsd.org/D34328

diff: Detect Objective-C methods

When searching back for function definitions, consider lines starting
with '+' and '-', this allows us to pick up Objective-C methods as well
as C style function definitions.

Reviewed by: bapt
Sponsored by: Klara Inc.
Differential Revision: https://reviews.freebsd.org/D34202

diff: consider two files with same inodes as identical

Obtained from: OpenBSD
MFC after: 1 week

diff: implement option -F (--show-function-line)

With unified and context diffs, show the last line that matches the
provided pattern before the context.

Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D31714

diff(1): Add --color support

Adds a --color flag to diff(1) that supports the same options as GNU's
diff(1). The colors are customizable with the env var DIFFCOLORS in
a format similar to grep(1)'s GREPCOLORS. An example would be 04;36:41
for additions to be underlined light blue, and deletions have a red
background.

Differential Revision: https://reviews.freebsd.org/D30545

diff: decrease indent level

An upcoming change will add more code in the loop.

diff: avoid applying offsets to null pointer

This was the only instance of undefined behavior I could find so far.

diff: replace isqrt() with sqrt()

Remove cruft and use a system-provided and maintained function instead.

diff: move functions around and reduce their visibility

Most of them become static. There will be more such functions added in
upcoming commits, so they would be inconsistent with existing code.
Improve the existing code instead of reinforcing the unwanted pattern.

diff: improve code style

Reflow comments, strip trailing space, improve wrapping of lines.

diff: read whole files to determine if they are ASCII text

Before this change, only the first BUFSIZE bytes were checked.

Reviewed by: bapt (previous version)
Differential Revision: https://reviews.freebsd.org/D31639

diff: don't output carriage returns that were stripped on input

--strip-trailing-cr worked as intended for comparison between files,
but the characters were still present in final output.

usr.bin/diff: fix UBSan error in readhash

UBSan complains about the `sum = sum * 127 + chrtran(t);` line below since
that can overflow an `int`. Use `unsigned int` instead to ensure that
overflow is well-defined.

Reviewed By: imp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D31075

Revert "diff: eliminate a useless lseek"

This changes breaks when one of the files is stdin

This reverts commit fa977a3b2bb2d0e6c2957b14474c31b58dd3a8e1.

Reported by: olivier

diff: eleminitate useless macros

The diff_output was not bringing any values but was obfuscating
the code.

diff: simplify the hash functions

Instead of 3 different complex case they have all been folded into a
simple on based on switch

diff: fix typo in a comment

diff: eliminate space at end of line

No functionnal changes

diff: eliminate a useless lseek

fdopen with the "r" already position the stream at the beginning
of the file.

diff: fix incorrectly displaying files as duplicates

When diff hits certain access errors, function diffreg() shows the error
message, and then returns to the calling function, which calls
print_status() with the return value.

However, in these cases, the return value isn't changed from the initial
default value of D_SAME.

Normally, print_status() with a value of D_SAME does nothing, so this
works out ok, however, if the "-s" flag is set, a message is displayed
showing identicality:

case D_SAME:
if (sflag)
printf("Files %s%s and %s%s are identical\n", path1, entry, path2, entry);
break;

This then produces such results as:

% diff -s /COPYRIGHT /var/run/rpcbind.sock
diff: /var/run/rpcbind.sock: Operation not supported
Files /COPYRIGHT and /var/run/rpcbind.sock are identical

% diff -s /COPYRIGHT /etc/master.passwd
diff: /etc/master.passwd: Permission denied
Files /COPYRIGHT and /etc/master.passwd are identical

Create a D_ERROR status which is returned in such cases, and
print_status() then deals with that status seperately from D_SAME

PR: 252614
MFC after: 1 week

diff: honour flags with -q

Previously -q (just print a line when files differ) ignored flags like
-w (ignore whitespace). Avoid the D_BRIEF short-circuit when flags are
in effect.

PR: 252515
Reported by: Scott Aitken
Reviewed by: kevans
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D28064

diff: always properly kill pr(1)

When diff is invoked with -l it will spawn the pr(1) program.
In some circumpstances the pr(1) was not properly killed when diff program
exits.

Submitted by: Bret Ketchum
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D26232

diff: implement -y (--side-by-side) along with -W and --suppress-common-lines

PR: 219933
Submitted by: fehmi noyan isi <fnoyanisi@yahoo.com>
MFC after: 3 weeks

Do not skip line-by-line comparison if -q and -I are specified.

This fixes a regression from r356695.

Submitted by: kevans
Reported by: Jenkins via lwhsu
MFC after: 6 days

When system calls indicate an error they return -1, not some arbitrary
value < 0. errno is only updated in this case.

Obtained from: OpenBSD
MFC after: 3 days

mkstemp returns -1

Obtained from: OpenBSD
MFC after: 3 days

Optimize diff -q.

Once we know whether the files differ, we don't need to do any further
work.

PR: 242828
Submitted by: fehmi noyan isi <fnoyanisi@yahoo.com> (original version)
Reviewed by: bapt, kevans
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D23152

capsicum: use a new capsicum helpers in tools

Use caph_{rights,ioctls,fcntls}_limit to simplify the code.

diff(1): Refactor -B a little bit

Instead of doing a second pass to skip empty lines if we've specified -I, go
ahead and check both at once. Ignore critera has been split out into its own
function to try and keep the logic cleaner.

diff(1): Implement -B/--ignore-blank-lines

As noted by cem in r338035, coccinelle invokes diff(1) with the -B flag.
This was not previously implemented here, so one was forced to create a link
for GNU diff to /usr/local/bin/diff

Implement the -B flag and add some primitive tests for it. It is implemented
in the same fashion that -I is implemented; each chunk's lines are scanned,
and if a non-blank line is encountered then the chunk will be output.
Otherwise, it's skipped.

MFC after: 2 weeks

Improve --strip-trailing-cr handling:

- Advance ctold for f1 and ctnew for f2
- ungetc() if the character is unexpected
- Don't break early when we hit the combination on one side

PR: 230049
Reported by: maskray <emacsray gmail com>
Reviewed by: bapt, maskray
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D16451

Convert `cap_enter() < 0 && errno != ENOSYS` to `caph_enter() < 0`.

No functional change intended.

Isolate the pr(1) related code in its own source files

This keeps diffreg.c closer to what it is supposed to do:
diffing regular files.

It also allows my code to get a proper license

Replace homemade equivalent of tolower(3) by towlower(3)

This will help in the futur making diff -i works with multibyte

spdx: initial adoption of licensing ID tags.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

Initially, only tag files that use BSD 4-Clause "Original" license.

RelNotes: yes
Differential Revision: https://reviews.freebsd.org/D13133

Don't emit "diff: diff <options> arguments" when diffing files if
-q is specified.

This improves compatibility with GNU diff.

Found by accident with `diff -Nrq /usr/tests /usr/tests.new | grep Kyuafile`.

MFC after: 2 months
Relnotes: yes

Fix the following warning from gcc 4.2 in usr.bin/diff:

usr.bin/diff/diffreg.c: In function 'change':
usr.bin/diff/diffreg.c:1085: warning: 'i' may be used uninitialized in this function

This version of gcc is not smart enough to see that 'i' cannot actually
be used unitialized. However, the variable is confusingly re-used, so
it is better to give it another name, and clearly initialize it before
attempting to use it.

Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D10484

Implement a basic --changed-group-format

etcupdate(8) requires that option, while GNU diff supports many more variation
of that options, their behaviour beside the simple verion implemented here are
quite inconsistent as such I do not plan to implement those.

The only special keyword supported by this implementation are: %< and %>
%= is not implemented as the documentation of GNU diff says: common lines, but
it actually when tested print the changes from the first file

Clean up headers declaration

diff: Show nanoseconds in -u/-c header line.

Show nanoseconds in the -u/-c header line.

The present portability conditionals cannot handle the POSIX standard
st_mtim, so remove them and unconditionally use st_mtim.

PR: 218018
Reported by: jbeich
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D10145

diff: Fix mtime of file1 in -u/-c header line.

PR: 218018
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D10140

Use MIN macros from sys/param.h

Use MAX and MIN macros from sys/param.h

Cache tzdata when running under capsicum

PR: 217957
Reported by: tobik@

Do not die if cap_rights_limit reports ENOSYS

Reported by: mmel

Readd codes that creates a tmp file for diffing stdout or devices

Fix building with recent gcc

Reported by: lwhsu, ngie

Import diff from OpenBSD and remove GNU diff

Some of the modifications from the previous summer of code has been integrated
Modification for compatibility with GNU diff output has been added

Main difference with OpenBSD:
Implement multiple GNU diff options:
* --ignore-file-name-case
* --no-ignore-file-name-case
* --normal
* --tabsize
* --strip-trailing-cr
Make diff -p compatible with GNU diff
Implement diff -l
Make diff -r compatible with GNU diff

Capsicumize diffing 2 regular files
Add a simple test suite

Approved by: AsiaBSDcon devsummit
Obtained from: OpenBSD, GSoC
Relnotes: yes