Copy stable/10@r272459 to releng/10.1 as part of
the 10.1-RELEASE process.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r268193, r268203:

Fix sed(1) usage: the -i extension is not an optional.

MFC r267692, r267703:
Add -u (unbuffered output) after GNU sed.

Obtained from: NetBSD

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

sed: use getline() instead of fgetln().

In BSD, fgetln() available in libc but in Illumos the Solaris port had to
include it internally. It also seems to have caused problems [1].

Aid portability by using getline() instead.

Reference:
https://www.illumos.org/issues/3820 [1]

Submitted by: Johann 'Myrkraverk' Oskarsson <johann@myrkraverk.com>
Reviewed by: dds
MFC after: 2 weeks

sed: Try hard links to make -i target available continually.

When creating a backup file, sed renamed the original before renaming the
changed copy into place, leading to a short time when no file with the
original name was present (usually only visible on SMP systems). Try
creating the backup file using a hard link instead, avoiding this problem.
If creating the hard link fails for any reason, fall back to the old rename
method.

When not creating a backup file, sed already renamed the changed copy onto
the original. This remains unchanged.

I am not adding the suppression of redundant fchown/fchmod to this commit,
because FreeBSD appears to check this in the kernel (for msdosfs at least).

PR: bin/153261
Submitted by: Pedro F. Giffuni
Reviewed by: dds (older version)
Obtained from: Illumos
MFC after: 2 weeks

Make -r mean exactly the same thing as -E for increased compatibility
with GNU sed.

MFC after: 7 days

Revert most part of 200420 as requested, as more review and polish is
needed.

Remove unneeded header includes from usr.bin/ except contributed code.

Tested with: make universe

WARNS fixes:
1) Add missing parens around assignment that is compared to zero.
2) Make some variables that only take non-negative values unsigned.
3) Some casts/type changes to fix other constness warnings.
4) Make one variable a const char *.
5) Make sure termwidth is positive, it doesn't make sense for it to be negative.

Approved by: dds

Fix visibility of "inplace".
Found by: CScout

Don't forget to clear out the hold space for each subsequent file
when in -i mode so that each file gets a clean context of its own.

Add a regression test for the bug.

Tested with: regression tests

Change the semantics of -i (in-place editing) so that it treats
each file independently from other files. The new semantics are
desired in the most of practical cases, e.g.: delete lines 5-9
from each file.

Keep the previous semantics of -i under a new option, -I, which
uses a single continuous address space covering all files to edit
in-place -- they are too cool to just drop them.

Add regression tests for -i and -I.

Approved by: dds
Compared with: GNU sed
Discussed on: -hackers
MFC after: 2 weeks

The function inplace_edit() doesn't exist anymore, remove the prototype.

Submitted by: Leonardo Chiquitto Filho

Add option -l, to make output line buffered.

PR: bin/78692
Submitted by: Eugene Grosbein
MFC after: 2 weeks

Include missing headers.

Per letter dated July 22, 1999 remove 3rd clause of Berkeley derived
software (original contributor).

Reviewed by: imp

Make the 'y' (translate) command aware of multibyte characters.

Plug a file descriptor leak.

When sed is asked to inline-edit files, it forgets to close the temporary
file and runs out of descriptors for long command lines (assuming you reset
kern.maxfilesperproc to something sane that's less than the number of files
passed to sed).

Fix the case where the file is not in the current directory.

Discovered by: Vladimir Kravchenko <jimson@mostcom.ru>
Pointy hat to: des

Reimplement in-place editing in a slightly less disgusting manner. Also,
make an effort to preserve the ownership and mode of the file we are
editing.

Sponsored by: Registrar AS

ANSIfy

Correct style bugs in the in-place editing code.

Fix some bugs in in-place editing:
(1) errors from freopen were not reported correctly
(2) large files were not handled correctly
(3) read-only files broke things

MFC after: 1 week

Avoid checking whether each line is the last line of the file when we don't
need to know. Instead, check when we are trying to match a "$" address.
This does not change the way sed processes regular files, but makes it behave
more sensibly when used interactively.

PR: 40101
MFC after: 2 weeks

Don't store newlines at the end of each line in the hold/pattern spaces,
instead add the newline when the pattern space is printed. Make the `G' and
`H' commands add a newline to the space before the data, remove bogus
addition of newline from `x' command.

PR: 29790, 38195

Correct the handling of (for example) the N command by only zeroing the
input space in mf_fgets() if we reach the end of all input files.

Exit with non-zero status if an input file can not be opened. This was
broken by rev. 1.22.

Fix one serious bug and one potential problem with in-place editing code:

- original version of code worked incorrectly when more than one
input files were specified - it was moving the last line from the 1st file
to be the first line of the 2nd, last line of the 2nd to be the first
line of the 3rd and so on;

- use mmap()->write() to create temporary file instead of
malloc()->read()->write(), which was not only slower, but also did not
bother to free allocated memory once backup file was created, potentially
leading to memory exhausting when regex is applied to a big file or a large
number of small ones.

If a file argument cannot be processed, process the remaining ones
and exit non-zero (SUSv3)

o Clean up tmp file usage a little by using mkstemp(3) instead of
mktemp(3). It would be amazingly unlikely, but the former method
could result in a symlink attack. A better solution would use
${TMPDIR}, though.
o Make sed not overwrite old backup files with no warning.

Oops, update usage() to have -i.

Add handling for any nil-length string passed to -i for the backup extension.

Add a note that this is kinda-sorta dangerous to the manual page.

Un-shadow the `fname' variable.

Add a -i option to sed(1) to do inplace editing, to give us an alternative to
Perl for such things. The key difference to Perl is that a backup extension
*MUST* be specified, because on one hand it isn't recommended to have options
which optionally take a parameter, and on the other hand, it'd be slightly
unpleasent to implement proper handling for that.

The difference between this and the version posted to developers@ is that it
does handle multiple files in argv after the getopt(3) handling "correctly",
in that the inplace editing-specific code has been moved out to a function,
and that function is used beyond the first file in our linked list.

This option has been documented as FreeBSD-specific in the manpage.

Reviewed by: developers@ (got feedback from: des, fanf, sobomax, roberto,
obrien)
MFC after: 1 week

remove __P

WARNS=2 partial fix; use NO_WERROR to protect against some hard-to-fix warnings.

Use __FBSDID(), kill register keyword.

Expand xmalloc in-place, along with xrealloc; which wasn't even ANSI in its
implementation.

Small style fix '=' -> ' = '

Fix compatibility issue in sed. Do so by explicitely adding a
newline to the end of any -e argument.

PR: bin/18474

Add a new flag: -E enables "extended" regular expressions.

$Id$ -> $FreeBSD$

Fix a new bug introduced by the previous bug fix

Remove local redefinition for err(). Add usage().

compare return value from getopt against -1 rather than EOF, per the final
posix standard on the topic.

fix C programmer's bug number 12.2

submitted by: bruce

Fix the C programmer's bug #1: EOF is of type `int', not `char'.

Strong 2.2 candidate.

Submitted by: wosch

Localize it
8bit cleanup

Fix spelling error (scipt -> script).
Submitted by: <jhawk@MIT.EDU>
Obtained from: NetBSD-bugs mailling list

This commit was generated by cvs2svn to compensate for changes in r1590,
which included commits to RCS files with non-trunk default branches.

BSD 4.4 Lite Usr.bin Sources