cp: avoid a resource leak

In copy_file(), make sure the from_fd file descriptor is closed even
when the operation failed early.

Reported by: Coverity Scan
CID: 1545036
Sponsored by: The FreeBSD Foundation

Reviewed by: imp, emaste
Pull Request: https://github.com/freebsd/freebsd-src/pull/1238

cp: Additional sanity check.

Once we've successfully opened the file we've been asked to copy, check
that it's of the same type as FTS told us it was.

MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: allanjude, markj
Differential Revision: https://reviews.freebsd.org/D44806

cp: Remove useless ssize_t cast

Both wcount and wresid are ssize_t so this cast is not needed. Just
remove it so the code is easier to read.

Signed-off-by: Collin Funk <collin.funk1@gmail.com>
Reviewed by: emaste, zlei
Pull Request: https://github.com/freebsd/freebsd-src/pull/1116

cp: Further simplify the core logic.

If the destination file exists but we decide unlink it, set the dne
flag. This means we don't need to re-check the conditions that would
have caused us to delete the file when we later need to decide whether
to create or replace it.

MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D43064

cp: Refactor the core logic.

Rewrite `copy_file()` so the lflag and sflag are handled as early as
possible instead of constantly checking that they're not set and then
handling them at the end. This also opens the door to changing the
failure logic at some future point (for instance, we might decide to
fall back to copying if `errno` indicates that the file system does not
support links).

MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: kevans, allanjude
Differential Revision: https://reviews.freebsd.org/D43055

cp: Don't warn for chflags() failing with EOPNOTSUPP if flags == 0

From NetBSD's utils.c 1.5 importing importing BSDI change, with light
formatting changes:
Author: cgd <cgd@NetBSD.org>
Date: Wed Feb 26 14:40:51 1997 +0000

Patch from BSDI (via Keith Bostic):
>NFS doesn't support chflags; ignore errors unless there's reason
>to believe we're losing bits. (Note, this still won't be right
>if the server supports flags and we were trying to *remove* flags
>on a file that we copied, i.e., that we didn't create.)

CVS Info: utils.c 1.6

Obtained from: NetBSD
Sponsored by: Netflix

Differential Revision: https://reviews.freebsd.org/D42674

cp: Add -N flag, inspired by NetBSD's similar flag

Add -N to supress copying of file flags when -p is specified (explicitly
or implicitly). Often times we don't care about the flags or wish to be
able to copy to NFS, and this comes in handy for that. FreeBSD's and
NetBSD's cp are somewhat different, so I had to reimplement all but one
of the patch hunks...

Obtained from: NetBSD (cp.1 1.25, cp.c 1.37, utils.c 1.28 by elad)
Sponsored by: Netflix

Differential Revision: https://reviews.freebsd.org/D42673

bin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

bin: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

cp: Revert "If copy_file_range(2) fails with EXDEV, use fall-back."

This reverts commit 64333654908153cd8e2dcb408c57cc3e3c4f29af.

The error is not valid per api contract, it showed up as a regression
after 15f0b8c309dea1dcb14d3e374686576ff68ac43f

15f0b8c309dea1dc ("zfs: merge openzfs/zfs@9cd71c860 (master)") and was
subsequently in d012836fb6165494 ("zfs: fix up EXDEV handling for
clone_range").

Sponsored by: Rubicon Communications, LLC ("Netgate")

If copy_file_range(2) fails with EXDEV, use fall-back.

cp: Minor code cleanup.

* Fix includes in utils.c, cf. style(9).
* Fix type mismatch: readlink(2) returns ssize_t, not int.
* It is not necessary to set errno to 0 as fts_read(3) already does it.

MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: allanjude
Differential Revision: https://reviews.freebsd.org/D38369

cp: Simplify the common case.

* The allocated buffer is only used in the fallback case, so move it
there. The argument for passing it in from the caller was that if
malloc(3) were to fail, we'd want it to fail before we started
copying anything, but firstly, it was already not in the right place
to ensure that, and secondly, malloc(3) never fails (except in very
contrived circumstances, such as an unreasonable RLIMIT_AS or
RLIMIT_DATA).

* Remove the mmap(2) option. It is almost never beneficial,
especially when the alternative is copy_file_range(2), and it adds
needless complexity and indentation.

MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: rmacklem, mav
Differential Revision: https://reviews.freebsd.org/D38291

cp: Fix build without VM_AND_BUFFER_CACHE_SYNCHRONIZED.

It allows to not use mmap() for small files, which is not helpful
in case of ZFS. Should be no functional change.

MFC after: 1 week

cp: fix indentation

No functional changes

cp(1): fix performance issue for large non-sparse file copies

PR252358 reported a serious performance problem when
copying a large non-sparse file on a UFS file system.
This problem seems to have been caused by a large
number of SEEK_HOLE operations, with one done
for each copy_file_range(2) call.

This patch modifies cp(1) to use a large (SSIZE_MAX)
len argument, reducing the number of system calls
and resolving the performance issue.

While here, convert the type of the "rcount" from "int"
to "ssize_t" so that it is consistent with that returned
by both read(2) and copy_file_range(2).

PR: 252358
Reviewed by: asomers
Differential Revision: https://reviews.freebsd.org/D27937

cp: fall back to read/write if copy_file_range fails

Even though copy_file_range has a file-system agnostic version, it still
fails on devfs (perhaps because the file descriptor is non-seekable?) In
that case, fallback to old-fashioned read/write. Fixes
"cp /dev/null /tmp/null"

PR: 249248
Reported by: Michael Butler
Reviewed by: mjg
MFC-With: 365549
Differential Revision: https://reviews.freebsd.org/D26395

cp: use copy_file_range(2)

This has three advantages over write(2)/read(2):

* Fewer context switches and data copies
* Mostly preserves a file's sparseness
* On some file systems (currently NFS 4.2) the file system will perform the
copy in an especially efficient way.

Reviewed by: rmacklem
MFC after: 2 weeks
Sponsored by: Axcient
Differential Revision: https://reviews.freebsd.org/D26377

General further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

Renumber copyright clause 4

Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96

Cleanup some style(9) issues.

- Whitespace.
- Comments.
- Wrap long lines.

MFC after: 2 weeks
X-MFC-with: r284105,r284106
Sponsored by: EMC / Isilon Storage Division

Implement '-s' to copy as symlink, similar to the current -l link(2) handling.

This is also implemented in at least GNU coreutils cp.

While here also improve the '-l' handling to not open(2) the source file as
it does not actually need the descriptor.

Sponsored by: EMC / Isilon Storage Division

Cleanup some indentation issues.

cp,mv: Use futimens() instead of utimensat() if possible.

cp,mv,touch: Set timestamps with nanosecond precision.

This uses utimensat().

Return with an error from copy_link(), copy_fifo() and copy_special() if
the -n option is specified and the destination file exists.

PR: bin/174489
Approved by: rstone (co-mentor)
MFC after: 2 weeks

Return "failure" as we do for 'cp -i' and a "n" answer.

Otherwise with '-v' we print out the file name as if it was copied:
/tmp/2gb-card/M0132.CTG not overwritten
/mnt/DCIM/CANONMSC/M0132.CTG -> /tmp/2gb-card/M0132.CTG

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Change all our own code to use st_*tim instead of st_*timespec.

Also remove some local patches to diff(1) which are now unneeded.

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

Merge r202461 from head:

Implement an "-x" option to cp(1), for compatibility with Linux and
feature parity with du(1) and similar: When set, cp(1) will not traverse
mount points.

PR: bin/88056
Initial patch by: Graham J Lee leeg teaching.physics.ox.ac.uk

MFC r196754:

Add NFSv4 ACL support to cp(1) and fix a few memory leaks.

Note that this changes error reporting behaviour somewhat - before,
no error was reported if ACL couldn't be copied because the target
filesystem doesn't support ACLs. Now, it will be reported - of course,
only if there actually is an ACL to copy.

Reviewed by: rwatson

Implement an "-x" option to cp(1), for compatibility with Linux and
feature parity with du(1) and similar: When set, cp(1) will not traverse
mount points.

Initial patch by: Graham J Lee leeg teaching.physics.ox.ac.uk

PR: bin/88056
Initial patch by: Graham J Lee leeg teaching.physics.ox.ac.uk
Approved by: ed (mentor)
MFC after: 1 month

Add NFSv4 ACL support to cp(1) and fix a few memory leaks.

Note that this changes error reporting behaviour somewhat - before,
no error was reported if ACL couldn't be copied because the target
filesystem doesn't support ACLs. Now, it will be reported - of course,
only if there actually is an ACL to copy.

Reviewed by: rwatson

Preserve file flags on symlinks in cp -Rp.
This reported ENOSYS before.

PR: bin/111226 (part of)
Submitted by: Martin Kammerhofer
Approved by: ed (mentor)
MFC after: 3 weeks

Teach cat(1) and cp(1) to use a larger buffer if enough memory is present
in the system. A simple heuristics is used to detect what is "enough"
memory: if number of physmem pages is greater than 32k (equalling 128 MB
on machines with 4 kB pages).

Typical immediate result of these changes is reduction in context switches
and the goal is to increase efficiency by using large buffers:
before: /usr/bin/time -hlp cat file1 > file2
...
163 voluntary context switches
11194 involuntary context switches
after: /usr/bin/time -hlp ./cat file1 > file2
...
417 voluntary context switches
272 involuntary context switches

Reviewed by: hackers@ (no objections to earlier version of cat patch)
Approved by: gnn (mentor)
MFC after: 4 months

Add fallback when mmap fails on regular files. Some filesystems,
like smbnetfs, do not support mmap.

Reported by: Harti Brandt
MFC after: 1 month

- Import the HEAD csup code which is the basis for the cvsmode work.

Remove redundant close(2).

Submitted by: kensmith
Approved by: rwatson (mentor)

Fix ACL preservation, apparently broken in 1.47.

Approved by: rwatson
MFC after: 3 weeks

Add an '-a' option which is identical to specifying '-RpP' for
compatibility with other implementations.

MFC after: 1 month

o Style(9).

Submitted by: bde

o Avoid division by zero.
o Place error checking code near to the syscall.

Submitted by: bde

o Hold copied bytes counter in off_t. This prevents statistics
displayed by SIGINFO handler from overflow.

PR: bin/104039
Submitted by: Geoffrey Giesemann
MFC after: 2 weeks

- Fix options order.
- Touch manpage's document date.

Remove a stray -a option that probably sneaked in from julian's
attempt to enter append mode twice in vi(1). :-)

Add an option to allow copying of a hierarchy while linking he regular files.
Bikeshedded to death on: hackers
Submitted by:andersonatcenttech.com
MFC in: 1 month

Attempt to complete the userspace integration of POSIX.1e extended ACLs.
This includes adding support for ACLs into cp(1) and mv(1) userspace
utilities.

For mv(1), if _PC_ACL_EXTENDED is in effect for the source AND destination
operands, the destination file's ACLs shall reflect the source.

For cp(1), if _PC_ACL_EXTENDED is in effect for both source and destination
operands, and -p has been specified, the ACLs from the source shall be
preserved on the destination.

MFC after: 1 month

Sync program's usage() with manpage's SYNOPSIS.

Plug file descriptor leak in implementation of -n option.

Remove clause 3 from the UCB licenses.

OK'ed by: imp, core

Don't mmap(2) and munmap(2) zero-length files.

Submitted by: Wiktor Niesiobedzki <bsd@w.evip.pl>

support saving both user/group and permissions on symlinks (from PR)

also fix a slight bogon that assumed an fd of 0 was not valid. Changed
it to be -1.

PR: bin/25017
Submitted by: Martin Kammerhofer

Add a SIGINFO handler.

Constify and staticify for lint.

Fix some easy WARNS.

Add the -n option, which automatically answers "no" to the overwrite question.

PR: 7828
Suggested by: Daniel O'Connor <doconnor@gsoft.com.au>
Approved by: sheldonh (mentor)
MFC after: 2 weeks

Fix some low-hanging lint-fruit: endianness and staticness warnings.

Consistently use __FBSDID

Fix warnings inspired by lint, a commercial lint and WARNS=4.

Modernization effort for bin/c*:

o __P has been reoved
o Old-style K&R declarations have been converted to new C89 style
o register has been removed
o prototype for main() has been removed (gcc3 makes it an error)
o int main(int argc, char *argv[]) is the preferred main definition.
o Attempt to not break style(9) conformance for declarations more than
they already are.

Approved by: arch@, new style(9)

Typo in warning message (chown should be chmod).

Add more headers that are required with -fno-builtin (stdlib and strings)

Removed the broken code which claimed to lose the set[ug]id bits in
the !(pflag && setfile()) case for regular files unless the copy is
owned by the same user and group. These bits have already been lost
(or never gained) in the correct way. The code didn't actually lose
the bits; it depended on them being lost already (apparently in all
cases) and attempted to gain them as necessary, but it often gained
them (and sometimes collateral bits) when wrong:
- pflag && setfile() == 0 case (i.e., for a successful cp -p):
setfile() copies all the attributes as correctly as possible (as
specified by POSIX), and we sometimes messed up the up the mode by
setting it again. Also, if the file is immutable, then setting the
mode again gave spurious errors (PR 20646).
- !pflag case. If the target is created, POSIX requires it to not
have the set[ug]id bits, but we sometimes copied them from the source.
If the target already exists, POSIX requires its mode to be unchanged,
but we sometimes copied the whole mode from the source.

PR: 20646
MFC after: 4 weeks

Migrate from using MAXPATHLEN to MAX_PATH. Use strlcpy to copy the
strings.

Don't depend on <sys/stat.h> bogusly including <sys/time.h> (and thereby
<time.h>).

fix cp -vi bug

Submitted by: Dan Papasian (bugg_ on irc)

brucify and move printf() to catch cases of special files

$Id$ -> $FreeBSD$

Add a verbose mode to show what files are being copied.
Idea taken from obrien.

Reviewed by: obrien

Various spelling/formatting changes.

Submitted by: Philippe Charnier <charnier@xp11.frmug.org>

First set of fixes to keep egcs happy. These include {} around single
statement if blocks[*] when the else could be ambiguous, not defaulting
to int type and removal of some unused variables.

[*] This is explicitly allowed by style(9) when the single statement
spans more than one line.

Reviewed by: obrien, chuckr

Don't use mmap() for non-regular files, since st_size is only meaningful
for regular files. This fixes recent breakage of cp'ing from /dev/zero.
/dev/zero doesn't support mmap(), but the device driver mmap routines are
not called for mapping 0 bytes, so the error was not detected. mmap()
can't even be used for cp'ing special files that support mmap(), since
there is general way to determine the file size.

Don't attempt to change owner/mode/flags that don't need to changed.
This should calm down attempts to `cp -p' to a nfs mount or some other
filesystem that doesn't accept flags or all combinations of flags.
It will warn if it fails to change flags though.

Make sure we pass the length - 1 to readlink, since it adds its own
NUL at the end of the path.
Inspired by: OpenBSD's changes in this area by theo de raadt

Restore Lite-2 sccsid.

Correct use of .Nm and other cosmetics. Add rcsid. Remove unused #inc.

Fixed syntax error in normally-unconfigured code in previous commit.

Fixed handling of short writes. Previously, we stopped copying and
printed a bogus warning with a stale errno if write() returns a short
count. Now we continue copying. We still print a bogus warning if
write() returns an "impossible" short count of 0.

cp(1) is too silent if used with the option -i. It should
print which input cp(1) expect (y/n) and print a warning if the
file was not overwritten.

Revert $FreeBSD$ to $Id$

Sweep through the tree fixing mmap() usage:

- Use MAP_FAILED instead of the constant -1 to indicate
failure (required by POSIX).
- Removed flag arguments of '0' (required by POSIX).
- Fixed code which expected an error return of 0.
- Fixed code which thought any address with the high bit set
was an error.
- Check for failure where no checks were present.

Discussed with: bde

Make the long-awaited change from $Id$ to $FreeBSD$

This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.

Boy, I'm glad we're not using sup anymore. This update would have been
insane otherwise.

Merge Lite2 mods and -Wall cleaning.

Option -f implemented (remove + create)
option -f and -i are exclusive
respond `Y' is equal to `y'
fix usage string
remove isatty(3) check

Reviewed by: pst

Don't initialize udata, remove bogus case, fix usage string

delete my last commit
Submitted by: pst, ache

Option -f implemented (remove + create)
option -f and -i are exclusive
all flag variables initialized with zero
respond `Y' is equal to `y'
update usage string

Preserve sticky bit for `cp -pR'. It was already preserved for `cp -R'.
This also fixes loss of the sticky bit for `mv' across file systems.

Don't unlink the target file if the copy failed. This behaviour isn't
documented and is incompatible with gnu cp. It has very few good effects
(it recovers some disk space) and many bad ones:
- special files are unlinked after certain errors.
- the data may not be recoverable if the source is a special file or fifo.
- unlinking destroys the target attributes as well as the target data.
- unlinking doesn't actually remove the target data if the target is multiply
linked.

Remove trailing whitespace.

Reviewed by: phk

Added $Id$

BSD 4.4 Lite bin Sources