Purge more stray embedded $FreeBSD$ strings

These do not use __FBSDID but instead use bare char arrays.

Reviewed by: imp, emaste
Differential Revision: https://reviews.freebsd.org/D41957

Add quitclean command to fsdb(8) to request a filesystem not be marked dirty.

A new command, quitclean, is added to fsdb(8) to request that the
filesystem not be marked as needing a full fsck(8). This is useful
when creating deliberately bad filesystem images to be used to check
that fsck is properly able to clean them up.

MFC-after: 1 week
Sponsored-by: The FreeBSD Foundation

Have fsdb(8) only mark a filesystem dirty when it is modified.

Until this update, the fsdb(8) command always marked a filesystem
as needing a full fsck unless it was run with the -n flag which
allowed no changes to be made.

This change tracks modifications to the filesystem. Two types of
changes are tracked. The first type of changes are those that are
not critical to the integrity of the filesystem such as changes to
owner, group, time stamps, access mode, and generation number. The
second type of changes are those that do affect the integrity of
the filesystem including zeroing inodes, changing block pointers,
directory entries, link counts, file lengths, file types, and file
flags.

When quitting having made no changes or only changes to data that
is not critical to filesystem integrity, the clean state of the
filesystem is left unchanged. But if filesystem critical data are
changed then fsdb will set the unclean flag which will require a
full fsck to be run before the filesystem can be mounted.

MFC-after: 1 week
Sponsored-by: The FreeBSD Foundation

Add `chdb' command to fsdb(8) to set direct block numbers.

Add the ability to set direct blocks numbers in inodes so that manual
corrections can be made. No checking of the values is attempted so
accidental or deliberate bad values can be set.

Submitted by: Chuck Silvers
MFC after: 1 week

Improvement in UFS/FFS directory placement when doing mkdir(2).

The algorithm for laying out new directories was devised in the 1980s
and markedly improved the performance of the filesystem. In those days
large disks had at most 100 cylinder groups and often as few as 10-20.
Modern multi-terrabyte disks have thousands of cylinder groups. The
original algorithm does not handle these large sizes well. This change
attempts to expand the scope of the original algorithm to work well
with these much larger disks while still retaining the properties
of the original algorithm for small disks.

The filesystem implementation is divided into policy routines and
implementation routines. The policy routines can be changed in any
way desired without risk of corrupting the filesystem. The policy
requests are handled by the implementation layer. If the policy
asks for an available resource, it is granted. But if it asks for
an already in-use resource, then the implementation will provide
an available one nearby the request. Thus it is impossible for a
policy to double allocate. This change is limited to the policy
implementation.

This change updates the ffs_dirpref() routine which is responsible
for selecting the cylinder group into which a new directory should
be placed. If we are near the root of the filesystem we aim to
spread them out as much as possible. As we descend deeper from the
root we cluster them closer together around their parent as we
expect them to be more closely interactive. Higher-level directories
like usr/src/sys and usr/src/bin should be separated while the
directories in these areas are more likely to be accessed together
so should be closer. And directories within commands or kernel
subsystems should be closer still.

We pick a range of cylinder groups around the cylinder group of the
directory in which we are being created. The size of the range for
our search is based on our depth from the root of our filesystem.
We then probe that range based on how many directories are already
present. The first new directory is at 1/2 (middle) of the range;
the second is in the first 1/4 of the range, then at 3/4, 1/8, 3/8,
5/8, 7/8, 1/16, 3/16, 5/16, etc.

It is desirable to store the depth of a directory in its on-disk
inode so that it is available when we need it. We add a new field
di_dirdepth to track the depth of each directory. Because there are
few spare fields left in the inode, we choose to share an existing
field in the inode rather than having one of our own. Specifically
we create a union with the di_freelink field. The di_freelink field
is used to track inodes that have been unlinked but remain referenced.
It is not needed until a rmdir(2) operation has been done on a
directory. At that point, the directory has no contents and even
if it is kept active as a current directory is no longer able to
have any new directories or files created in it. Thus the use of
di_dirdepth and di_freelink will never coincide.

Reported by: Timo Voelker
Reviewed by: kib
Tested by: Peter Holm
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39246

Move the ability to search for alternate UFS superblocks from fsck_ffs(8)
into ffs_sbsearch() to allow use by other parts of the system.

Historically only fsck_ffs(8), the UFS filesystem checker, had code
to track down and use alternate UFS superblocks. Since fsdb(8) used
much of the fsck_ffs(8) implementation it had some ability to track
down alternate superblocks.

This change extracts the code to track down alternate superblocks
from fsck_ffs(8) and puts it into a new function ffs_sbsearch() in
sys/ufs/ffs/ffs_subr.c. Like ffs_sbget() and ffs_sbput() also found
in ffs_subr.c, these functions can be used directly by the kernel
subsystems. Additionally they are exported to the UFS library,
libufs(8) so that they can be used by user-level programs. The new
functions added to libufs(8) are sbfind(3) that is an alternative
to sbread(3) and sbsearch(3) that is an alternative to sbget(3).
See their manual pages for further details.

The utilities that have been changed to search for superblocks are
dumpfs(8), fsdb(8), ffsinfo(8), and fsck_ffs(8). Also, the prtblknos(8)
tool found in tools/diag/prtblknos searches for superblocks.

The UFS specific mount code uses the superblock search interface
when mounting the root filesystem and when the administrator doing
a mount(8) command specifies the force flag (-f). The standalone UFS
boot code (found in stand/libsa/ufs.c) uses the superblock search
code in the hope of being able to get the system up and running so
that fsck_ffs(8) can be used to get the filesystem cleaned up.

The following utilities have not been changed to search for
superblocks: clri(8), tunefs(8), snapinfo(8), fstyp(8), quot(8),
dump(8), fsirand(8), growfs(8), quotacheck(8), gjournal(8), and
glabel(8). When these utilities fail, they do report the cause of
the failure. The one exception is the tasting code used to try and
figure what a given disk contains. The tasting code will remain
silent so as not to put out a slew of messages as it trying to taste
every new mass storage device that shows up.

Reviewed by: kib
Reviewed by: Warner Losh
Tested by: Peter Holm
Differential Revision: https://reviews.freebsd.org/D36053
Sponsored by: The FreeBSD Foundation

Update fsdb(8) to reflect new structure of fsck_ffs(8).

The cleanup of fsck_ffs(8) in commit c0bfa109b942659f6 broke fsdb(8).
This commit adds the one-line update needed in fsdb(8) to make it
work with the new fsck_ffs(8) structure.

Reported by: Chuck Silvers
Tested by: Chuck Silvers
MFC after: 3 days

Revert "fsdb: add missing bufinit() call"

This reverts commit 84768d114951e88288024f09d4beae0956c3cf21.

fsdb: add missing bufinit() call

The bufinit() call in fsck_ffs was moved in commit f190f9193bc10
from a function that is shared with fsdb to one that is private to fsck_ffs,
so add a bufinit() call in fsdb to compensate for that.

Reviewed by: mckusick
Sponsored by: Netflix

Rewrite the disk I/O management system in fsck_ffs(8). Other than
making fsck_ffs(8) run faster, there should be no functional change.

The original fsck_ffs(8) had its own disk I/O management system.
When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8)
to do the necessary gjournal rollback. Rather than use the existing
fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly
when journalled soft updates were added in FreeBSD 9, code was added
to fsck_ffs(8) to do the necessary journal rollback. And once again,
rather than using either of the existing fsck_ffs(8) disk I/O
systems, it wrote its own from scratch. Lastly the fsdb(8) utility
uses the fsck_ffs(8) disk I/O management system. In preparation for
making the changes necessary to enable snapshots to be taken when
using journalled soft updates, it was necessary to have a single
disk I/O system used by all the various subsystems in fsck_ffs(8).

This commit merges the functionality required by all the different
subsystems into a single disk I/O system that supports all of their
needs. In so doing it picks up optimizations from each of them
with the results that each of the subsystems does fewer reads and
writes than it did with its own customized I/O system. It also
greatly simplifies making changes to fsck_ffs(8) since everything
goes through a single place. For example the ginode() function
fetches an inode from the disk. When inode check hashes were added,
they previously had to be checked in the code implementing inode
fetch in each of the three different disk I/O systems. Now they
need only be checked in ginode().

Tested by: Peter Holm
Sponsored by: Netflix

The fsdb(8) utility uses the fsck_ffs(8) disk I/O interfaces, so
switch from using libufs's bread() to using fsck_ffs's getdatablk()
when importing tools/diag/prtblnos's prtblknos().

Sponsored by: Netflix

fsck_ffs/fsdb: fix -fno-common build

This one is also a small list:

- 3x duplicate definition (ufs2_zino, returntosingle, nflag)
- 5x 'needs extern', 3/5 of which are referenced in fsdb

-fno-common will become the default in GCC10/LLVM11.

MFC after: 1 week

After a crash, a file that extends into indirect blocks may end up
shorter than its size resulting in a hole as its final block (which
is a violation of the invarients of the UFS filesystem).

Soft updates will always ensure that the file size is correct when
writing inodes to disk for files that contain only direct block
pointers. However soft updates does not roll back sizes for files
with indirect blocks that it has set to unallocated because their
contents have not yet been written to disk. Hence, the file can
appear to have a hole at its end because the block pointer has been
rolled back to zero when its inode was written to disk. Thus,
fsck_ffs calculates the last allocated block in the file. For files
that extend into indirect blocks, fsck_ffs checks for a size past
the last allocated block of the file and if that is found, shortens
the file to reference the last allocated block thus avoiding having
it reference a hole at its end.

Submitted by: Chuck Silvers <chs@netflix.com>
Tested by: Chuck Silvers <chs@netflix.com>
MFC after: 1 week
Sponsored by: Netflix

In preparation for adding inode check-hashes, change the fsck_ffs
inodirty() function to have a pointer to the inode being dirtied.
No functional change (as for now the parameter is ununsed).

Sponsored by: Netflix

Revert r313780 (UFS_ prefix)

Prefix UFS symbols with UFS_ to reduce namespace pollution

Followup to r313780. Also prefix ext2's and nandfs's versions with
EXT2_ and NANDFS_.

Reported by: kib
Reviewed by: kib, mckusick
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D9623

Fix buildworld after r328075, by also renaming cgget to cglookup in
fsdb.

Reported by: ohartmann@walstatt.org,david@catwhisker.org
Pointy hat to: mckusick

The fix in r327273 turns a memory leak into freeing wild pointer.
Fix this by freeing only the initialized pointer.

Plug memory leak by freeing wantedblk{32,64}.

CID: 273655, 273656

various: general adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.

prefix UFS symbols with UFS_ to reduce namespace pollution

Specifically:
ROOTINO -> UFS_ROOTINO
WINO -> UFS_WINO
NXADDR -> UFS_NXADDR
NDADDR -> UFS_NDADDR
NIADDR -> UFS_NIADDR
MAXSYMLINKLEN_UFS[12] -> UFS[12]_MAXSYMLINKLEN (for consistency)

Also prefix ext2's and nandfs's NDADDR and NIADDR with EXT2_ and NANDFS_

Reviewed by: kib, mckusick
Obtained from: NetBSD
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D9536

Use nitems() from sys/param.h.

Sponsored by: gandi.net (BSD Day Taiwan)

Revert 248634 and 248643 (e.g., restoring 248625 and 248639).

Build verified by: Glen Barber (gjb@)

Revert r248639 to fix build failure on head/

Fix the build after addition of cylinder group cacheing (r248625)

Reported by: Glen Barber (gjb@)
Pointy hat to: Kirk McKusick (mckusick@)

Fix sbin/ build with a 64-bit ino_t.

Original code by: Gleb Kurtsou

The dump, fsck_ffs, fsdb, fsirand, newfs, makefs, and quot utilities
include sys/time.h instead of time.h. This include is incorrect as
per the manpages for the APIs and the POSIX definitions. This commit
replaces sys/time.h where necessary with time.h.

The commit also includes some minor style(9) header fixup in newfs.

This commit is part of a larger effort by Garrett Cooper started in
//depot/user/gcooper/posix-conformance-work/ -- to make FreeBSD more
POSIX compliant.

Submitted by: Garrett Cooper yanegomi at gmail dot com

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

- Merge soft-updates journaling from projects/suj/head into head. This
brings in support for an optional intent log which eliminates the need
for background fsck on unclean shutdown.

Sponsored by: iXsystems, Yahoo!, and Juniper.
With help from: McKusick and Peter Holm

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

- Import the HEAD csup code which is the basis for the cvsmode work.

Update after function renames.

Sponsored by: home.pl

Allow fsdb to manipulate the birthtime entries on UFS2.

Approved by: jhb
MFC after: 1 month

o Implement findblk command: find the inode(s) owning the specified
disk block(s) number(s).

Obtained from: NetBSD
MFC after: 2 months

o Do recrack(arguments) for commands which actually take NAME as
arguments so we do not coredump at "help foo", "back bar" and such.

o Be consistent and print argc - 1 as a command arguments number in
all cases.

PR: bin/37096
Submitted by: Joshua Goodall
MFC after: 1 month

Use the new name H_SETSIZE instead of the old H_EVENT to set the history
size.

PR: 86355

The libedit update made a const cast necessary.

Make fsck WARNS=2 clean.

Make this WARNS=2 clean by
- #include <timeconv.h> for _time_to_time32 et al
- use (uintmax_t) and %j
- remove unused variable 'j' (from PR 39866)

PR: 39866
Submitted by: Dan Lukes <dan@obluda.cz>
Tested by: make universe

s/filesystem/file system/g as discussed on -developers

This commit adds basic support for the UFS2 filesystem. The UFS2
filesystem expands the inode to 256 bytes to make space for 64-bit
block pointers. It also adds a file-creation time field, an ability
to use jumbo blocks per inode to allow extent like pointer density,
and space for extended attributes (up to twice the filesystem block
size worth of attributes, e.g., on a 16K filesystem, there is space
for 32K of attributes). UFS2 fully supports and runs existing UFS1
filesystems. New filesystems built using newfs can be built in either
UFS1 or UFS2 format using the -O option. In this commit UFS1 is
the default format, so if you want to build UFS2 format filesystems,
you must specify -O 2. This default will be changed to UFS2 when
UFS2 proves itself to be stable. In this commit the boot code for
reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c)
as there is insufficient space in the boot block. Once the size of the
boot block is increased, this code can be defined.

Things to note: the definition of SBSIZE has changed to SBLOCKSIZE.
The header file <ufs/ufs/dinode.h> must be included before
<ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and
ufs_lbn_t.

Still TODO:
Verify that the first level bootstraps work for all the architectures.
Convert the utility ffsinfo to understand UFS2 and test growfs.
Add support for the extended attribute storage. Update soft updates
to ensure integrity of extended attribute storage. Switch the
current extended attribute interfaces to use the extended attribute
storage. Add the extent like functionality (framework is there,
but is currently never used).

Sponsored by: DARPA & NAI Labs.
Reviewed by: Poul-Henning Kamp <phk@freebsd.org>

more file system > filesystem

o __P removal.
o ansi function definitions.
o main prototype removal
o unifdef __STDC__

Remove 'register' keyword.
It does not help modern compilers, and some may take some hit from it.
(I also found several functions that listed *every* of its 10 local vars with
"register" -- just how many free registers do people think machines have?)

Something i always wanted to see: add a function to print the list of
blocks allocated by some inode. Indirect blocks are printed
recursively, so beware :), the list could become lengthy...
(We should probably add some output pager to fsdb.)

MFC after: 1 month

Avoid pointless initialization of global variables to 0. This only
bloats the resulting binary file by forcing them out of .bss into
.data, while the C standard already guarantees them to become
initialized to 0 at program startup.

MFC after: 1 week

Don't exit with -1 if the user typed "quit".

MFC after: 1 week

Allow fsdb the ability to work with entries named with whitespace embedded.

This works by retokenizing a line with a split limit so that if the
argument count for a command is greater than the number of arguments
formed by splitting apart the line of user input, the last argument
is instead all of the remainder of the input line.

Yes, I needed this capability at one point to fix a filesystem manually,
which happened to break with a problematic space-containing directory
entry.

Check that the mode argument to fsdb's `chmod' command contains no
inode type bits set. Previously it would let you set IFMT bits (but
not clear them). The `chtype' command should be be used instead
for changing the inode type; having chmod half-work only causes
confusion.

*** empty log message ***

In fsdb, call sblock_init() which is now necessary to initialise
the global variable dev_bsize. Add a prototype for sblock_init()
to fsck.h, and set the return type correctly.

Remove unused includes.

$Id$ -> $FreeBSD$

More egcs warning fixes:
o main returns int not void
o use braces to avoid potentially ambiguous else

Note: The fix to natd is potentially functional in nature since I used
the indentation as the right thing rather than the struct semantics.
Someone more familiar with the code should double check me on this one.

Reviewed by: obrien and chuckr

Fix some calculations that use sizeof to attempt to find the end of an
array that were doing sizeof on an unrelated variable. This just happened
to work right on the i386, but would not on the alpha.

PR: bin/8427

Correct use of .Nm. Use .Bl/.El for enumerating options. Use .An. Correct
formatting of rcsid. Remove unused #includes. Do not use memory after
freeing it.

Remove __progname. Make -f a no-op flag as stated in the man page. Remove
unused variables.

Implement a -r option to fsdb(8), ``read/only''.

Missing $Id$

Make this compile. Mostly use the new names for the ctime/atime/mtime
stamps in the inodes and call one of fsck's utility funcs with a new arg.

Get rid of useless -f flag (though left for historical reasons).

Add chlen command so you can set the size of an inode. This was handy
in order to create sparse directory files that caused a panic of a
filesystem where fsck would not find anything. A fix for fsck is in the
make but still has to be reviewed by Kirk McKusick.

ts_sec -> tv_sec
ts_nsec -> tv_nsec

Commit FreeBSD-specific changes. Mainly to do with structure layout
differences that we dont have.

Whoops. RE-Import NetBSD's fsdb - I believe this was written by John Kohl.

Obtained from: NetBSD