#
772430dd |
|
17-Nov-2023 |
Kirk McKusick <mckusick@FreeBSD.org> |
Ensure I/O buffers in libufs(3) are 128-byte aligned. Various disk controllers require their buffers to be aligned to a cache-line size (128 bytes). For buffers allocated in structures, ensure that they are 128-byte aligned. Use aligned_malloc to allocate memory to ensure that the returned memory is 128-byte aligned. While we are here, we replace the dynamically allocated inode buffer with a buffer allocated in the uufsd structure just as the superblock and cylinder group buffers do. This can be removed if/when the kernel is fixed. Because this problem has existed on one I/O subsystem or another since the 1990's, we are probably stuck with dealing with it forever. The problem most recent showed up in Azure, see: https://reviews.freebsd.org/D41728 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=267654 Before these fixes were applied, it was confirmed that the changes in this commit also fixed the issue in Azure. Reviewed-by: Warner Losh, kib Tested-by: Souradeep Chakrabarti of Microsoft (earlier version) PR: 267654 Differential Revision: https://reviews.freebsd.org/D41724
|
#
b3e76948 |
|
16-Aug-2023 |
Warner Losh <imp@FreeBSD.org> |
Remove $FreeBSD$: two-line .h pattern Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/
|
#
4d846d26 |
|
10-May-2023 |
Warner Losh <imp@FreeBSD.org> |
spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch up to that fact and revert to their recommended match of BSD-2-Clause. Discussed with: pfg MFC After: 3 days Sponsored by: Netflix
|
#
e6886616 |
|
13-Aug-2022 |
Kirk McKusick <mckusick@FreeBSD.org> |
Move the ability to search for alternate UFS superblocks from fsck_ffs(8) into ffs_sbsearch() to allow use by other parts of the system. Historically only fsck_ffs(8), the UFS filesystem checker, had code to track down and use alternate UFS superblocks. Since fsdb(8) used much of the fsck_ffs(8) implementation it had some ability to track down alternate superblocks. This change extracts the code to track down alternate superblocks from fsck_ffs(8) and puts it into a new function ffs_sbsearch() in sys/ufs/ffs/ffs_subr.c. Like ffs_sbget() and ffs_sbput() also found in ffs_subr.c, these functions can be used directly by the kernel subsystems. Additionally they are exported to the UFS library, libufs(8) so that they can be used by user-level programs. The new functions added to libufs(8) are sbfind(3) that is an alternative to sbread(3) and sbsearch(3) that is an alternative to sbget(3). See their manual pages for further details. The utilities that have been changed to search for superblocks are dumpfs(8), fsdb(8), ffsinfo(8), and fsck_ffs(8). Also, the prtblknos(8) tool found in tools/diag/prtblknos searches for superblocks. The UFS specific mount code uses the superblock search interface when mounting the root filesystem and when the administrator doing a mount(8) command specifies the force flag (-f). The standalone UFS boot code (found in stand/libsa/ufs.c) uses the superblock search code in the hope of being able to get the system up and running so that fsck_ffs(8) can be used to get the filesystem cleaned up. The following utilities have not been changed to search for superblocks: clri(8), tunefs(8), snapinfo(8), fstyp(8), quot(8), dump(8), fsirand(8), growfs(8), quotacheck(8), gjournal(8), and glabel(8). When these utilities fail, they do report the cause of the failure. The one exception is the tasting code used to try and figure what a given disk contains. The tasting code will remain silent so as not to put out a slew of messages as it trying to taste every new mass storage device that shows up. Reviewed by: kib Reviewed by: Warner Losh Tested by: Peter Holm Differential Revision: https://reviews.freebsd.org/D36053 Sponsored by: The FreeBSD Foundation
|
#
b21582ee |
|
30-Jul-2022 |
Kirk McKusick <mckusick@FreeBSD.org> |
Add a flags parameter to the ffs_sbget() function that reads UFS superblocks. Rather than trying to shoehorn flags into the requested superblock address, create a separate flags parameter to the ffs_sbget() function in sys/ufs/ffs/ffs_subr.c. The ffs_sbget() function is used both in the kernel and in user-level utilities through export to the sbget() function in the libufs(3) library (see sbget(3) for details). The kernel uses ffs_sbget() when mounting UFS filesystems, in the glabel(8) and gjournal(8) GEOM utilities, and in the standalone library used when booting the system from a UFS root filesystem. The ffs_sbget() function reads the superblock located at the byte offset specified by its sblockloc parameter. The value UFS_STDSB may be specified for sblockloc to request that the standard location for the superblock be read. The two existing options are now flags: UFS_NOHASHFAIL will note if the check hash is wrong but will still return the superblock. This is used by the bootstrap code to give the system a chance to come up so that fsck can be run to correct the problem. UFS_NOMSG indicates that superblock inconsistency error messages should not be printed. It is used by programs like fsck that want to print their own error message and programs like glabel(8) that just want to know if a UFS filesystem exists on a partition. One additional flag is added: UFS_NOCSUM causes only the superblock itself to be returned, but does not read in any auxiliary data structures like the cylinder group summary information. It is used by clients like glabel(8) that just want to check for possible filesystem types. Using UFS_NOCSUM skips the superblock checks for csum data which allows superblocks that have corrupted csum data to be read and used. The validate_sblock() function checks that the superblock has not been corrupted in a way that can crash or hang the system. Unless the UFS_NOMSG flag is specified, it will print out any errors that it finds. Prior to this commit, validate_sblock() returned as soon as it found an inconsistency so would print at most one message. It now does all its checks so when UFS_NOMSG has not been specified will print out everything that it finds inconsistent. Sponsored by: The FreeBSD Foundation
|
#
82e72f1d |
|
24-Jul-2022 |
Kirk McKusick <mckusick@FreeBSD.org> |
Add d_sblockloc to libufs(3) disk structure to allow options to be added. By making the disk block parameter used by the libufs(3) sbread(3) function visible, applications using sbread(3) can set their own addition options such as using the STDSB_NOHASHFAIL request to say that they want the superblock read to succeed even when the superblock checkhash is incorrect. While here also add an error message when a check-hash failure is detected.
|
#
b366ee48 |
|
14-Nov-2021 |
Kirk McKusick <mckusick@FreeBSD.org> |
Consolodate four copies of the STDSB define into a single place. The STDSB macro is passed to the ffs_sbget() routine to fetch a UFS/FFS superblock "from the stadard place". It was identically defined in lib/libufs/libufs.h, stand/libsa/ufs.c, sys/ufs/ffs/ffs_extern.h, and sys/ufs/ffs/ffs_subr.c. Delete it from these four files and define it instead in sys/ufs/ffs/fs.h. All existing uses of this macro already include sys/ufs/ffs/fs.h so no include changes need to be made. No functional change intended. Sponsored by: Netflix
|
#
85ee267a |
|
19-Sep-2020 |
Kirk McKusick <mckusick@FreeBSD.org> |
Update the libufs cgget() and cgput() interfaces to have a similar API to the sbget() and sbput() interfaces. Specifically they take a file descriptor pointer rather than the struct uufsd *disk pointer used by the libufs cgread() and cgwrite() interfaces. Update fsck_ffs to use these revised interfaces. No functional changes intended. Sponsored by: Netflix
|
#
92c839a1 |
|
23-Jun-2020 |
Kirk McKusick <mckusick@FreeBSD.org> |
The libufs library needs to track and free the new fs_si structure in addition to the fs_csp structure that it references. PR: 247425 Sponsored by: Netflix
|
#
7ac5cc13 |
|
23-Jun-2020 |
Kirk McKusick <mckusick@FreeBSD.org> |
Align comments in struct uufsd structure. No semantic change. Sponsored by: Netflix
|
#
8f829a5c |
|
11-Dec-2018 |
Kirk McKusick <mckusick@FreeBSD.org> |
Continuing efforts to provide hardening of FFS. This change adds a check hash to the filesystem inodes. Access attempts to files associated with an inode with an invalid check hash will fail with EINVAL (Invalid argument). Access is reestablished after an fsck is run to find and validate the inodes with invalid check-hashes. This check avoids a class of filesystem panics related to corrupted inodes. The hash is done using crc32c. Note this check-hash is for the inode itself and not any of its indirect blocks. Check-hash validation may be extended to also cover indirect block pointers, but that will be a separate (and more costly) feature. Check hashes are added only to UFS2 and not to UFS1 as UFS1 is primarily used in embedded systems with small memories and low-powered processors which need as light-weight a filesystem as possible. Reviewed by: kib Tested by: Peter Holm Sponsored by: Netflix
|
#
fb14e73c |
|
05-Dec-2018 |
Kirk McKusick <mckusick@FreeBSD.org> |
Normally when an attempt is made to mount a UFS/FFS filesystem whose superblock has a check-hash error, an error message noting the superblock check-hash failure is printed and the mount fails. The administrator then runs fsck to repair the filesystem and when successful, the filesystem can once again be mounted. This approach fails if the filesystem in question is a root filesystem from which you are trying to boot. Here, the loader fails when trying to access the filesystem to get the kernel to boot. So it is necessary to allow the loader to ignore the superblock check-hash error and make a best effort to read the kernel. The filesystem may be suffiently corrupted that the read attempt fails, but there is no harm in trying since the loader makes no attempt to write to the filesystem. Once the kernel is loaded and starts to run, it attempts to mount its root filesystem. Once again, failure means that it breaks to its prompt to ask where to get its root filesystem. Unless you have an alternate root filesystem, you are stuck. Since the root filesystem is initially mounted read-only, it is safe to make an attempt to mount the root filesystem with the failed superblock check-hash. Thus, when asked to mount a root filesystem with a failed superblock check-hash, the kernel prints a warning message that the root filesystem superblock check-hash needs repair, but notes that it is ignoring the error and proceeding. It does mark the filesystem as needing an fsck which prevents it from being enabled for writing until fsck has been run on it. The net effect is that the reboot fails to single user, but at least at that point the administrator has the tools at hand to fix the problem. Reported by: Rick Macklem (rmacklem@) Discussed with: Warner Losh (imp@) Sponsored by: Netflix
|
#
9fc5d538 |
|
13-Nov-2018 |
Kirk McKusick <mckusick@FreeBSD.org> |
In preparation for adding inode check-hashes, clean up and document the libufs interface for fetching and storing inodes. The undocumented getino / putino interface has been replaced with a new getinode / putinode interface. Convert the utilities that had been using the undocumented interface to use the new documented interface. No functional change (as for now the libufs library does not do inode check-hashes). Reviewed by: kib Tested by: Peter Holm Sponsored by: Netflix
|
#
dffce215 |
|
25-Jan-2018 |
Kirk McKusick <mckusick@FreeBSD.org> |
Refactoring of reading and writing of the UFS/FFS superblock. Specifically reading is done if ffs_sbget() and writing is done in ffs_sbput(). These functions are exported to libufs via the sbget() and sbput() functions which then used in the various filesystem utilities. This work is in preparation for adding subperblock check hashes. No functional change intended. Reviewed by: kib
|
#
72f854ce |
|
17-Jan-2018 |
Kirk McKusick <mckusick@FreeBSD.org> |
Correct fsck journal-recovery code to update a cylinder-group check-hash after making changes to the cylinder group. The problem was that the journal-recovery code was calling the libufs bwrite() function instead of the cgput() function. The cgput() function updates the cylinder-group check-hash before writing the cylinder group. This change required the additions of the cgget() and cgput() functions to the libufs API to avoid a gratuitous bcopy of every cylinder group to be read or written. These new functions have been added to the libufs manual pages. This was the first opportunity that I have had to use and document the use of the EDOOFUS error code. Reviewed by: kib Reported by: emaste and others
|
#
5e53a4f9 |
|
25-Nov-2017 |
Pedro F. Giffuni <pfg@FreeBSD.org> |
lib: further adoption of SPDX licensing ID tags. Mainly focus on files that use BSD 2-Clause license, however the tool I was using mis-identified many licenses so this was mostly a manual - error prone - task. The Software Package Data Exchange (SPDX) group provides a specification to make it easier for automated tools to detect and summarize well known opensource licenses. We are gradually adopting the specification, noting that the tags are considered only advisory and do not, in any way, superceed or replace the license texts.
|
#
75e3597a |
|
21-Sep-2017 |
Kirk McKusick <mckusick@FreeBSD.org> |
Continuing efforts to provide hardening of FFS, this change adds a check hash to cylinder groups. If a check hash fails when a cylinder group is read, no further allocations are attempted in that cylinder group until it has been fixed by fsck. This avoids a class of filesystem panics related to corrupted cylinder group maps. The hash is done using crc32c. Check hases are added only to UFS2 and not to UFS1 as UFS1 is primarily used in embedded systems with small memories and low-powered processors which need as light-weight a filesystem as possible. Specifics of the changes: sys/sys/buf.h: Add BX_FSPRIV to reserve a set of eight b_xflags that may be used by individual filesystems for their own purpose. Their specific definitions are found in the header files for each filesystem that uses them. Also add fields to struct buf as noted below. sys/kern/vfs_bio.c: It is only necessary to compute a check hash for a cylinder group when it is actually read from disk. When calling bread, you do not know whether the buffer was found in the cache or read. So a new flag (GB_CKHASH) and a pointer to a function to perform the hash has been added to breadn_flags to say that the function should be called to calculate a hash if the data has been read. The check hash is placed in b_ckhash and the B_CKHASH flag is set to indicate that a read was done and a check hash calculated. Though a rather elaborate mechanism, it should also work for check hashing other metadata in the future. A kernel internal API change was to change breada into a static fucntion and add flags and a function pointer to a check-hash function. sys/ufs/ffs/fs.h: Add flags for types of check hashes; stored in a new word in the superblock. Define corresponding BX_ flags for the different types of check hashes. Add a check hash word in the cylinder group. sys/ufs/ffs/ffs_alloc.c: In ffs_getcg do the dance with breadn_flags to get a check hash and if one is provided, check it. sys/ufs/ffs/ffs_vfsops.c: Copy across the BX_FFSTYPES flags in background writes. Update the check hash when writing out buffers that need them. sys/ufs/ffs/ffs_snapshot.c: Recompute check hash when updating snapshot cylinder groups. sys/libkern/crc32.c: lib/libufs/Makefile: lib/libufs/libufs.h: lib/libufs/cgroup.c: Include libkern/crc32.c in libufs and use it to compute check hashes when updating cylinder groups. Four utilities are affected: sbin/newfs/mkfs.c: Add the check hashes when building the cylinder groups. sbin/fsck_ffs/fsck.h: sbin/fsck_ffs/fsutil.c: Verify and update check hashes when checking and writing cylinder groups. sbin/fsck_ffs/pass5.c: Offer to add check hashes to existing filesystems. Precompute check hashes when rebuilding cylinder group (although this will be done when it is written in fsutil.c it is necessary to do it early before comparing with the old cylinder group) sbin/dumpfs/dumpfs.c Print out the new check hash flag(s) sbin/fsdb/Makefile: Needs to add libufs now used by pass5.c imported from fsck_ffs. Reviewed by: kib Tested by: Peter Holm (pho)
|
#
69921123 |
|
23-May-2017 |
Konstantin Belousov <kib@FreeBSD.org> |
Commit the 64-bit inode project. Extend the ino_t, dev_t, nlink_t types to 64-bit ints. Modify struct dirent layout to add d_off, increase the size of d_fileno to 64-bits, increase the size of d_namlen to 16-bits, and change the required alignment. Increase struct statfs f_mntfromname[] and f_mntonname[] array length MNAMELEN to 1024. ABI breakage is mitigated by providing compatibility using versioned symbols, ingenious use of the existing padding in structures, and by employing other tricks. Unfortunately, not everything can be fixed, especially outside the base system. For instance, third-party APIs which pass struct stat around are broken in backward and forward incompatible ways. Kinfo sysctl MIBs ABI is changed in backward-compatible way, but there is no general mechanism to handle other sysctl MIBS which return structures where the layout has changed. It was considered that the breakage is either in the management interfaces, where we usually allow ABI slip, or is not important. Struct xvnode changed layout, no compat shims are provided. For struct xtty, dev_t tty device member was reduced to uint32_t. It was decided that keeping ABI compat in this case is more useful than reporting 64-bit dev_t, for the sake of pstat. Update note: strictly follow the instructions in UPDATING. Build and install the new kernel with COMPAT_FREEBSD11 option enabled, then reboot, and only then install new world. Credits: The 64-bit inode project, also known as ino64, started life many years ago as a project by Gleb Kurtsou (gleb). Kirk McKusick (mckusick) then picked up and updated the patch, and acted as a flag-waver. Feedback, suggestions, and discussions were carried by Ed Maste (emaste), John Baldwin (jhb), Jilles Tjoelker (jilles), and Rick Macklem (rmacklem). Kris Moore (kris) performed an initial ports investigation followed by an exp-run by Antoine Brodin (antoine). Essential and all-embracing testing was done by Peter Holm (pho). The heavy lifting of coordinating all these efforts and bringing the project to completion were done by Konstantin Belousov (kib). Sponsored by: The FreeBSD Foundation (emaste, kib) Differential revision: https://reviews.freebsd.org/D10439
|
#
49e51fb0 |
|
11-Feb-2011 |
Konstantin Belousov <kib@FreeBSD.org> |
Replace ERROR() macro with inline function. In-tree gcc cannot tolerate the construct like printf("%\s", NULL) resulting from macroexpand of ERROR(u, NULL), making it impossible to use LIBUFS_DEBUGGING. With inline function, compiler cannot detect the NULL argument to known function and does not try to convert it into puts(). In collaboration with: pho
|
#
a7d5f7eb |
|
19-Oct-2010 |
Jamie Gritton <jamie@FreeBSD.org> |
A new jail(8) with a configuration file, to replace the work currently done by /etc/rc.d/jail.
|
#
113db2dd |
|
24-Apr-2010 |
Jeff Roberson <jeff@FreeBSD.org> |
- Merge soft-updates journaling from projects/suj/head into head. This brings in support for an optional intent log which eliminates the need for background fsck on unclean shutdown. Sponsored by: iXsystems, Yahoo!, and Juniper. With help from: McKusick and Peter Holm
|
#
fe0506d7 |
|
09-Mar-2010 |
Marcel Moolenaar <marcel@FreeBSD.org> |
Create the altix project branch. The altix project will add support for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting is a two-module system, consisting of a base compute module and a CPU expansion module. SGI's NUMAFlex architecture can be an excellent platform to test CPU affinity and NUMA-aware features in FreeBSD.
|
#
d7f03759 |
|
19-Oct-2008 |
Ulf Lilleengen <lulf@FreeBSD.org> |
- Import the HEAD csup code which is the basis for the cvsmode work.
|
#
20a0f65b |
|
16-Dec-2007 |
Poul-Henning Kamp <phk@FreeBSD.org> |
Add a berase() function which uses ioctl(DIOCGDELETE) to erase a slab of the disk.
|
#
cf4652e7 |
|
31-Oct-2006 |
Pawel Jakub Dawidek <pjd@FreeBSD.org> |
Implement cgwrite1(3) function which stored a given cylinder group on disk. Sponsored by: home.pl
|
#
b52f85e7 |
|
09-Jun-2003 |
Juli Mallett <jmallett@FreeBSD.org> |
Remove ufs_disk_ctor and ufs_disk_dtor, they never came to fruition. I do not know of any software using them, and there is no "published API" for libufs, as it were.
|
#
7dc95357 |
|
29-Jan-2003 |
Juli Mallett <jmallett@FreeBSD.org> |
API for opening (and tracking) writable file descriptors per disk.
|
#
d934deef |
|
23-Jan-2003 |
Juli Mallett <jmallett@FreeBSD.org> |
API to fillout a blank disk. For e.g. newfs.
|
#
cc3dd528 |
|
18-Jan-2003 |
Juli Mallett <jmallett@FreeBSD.org> |
Store not only the current cylinder group in the series (i.e. next that needs to be read in) but also the last cylinder group in the series (i.e. what is stored in the structure).
|
#
22ec2ef3 |
|
18-Jan-2003 |
Juli Mallett <jmallett@FreeBSD.org> |
Add facility to read one, or a string of, cylinger groups.
|
#
65612858 |
|
18-Jan-2003 |
Juli Mallett <jmallett@FreeBSD.org> |
Hunt for a disk to operate on, if we're passed a partition mountpoint, etc. Concept reviewed by: phk
|
#
063a27dc |
|
17-Jan-2003 |
Juli Mallett <jmallett@FreeBSD.org> |
Fix typo.
|
#
49b2a686 |
|
17-Jan-2003 |
Juli Mallett <jmallett@FreeBSD.org> |
Nuke dumb error reporting code, people can just use disk::d_error. Unify the DEBUG and d_error initialisation into an ERROR macro, which can both trace and set the d_error field. Much a more meaningful thing, I should say.
|
#
8b8cd355 |
|
22-Oct-2002 |
Juli Mallett <jmallett@FreeBSD.org> |
Add the concept of a per-disk error string, and a function which prints it along with the errno, if one is set.
|
#
95289fb2 |
|
22-Aug-2002 |
Juli Mallett <jmallett@FreeBSD.org> |
Wrap the header to prevent multiple inclusion, and mark the DECLS section. Reminded by: Rachel Hestilow <hestilow@ximian.com>
|
#
20938dbf |
|
30-Jun-2002 |
Juli Mallett <jmallett@FreeBSD.org> |
Add libufs, a library for dealing with UFS filesystems from userland to the build. It is here to compartmentalise functionality currently duplicated in many notable programs in the base system. It currently handles block reads and writes, as well as reading and writing of the filesystem superblock, and the reading/lookup of inode data. It supports both UFS and UFS2. I will be maintaining it, and porting programs to use it, however for now, it is simply being built as part of world.
|