#
deeb1d34 |
|
20-Jan-2024 |
John F. Carr <jfc@mit.edu> |
fsck_msdosfs: do not call checksize() if the cluster is bad PR: 276464 MFC after: 1 week
|
#
eba230af |
|
25-Sep-2023 |
John Baldwin <jhb@FreeBSD.org> |
Purge more stray embedded $FreeBSD$ strings These do not use __FBSDID but instead use bare char arrays. Reviewed by: imp, emaste Differential Revision: https://reviews.freebsd.org/D41957
|
#
4d846d26 |
|
10-May-2023 |
Warner Losh <imp@FreeBSD.org> |
spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch up to that fact and revert to their recommended match of BSD-2-Clause. Discussed with: pfg MFC After: 3 days Sponsored by: Netflix
|
#
890cae19 |
|
03-Nov-2021 |
Xin LI <delphij@FreeBSD.org> |
fsck_msdosfs: truncate directory entry when the head pointer is invalid. As far as we know, there is no FAT implementation that supported hard links, and our msdosfs driver assumed one cluster chain is only referenced by one directory entry and clears it out when the file is deleted. On the other hand, the current code would proceed with checkchain() when the directory entry's head cluster is a valid numbered cluster without checking if it was a valid head node of a cluster chain. So if the cluster do not being a chain (e.g. CLUST_FREE, CLUST_BAD), or was already referenced by another directory entry, this would trigger an assertion in check_chain() at a later time. Fix this by giving the user an option to truncate the directory entry when the head cluster is an invalid cluster, an visited head node, or not a head node. Reported by: NetApp (kevans@) Reviewed by: kevans, emaste (no objection) MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D32699
|
#
5e822110 |
|
27-Sep-2020 |
Xin LI <delphij@FreeBSD.org> |
Use %ju and cast to (uintmax_t) to avoid using PRI* macros. Suggested by: kevlo
|
#
6155466a |
|
23-Sep-2020 |
Xin LI <delphij@FreeBSD.org> |
Fix build. Pointy hat to: delphij MFC after: 3 days
|
#
4d2c300b |
|
23-Sep-2020 |
Xin LI <delphij@FreeBSD.org> |
sbin/fsck_msdosfs: Fix an integer overflow on 32-bit platforms. The purpose of checksize() is to verify that the referenced cluster chain size matches the recorded file size (up to 2^32 - 1) in the directory entry. We follow the cluster chain, then multiple the cluster count by bytes per cluster to get the physical size, then check it against the recorded size. When a file is close to 4 GiB (between 4GiB - cluster size and 4GiB, both non-inclusive), the product of cluster count and bytes per cluster would be exactly 4 GiB. On 32-bit systems, because size_t is 32-bit, this would wrap back to 0, which will cause the file be truncated to 0. Fix this by using 64-bit physicalSize instead. This fix is inspired by an Android change request at https://android-review.googlesource.com/c/platform/external/fsck_msdos/+/1428461 PR: 249533 Reviewed by: kevlo MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D26524
|
#
9708ba9f |
|
02-Jan-2020 |
Xin LI <delphij@FreeBSD.org> |
Reduce memory footprint of fsck_msdosfs. This is a re-apply r356249 with changes to make GCC happy. This utility was initially written for FAT12/16, which were inherently small. When FAT32 support was added, the old data structure and algorithms remain used with minimal changes. With growing size of FAT32 media, the current data structure that requires 4 32-bit variables per each FAT32 table entry would consume up to 4 GiB of RAM, which can be too big for systems with limited RAM available. Address this by taking a different approach of validating the FAT. The FAT is essentially a set of linked lists of chains that was referenced by directory entries, and the checker needs to make sure that the linked chains of clusters do not have cross-linked chains, and every chain were referenced by one and only one directory entry. Instead of keeping track of the chain's 'head' cluster number, the size of the chain, the used status of the chain and the "next" pointer which is content of the FAT table, we create accessors for the FAT table data for the "next" pointer, and keep only one bit to indicate if the current cluster is a 'head' node of a cluster chain, in a bitmap. We further overhaul the FAT checker to find out the possible head nodes by excluding ones that are not (in other words, nodes that have some other nodes claiming them as the next node) instead of marking the head nodes for each node on the chain. This approach greatly reduced the complexiety of computation from O(N^2) worst case, to an O(N) scan for worst case. The file (cluster chain) length is not useful for the FAT checker, so don't bother to calculate them in the FAT checker and instead leave the task to the directory structure check, at which point we would have non-crossed cluster chains, and we are guaranteed that each cluster will be visited for at most one time. When checking the directory structures, we use the head node indicator to as the visited (used) flag: every cluster chain can only be referenced by one directory entry, so we clear them when calculating the length of the chain, and we can immediately tell if there are anomalies in the directory entry. As a result, the required RAM size is now 1 bit per each entry of the FAT table, plus memory needed to hold the FAT table in memory, instead of 16 bytes (=128 bits) per each entry. For FAT12 and FAT16, we will load the whole FAT table into memory as they are smaller than 128KiB, and for FAT32, we first attempt to mmap() it into memory, and when that fails, we would fall back to a simple LRU cache of 4 MiB of RAM. sbin/fsck_msdosfs/boot.c: - Added additional sanity checks for valid FAT32/FAT16/FAT12 cluster number. - FAT32: check if root directory starts with a valid cluster number, moved from dir.c. There is no point to proceed if the filesystem is already damaged beyond repair. sbin/fsck_msdosfs/check.c: - Combine phase 1 and phase 2, now that the readfat() is able to detect cross chains. sbin/fsck_msdosfs/dir.c: - Refactor code to use FAT accessor instead of accessing the internal representation of FAT table. - Make use of the cluster chain head bitmap. - Clarify and simplify directory entry check, remove unnecessary checks that are would be done at a later time (for example, whether the directory's second cluster is a valid one, which is examined more throughly in a later checkchain() and does not prevent us from proceeding further). sbin/fsck_msdosfs/dosfs.h: - Remove internal representation of FAT table, which is replaced by the head bitmap that is opaque to other code. - Added a special CLUST_DEAD cluster type to indicate errors. sbin/fsck_msdosfs/ext.h: - Added a flag that overrides mmap(2) setting. The corresponding command line option, -M is intentionally undocumented as we do not expect users to need it. - Added accessors for FAT table and convert existing interface to use it. sbin/fsck_msdosfs/fat.c: - Added head bitmap to represent whether a cluster is a head cluster. - Converted FAT internal representation to accessors. - Implemented a LRU cache for FAT32 when mmap(2) should not or can not be used. - _readfat: Attempt a mmap(2) and fall back to regular read for non-FAT32 file systems; use the LRU cache for FAT32 and prepopulate the cache with the first 4MiB of the entries. - readfat: Added support of head bitmap and use the population scan to detect bogus chains. - clusterdiff: removed, FATs are copied from the checked copy via writefat()/copyfat(). - checkchain: calculates the length of a cluster chain and make sure that it ends with a valid EOF marker. - clearchain: follow and clear a chain and maintain the free cluster count. - checklost: convert to use head bitmap. At the end of all other scans, the remaining 'head' nodes are leaders of lost cluster chains. sbin/fsck_msdosfs/fat.c: - Added a new -M option which is intentionally undocumented, to disable the use of mmap(). Reviewed by: kevlo MFC after: 1 month Relnotes: yes Differential Revision: https://reviews.freebsd.org/D22965
|
#
73db93b8 |
|
01-Jan-2020 |
Xin LI <delphij@FreeBSD.org> |
Revert r356249 for now as it broke GCC builds.
|
#
b06cf1e4 |
|
01-Jan-2020 |
Xin LI <delphij@FreeBSD.org> |
Reduce memory footprint of fsck_msdosfs. This utility was initially written for FAT12/16, which were inherently small. When FAT32 support was added, the old data structure and algorithms remain used with minimal changes. With growing size of FAT32 media, the current data structure that requires 4 32-bit variables per each FAT32 table entry would consume up to 4 GiB of RAM, which can be too big for systems with limited RAM available. Address this by taking a different approach of validating the FAT. The FAT is essentially a set of linked lists of chains that was referenced by directory entries, and the checker needs to make sure that the linked chains of clusters do not have cross-linked chains, and every chain were referenced by one and only one directory entry. Instead of keeping track of the chain's 'head' cluster number, the size of the chain, the used status of the chain and the "next" pointer which is content of the FAT table, we create accessors for the FAT table data for the "next" pointer, and keep only one bit to indicate if the current cluster is a 'head' node of a cluster chain, in a bitmap. We further overhaul the FAT checker to find out the possible head nodes by excluding ones that are not (in other words, nodes that have some other nodes claiming them as the next node) instead of marking the head nodes for each node on the chain. This approach greatly reduced the complexiety of computation from O(N^2) worst case, to an O(N) scan for worst case. The file (cluster chain) length is not useful for the FAT checker, so don't bother to calculate them in the FAT checker and instead leave the task to the directory structure check, at which point we would have non-crossed cluster chains, and we are guaranteed that each cluster will be visited for at most one time. When checking the directory structures, we use the head node indicator to as the visited (used) flag: every cluster chain can only be referenced by one directory entry, so we clear them when calculating the length of the chain, and we can immediately tell if there are anomalies in the directory entry. As a result, the required RAM size is now 1 bit per each entry of the FAT table, plus memory needed to hold the FAT table in memory, instead of 16 bytes (=128 bits) per each entry. For FAT12 and FAT16, we will load the whole FAT table into memory as they are smaller than 128KiB, and for FAT32, we first attempt to mmap() it into memory, and when that fails, we would fall back to a simple LRU cache of 4 MiB of RAM. sbin/fsck_msdosfs/boot.c: - Added additional sanity checks for valid FAT32/FAT16/FAT12 cluster number. - FAT32: check if root directory starts with a valid cluster number, moved from dir.c. There is no point to proceed if the filesystem is already damaged beyond repair. sbin/fsck_msdosfs/check.c: - Combine phase 1 and phase 2, now that the readfat() is able to detect cross chains. sbin/fsck_msdosfs/dir.c: - Refactor code to use FAT accessor instead of accessing the internal representation of FAT table. - Make use of the cluster chain head bitmap. - Clarify and simplify directory entry check, remove unnecessary checks that are would be done at a later time (for example, whether the directory's second cluster is a valid one, which is examined more throughly in a later checkchain() and does not prevent us from proceeding further). sbin/fsck_msdosfs/dosfs.h: - Remove internal representation of FAT table, which is replaced by the head bitmap that is opaque to other code. - Added a special CLUST_DEAD cluster type to indicate errors. sbin/fsck_msdosfs/ext.h: - Added a flag that overrides mmap(2) setting. The corresponding command line option, -M is intentionally undocumented as we do not expect users to need it. - Added accessors for FAT table and convert existing interface to use it. sbin/fsck_msdosfs/fat.c: - Added head bitmap to represent whether a cluster is a head cluster. - Converted FAT internal representation to accessors. - Implemented a LRU cache for FAT32 when mmap(2) should not or can not be used. - _readfat: Attempt a mmap(2) and fall back to regular read for non-FAT32 file systems; use the LRU cache for FAT32 and prepopulate the cache with the first 4MiB of the entries. - readfat: Added support of head bitmap and use the population scan to detect bogus chains. - clusterdiff: removed, FATs are copied from the checked copy via writefat()/copyfat(). - checkchain: calculates the length of a cluster chain and make sure that it ends with a valid EOF marker. - clearchain: follow and clear a chain and maintain the free cluster count. - checklost: convert to use head bitmap. At the end of all other scans, the remaining 'head' nodes are leaders of lost cluster chains. sbin/fsck_msdosfs/fat.c: - Added a new -M option which is intentionally undocumented, to disable the use of mmap(). Reviewed by: kevlo MFC after: 1 month Relnotes: yes Differential Revision: https://reviews.freebsd.org/D22965
|
#
eb1c42c1 |
|
15-Sep-2019 |
Xin LI <delphij@FreeBSD.org> |
Avoid mixing cluster numbers and sector numbers. Makes code more readable. Obtained from: NetBSD MFC after: 2 weeks
|
#
07282103 |
|
03-Sep-2019 |
Xin LI <delphij@FreeBSD.org> |
Correct overflow logic in fullpath(). Obtained from: OpenBSD MFC after: 3 days
|
#
b9919ec9 |
|
18-Aug-2019 |
Xin LI <delphij@FreeBSD.org> |
Remove redundant check and wrong fix: fat.c checks already take care about cluster chains. Obtained from: OpenBSD MFC after: 2 weeks
|
#
bf6ed416 |
|
11-Jun-2019 |
Xin LI <delphij@FreeBSD.org> |
Avoid out of boundary access when checking invalid long filenames. Obtained from: OpenBSD (dir.c,v 1.25) MFC after: 3 days
|
#
dad02d7d |
|
15-Apr-2019 |
Xin LI <delphij@FreeBSD.org> |
Don't cast result from malloc(). MFC after: 2 weeks
|
#
0e6eb62d |
|
05-Apr-2019 |
Xin LI <delphij@FreeBSD.org> |
Write string constant differently to improve readability. Reported by: rgrimes Reviewed by: rgrimes, emaste MFC after: 13 days Differential Revision: https://reviews.freebsd.org/D19829
|
#
d325f3c2 |
|
04-Apr-2019 |
Xin LI <delphij@FreeBSD.org> |
Fix build. MFC after: 2 weeks X-MFC-With: r345900
|
#
d78b6f1e |
|
04-Apr-2019 |
Xin LI <delphij@FreeBSD.org> |
Implement checking of `.' and `..' entries of subdirectory. Reviewed by: pfg Obtained from: Android https://android.googlesource.com/platform/external/fsck_msdos/+/b6ee08aadb580341a4d80943741b80de16a88b5d%5E%21/ MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D19824
|
#
fbf8a560 |
|
04-Apr-2019 |
Xin LI <delphij@FreeBSD.org> |
Restore lfcl when LOSTDIR's chain was corrupted and overwritten with invalid value. Obtained from: Android https://android.googlesource.com/platform/external/fsck_msdos/+/4d6d6f8a3674ad67f970e2ae908d34f64e4854cf%5E%21/ MFC after: 2 weeks
|
#
f806eb62 |
|
03-Apr-2019 |
Xin LI <delphij@FreeBSD.org> |
Assert that q can't be NULL. 'empty' is always non-NULL when DIREMPTY is set earlier. MFC after: 1 month
|
#
9ace3717 |
|
28-Mar-2019 |
Xin LI <delphij@FreeBSD.org> |
Distinguish between lseek errors and read errores. MFC after: 2 weeks
|
#
aa031ef4 |
|
30-Nov-2017 |
Xin LI <delphij@FreeBSD.org> |
Prevent OOB access on corrupted msdos directories. Submitted by: Veo Zhang <veo live com> MFC after: 2 weeks
|
#
1de7b4b8 |
|
27-Nov-2017 |
Pedro F. Giffuni <pfg@FreeBSD.org> |
various: general adoption of SPDX licensing ID tags. Mainly focus on files that use BSD 2-Clause license, however the tool I was using misidentified many licenses so this was mostly a manual - error prone - task. The Software Package Data Exchange (SPDX) group provides a specification to make it easier for automated tools to detect and summarize well known opensource licenses. We are gradually adopting the specification, noting that the tags are considered only advisory and do not, in any way, superceed or replace the license texts. No functional change intended.
|
#
091ea4a0 |
|
21-Apr-2016 |
Marcelo Araujo <araujo@FreeBSD.org> |
Use macro MAX() from sys/param.h. MFC after: 2 weeks.
|
#
f3858ada |
|
19-Apr-2016 |
Pedro F. Giffuni <pfg@FreeBSD.org> |
fsck_msdosfs: use NULL instead of zero for pointers.
|
#
33bed970 |
|
06-Apr-2016 |
Pedro F. Giffuni <pfg@FreeBSD.org> |
fsck_msdosfs(8): Optimimize memsets Obtained from: NetBSD (bin/50908) MFC after: 2 weeks
|
#
ea4cbe31 |
|
14-Jul-2014 |
Pedro F. Giffuni <pfg@FreeBSD.org> |
msdosfs: Assorted fixes from other BSDs. When truncating cluster chains fix the length of the cluster head. http://marc.info/?t=140304310700005&r=1&w=2 Avoid infinite loops in cluster chain linked lists. http://marc.info/?l=openbsd-tech&m=140275150804337&w=2 Avoid off-by-one on FAT12 filesystems. http://marc.info/?l=openbsd-tech&m=140234174104724&w=2 Obtained from: NetBSD (from OpenBSD) MFC after: 1 week
|
#
2b9fea6c |
|
14-Jul-2014 |
Pedro F. Giffuni <pfg@FreeBSD.org> |
Minor (mostly cosmetic) cleanups Several whitespace fixes convert *rootDir from external to static. Obtained from: NetBSD, OpenBSD (partial) MFC after: 3 days
|
#
6cf357bc |
|
20-Oct-2012 |
Ulrich Spörlein <uqs@FreeBSD.org> |
sbin/fsck: s/perror/perr/ to avoid shadowing - rename some other vars too - merge NetBSD license changes Obtained from: NetBSD PR: bin/139802 Reviewed by: ed
|
#
1efe3c6b |
|
04-Nov-2011 |
Ed Schouten <ed@FreeBSD.org> |
Add missing static keywords for global variables to tools in sbin/. These tools declare global variables without using the static keyword, even though their use is limited to a single C-file, or without placing an extern declaration of them in the proper header file.
|
#
a7d5f7eb |
|
19-Oct-2010 |
Jamie Gritton <jamie@FreeBSD.org> |
A new jail(8) with a configuration file, to replace the work currently done by /etc/rc.d/jail.
|
#
48790afa |
|
20-Jun-2010 |
Brian Somers <brian@FreeBSD.org> |
Fix some style(9), although there's a lot more issues here. Fix some casting errors. PR: 142384 Submitted by: giffunip at tutopia dot com Obtained from: NetBSD MFC after: 3 weeks
|
#
74d05e56 |
|
02-May-2010 |
Konstantin Belousov <kib@FreeBSD.org> |
MFC r203874: Rename fields to match better the msdosfs headers.
|
#
66db13cc |
|
02-May-2010 |
Konstantin Belousov <kib@FreeBSD.org> |
MFC r203872: Bug fixes from NetBSD.
|
#
27408439 |
|
02-May-2010 |
Konstantin Belousov <kib@FreeBSD.org> |
MFC r203871: License changes from NetBSD. Move to 2 clause license, approved by Wolfgang Solfrank.
|
#
fe0506d7 |
|
09-Mar-2010 |
Marcel Moolenaar <marcel@FreeBSD.org> |
Create the altix project branch. The altix project will add support for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting is a two-module system, consisting of a base compute module and a CPU expansion module. SGI's NUMAFlex architecture can be an excellent platform to test CPU affinity and NUMA-aware features in FreeBSD.
|
#
75fb5353 |
|
13-Feb-2010 |
Konstantin Belousov <kib@FreeBSD.org> |
Rename fields to match better the msdosfs headers. This work is still incomplete as some info doesn't really belong to the structs where it is defined. Submitted by: Pedro F. Giffuni <giffunip tutopia com> Reviewed by: bde MFC after: 2 weeks
|
#
6069db97 |
|
13-Feb-2010 |
Konstantin Belousov <kib@FreeBSD.org> |
Bug fixes from NetBSD - fix sign-compare issues. - ANSIfy a couple of functions. - Remove more duplicate #includes. - Memory leak found by Coverity on NetBSD. Submitted by: Pedro F. Giffuni <giffunip tutopia com> Reviewed by: bde MFC after: 2 weeks
|
#
a1882ff2 |
|
13-Feb-2010 |
Konstantin Belousov <kib@FreeBSD.org> |
License changes from NetBSD. Move to 2 clause license, approved by Wolfgang Solfrank. Submitted by: Pedro F. Giffuni <giffunip tutopia com> MFC after: 2 weeks
|
#
d7f03759 |
|
19-Oct-2008 |
Ulf Lilleengen <lulf@FreeBSD.org> |
- Import the HEAD csup code which is the basis for the cvsmode work.
|
#
b70cd7ee |
|
20-Mar-2002 |
Warner Losh <imp@FreeBSD.org> |
o __P removed o ansi function prototypes o unifdef -D__STDC__ o __dead2 on usage prototype o remove now-bogus main prototype
|
#
0121b42a |
|
09-Jul-2001 |
David E. O'Brien <obrien@FreeBSD.org> |
Add fsck_msdosfs. Obtained from: NetBSD
|