#
323745 |
|
19-Sep-2017 |
avg |
MFC r320352: zfs: port vdev_file part of illumos change 3306
3306 zdb should be able to issue reads in parallel illumos/illumos-gate/31d7e8fa33fae995f558673adb22641b5aa8b6e1 https://www.illumos.org/issues/3306
The upstream change was made before we started to import upstream commits individually. It was imported into the illumos vendor area as r242733. That commit was MFV-ed in r260138, but as the commit message says vdev_file.c was left intact.
This commit actually implements the parallel I/O for vdev_file using a taskqueue with multiple thread. This implementation does not depend on the illumos or FreeBSD bio interface at all, but uses zio_t to pass around all the relevent data. So, the code looks a bit different from the upstream.
This commit also incorporates ZoL commit zfsonlinux/zfs/bc25c9325b0e5ced897b9820dad239539d561ec9 that fixed https://github.com/zfsonlinux/zfs/issues/2270 We need to use a dedicated taskqueue for exactly the same reason as ZoL as we do not implement TASKQ_DYNAMIC.
|
#
318786 |
|
24-May-2017 |
avg |
MFC r316854: rename vfs.zfs.debug_flags to vfs.zfs.debugflags
Since this is a stable branch vfs.zfs.debug_flags sysctl is also kept. The corresponing tunable could never work.
|
#
314668 |
|
04-Mar-2017 |
avg |
MFC r314273: zfs: call spa_deadman on a taskqueue thread
|
#
314667 |
|
04-Mar-2017 |
avg |
MFC r283291: don't use CALLOUT_MPSAFE with callout_init()
The main purpose of this MFC is to reduce conflicts for other merges. Parts of the original change have already "trickled down" via individual MFCs.
|
#
307279 |
|
14-Oct-2016 |
mav |
MFC r305331: MFV r304155: 7090 zfs should improve allocation order and throttle allocations
illumos/illumos-gate@0f7643c7376dd69a08acbfc9d1d7d548b10c846a https://github.com/illumos/illumos-gate/commit/0f7643c7376dd69a08acbfc9d1d7d548b 10c846a
https://www.illumos.org/issues/7090 When write I/Os are issued, they are issued in block order but the ZIO pipelin e will drive them asynchronously through the allocation stage which can result i n blocks being allocated out-of-order. It would be nice to preserve as much of the logical order as possible. In addition, the allocations are equally scattered across all top-level VDEVs but not all top-level VDEVs are created equally. The pipeline should be able t o detect devices that are more capable of handling allocations and should allocate more blocks to those devices. This allows for dynamic allocation distribution when devices are imbalanced as fuller devices will tend to be slower than empty devices. The change includes a new pool-wide allocation queue which would throttle and order allocations in the ZIO pipeline. The queue would be ordered by issued time and offset and would provide an initial amount of allocation of work to each top-level vdev. The allocation logic utilizes a reservation system to reserve allocations that will be performed by the allocator. Once an allocatio n is successfully completed it's scheduled on a given top-level vdev. Each top- level vdev maintains a maximum number of allocations that it can handle (mg_alloc_queue_depth). The pool-wide reserved allocations (top-levels * mg_alloc_queue_depth) are distributed across the top-level vdevs metaslab groups and round robin across all eligible metaslab groups to distribute the work. As top-levels complete their work, they receive additional work from the pool-wide allocation queue until the allocation queue is emptied.
Reviewed by: Adam Leventhal <ahl@delphix.com> Reviewed by: Alex Reece <alex@delphix.com> Reviewed by: Christopher Siden <christopher.siden@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Paul Dagnelie <paul.dagnelie@delphix.com> Reviewed by: Prakash Surya <prakash.surya@delphix.com> Reviewed by: Sebastien Roy <sebastien.roy@delphix.com> Approved by: Robert Mustacchi <rm@joyent.com> Author: George Wilson <george.wilson@delphix.com>
|
#
307126 |
|
11-Oct-2016 |
mav |
MFC r305222: MFV r302993: 7104 increase indirect block size
illumos/illumos-gate@4b5c8e93cab28d3c65ba9d407fd8f46e3be1db1c https://github.com/illumos/illumos-gate/commit/4b5c8e93cab28d3c65ba9d407fd8f46e3 be1db1c
https://www.illumos.org/issues/7104 The current default indirect block size is 16KB. We can improve performance by increasing it to 128KB. This is especially helpful for any workload that needs to read most of the metadata, e.g. scrub/resilver, file deletion, filesystem deletion, and zfs send. We also need to fix a few space estimation errors to make the tests pass.
Reviewed by: George Wilson <george.wilson@delphix.com> Reviewed by: Paul Dagnelie <pcd@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Robert Mustacchi <rm@joyent.com> Author: Matthew Ahrens <mahrens@delphix.com>
|
#
297112 |
|
20-Mar-2016 |
mav |
MFC r296519: MFV r296518: 5027 zfs large block support (add copyright)
Author: Matthew Ahrens <matt@mahrens.org>
illumos/illumos-gate@c3d26abc9ee97b4f60233556aadeb57e0bd30bb9
|
#
297092 |
|
20-Mar-2016 |
mav |
MFC r294794: MFV r294793: 6367 spa_config_tryenter incorrectly handles the multiple-lock case
Reviewed by: Alek Pinchuk <alek.pinchuk@nexenta.com> Reviewed by: Josef 'Jeff' Sipek <josef.sipek@nexenta.com> Reviewed by: Prashanth Sreenivasa <prashksp@gmail.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Reviewed by: Steven Hartland <steven.hartland@multiplay.co.uk> Approved by: Matthew Ahrens <mahrens@delphix.com>
illumos/illumos-gate@e495b6e6735b803e422025a630352ef9bba788c5
|
#
297075 |
|
20-Mar-2016 |
mav |
MFC r269222: Reschedule the 'deadman' callout after handling, this makes our code behave more like it is on Solaris.
Differential Revision: https://phabric.freebsd.org/D457
|
#
297074 |
|
20-Mar-2016 |
mav |
MFC r271788 (by will): Enable ZFS debug flags to be modified via vfs.zfs.debug_flags.
This is primarily only of interest to ZFS developers, but it makes it easier to get additional debugging.
MFSpectraBSD: 517074 on 2011/12/15 (by will), 662343 on 2013/03/20 (by gibbs)
|
#
290757 |
|
13-Nov-2015 |
mav |
MFC r289422: 4185 add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R
Reviewed by: George Wilson <george.wilson@delphix.com> Reviewed by: Prakash Surya <prakash.surya@delphix.com> Reviewed by: Saso Kiselkov <saso.kiselkov@nexenta.com> Reviewed by: Richard Lowe <richlowe@richlowe.net> Approved by: Garrett D'Amore <garrett@damore.org> Author: Matthew Ahrens <mahrens@delphix.com>
illumos/illumos-gate@45818ee124adeaaf947698996b4f4c722afc6d1f
This is only a partial merge of respective ZFS infrastructure changes. At this moment FreeBSD kernel has no those crypto algorithms, so the parts of the code to enable them are commented out. When they are implemented, it will be trivial to plug them in.
|
#
290713 |
|
12-Nov-2015 |
smh |
MFC r290401 & r290466
Provide information about bad DVA
Sponsored by: Multiplay
|
#
290712 |
|
12-Nov-2015 |
smh |
MFC r290399:
Allow zfs_recover to be changed at runtime
Sponsored by: Multiplay
|
#
288549 |
|
03-Oct-2015 |
mav |
MFC r286575: 5056 ZFS deadlock on db_mtx and dn_holds
Reviewed by: Will Andrews <willa@spectralogic.com> Reviewed by: Matt Ahrens <mahrens@delphix.com> Reviewed by: George Wilson <george.wilson@delphix.com> Approved by: Dan McDonald <danmcd@omniti.com> Author: Justin Gibbs <justing@spectralogic.com>
illumos/illumos-gate@bc9014e6a81272073b9854d9f65dd59e18d18c35
|
#
285001 |
|
01-Jul-2015 |
avg |
MFC r284304: MFV r284030: 5818 zfs {ref}compressratio is incorrect with 4k sector size
Note: no MFC to stable/9 because r268075 (vendor r267565) has not been MFC-ed.
|
#
276081 |
|
22-Dec-2014 |
delphij |
MFC r274337,r274673,274681,r275515:
ZFS large block support. The default recordsize remains at 128KB.
A new tunable/sysctl variable, vfs.zfs.max_recordsize is added to allow adjusting the permitted maximum record size, or zfs_max_recordsize, with a default of 1MB. ZFS will not allow setting recordsize greater than zfs_max_recordsize as a safety belt, because larger recordsize means greater read and write latency and more memory usage.
Please note that booting from datasets that have recordsize greater than 128KB is not supported (but it's Okay to enable the feature on the pool).
Limited safety belt is provided for mounted root filesystem but use caution when using a larger value.
Illumos issue: 5027 zfs large block support
|
#
275490 |
|
04-Dec-2014 |
delphij |
MFC r274674:
Add a tunable for spa_slop_shift which controls how much space we would reserve by default. Tuning is not recommended.
Relnotes: yes
|
#
269418 |
|
02-Aug-2014 |
delphij |
MFC r268859: MFV r268851:
When a sync task is waiting for a txg to complete, we should hurry it along by increasing the number of outstanding async writes (i.e. make vdev_queue_max_async_writes() return a larger number).
Illumos issue: 4753 increase number of outstanding async writes when sync task is waiting
|
#
269416 |
|
02-Aug-2014 |
delphij |
MFC r268855: MFV r268848:
Instead of asserting all zio's be properly aligned, only assert on the logical ones.
Cap uberblocks at 8k, otherwise with ashift=17, there would be only one uberblock.
This fixes a problem that zdb would trip assert on pools with ashift >= 0xe (8k).
While there, also change the code so it only attempt to condense space map unless the uncondensed size consumes greater than zfs_metaslab_condense_block_threshold blocks.
Illumos issue: 4958 zdb trips assert on pools with ashift >= 0xe
|
#
269006 |
|
22-Jul-2014 |
delphij |
MFC r268473: MFV r268455:
Use reserved space for ZFS administrative commands.
|
#
268650 |
|
15-Jul-2014 |
delphij |
MFC r268079: MFV r267566:
4390 i/o errors when deleting filesystem/zvol can lead to space map corruption
|
#
268649 |
|
15-Jul-2014 |
delphij |
MFC r268075: MFV r267565:
4757 ZFS embedded-data block pointers ("zero block compression") 4913 zfs release should not be subject to space checks
|
#
263407 |
|
19-Mar-2014 |
delphij |
MFC r260183: MFV r260154 + 260182:
4369 implement zfs bookmarks 4368 zfs send filesystems from readonly pools
Illumos/illumos-gate@78f171005391b928aaf1642b3206c534ed644332
|
#
263397 |
|
19-Mar-2014 |
delphij |
MFC r260150: MFV r259170:
4370 avoid transmitting holes during zfs send
4371 DMU code clean up
illumos/illumos-gate@43466aae47bfcd2ad9bf501faec8e75c08095e4f
NOTE: Make sure the boot code is updated if a zpool upgrade is done on boot zpool.
|
#
263390 |
|
19-Mar-2014 |
delphij |
MFC r259813 + r259813: MFV r258374:
4171 clean up spa_feature_*() interfaces
4172 implement extensible_dataset feature for use by other zpool features
illumos/illumos-gate@2acef22db7808606888f8f92715629ff3ba555b9
|
#
262179 |
|
18-Feb-2014 |
avg |
MFC r259052: Expose spa_asize_inflation
|
#
262120 |
|
17-Feb-2014 |
avg |
MFC r260185: MFV r260155: 4391 panic system rather than corrupting pool if we hit bug 4390
|
#
262093 |
|
17-Feb-2014 |
avg |
MFC r258717: MFV r258371,r258372: 4101 metaslab_debug should allow for fine-grained control
|
#
260763 |
|
16-Jan-2014 |
avg |
MFC r258632,258704: MFV r255255: 4045 zfs write throttle & i/o scheduler performance work
Sponsored by: HybridCluster [merge]
|
#
260731 |
|
16-Jan-2014 |
avg |
MFC r258638,258642: expose zfs_flags as debug.zfs_flags r/w tunable and sysctl
Sponsored by: HybridCluster
|
#
260337 |
|
05-Jan-2014 |
mav |
MFC r258137: Introduce allocation cache to store LZ4 compression contexts without kicking VM subsystem twice for every written record.
Tests on 24-core system show double reduction of CPU time spent on copying single large well-compressed file.
This patch is not really needed on illumos (while not harm either) since their memory allocator by default uses caching for all requests up to 128K.
|
#
288549 |
|
03-Oct-2015 |
mav |
MFC r286575: 5056 ZFS deadlock on db_mtx and dn_holds
Reviewed by: Will Andrews <willa@spectralogic.com> Reviewed by: Matt Ahrens <mahrens@delphix.com> Reviewed by: George Wilson <george.wilson@delphix.com> Approved by: Dan McDonald <danmcd@omniti.com> Author: Justin Gibbs <justing@spectralogic.com>
illumos/illumos-gate@bc9014e6a81272073b9854d9f65dd59e18d18c35
|
#
285001 |
|
01-Jul-2015 |
avg |
MFC r284304: MFV r284030: 5818 zfs {ref}compressratio is incorrect with 4k sector size
Note: no MFC to stable/9 because r268075 (vendor r267565) has not been MFC-ed.
|
#
276081 |
|
22-Dec-2014 |
delphij |
MFC r274337,r274673,274681,r275515:
ZFS large block support. The default recordsize remains at 128KB.
A new tunable/sysctl variable, vfs.zfs.max_recordsize is added to allow adjusting the permitted maximum record size, or zfs_max_recordsize, with a default of 1MB. ZFS will not allow setting recordsize greater than zfs_max_recordsize as a safety belt, because larger recordsize means greater read and write latency and more memory usage.
Please note that booting from datasets that have recordsize greater than 128KB is not supported (but it's Okay to enable the feature on the pool).
Limited safety belt is provided for mounted root filesystem but use caution when using a larger value.
Illumos issue: 5027 zfs large block support
|
#
275490 |
|
04-Dec-2014 |
delphij |
MFC r274674:
Add a tunable for spa_slop_shift which controls how much space we would reserve by default. Tuning is not recommended.
Relnotes: yes
|
#
269418 |
|
02-Aug-2014 |
delphij |
MFC r268859: MFV r268851:
When a sync task is waiting for a txg to complete, we should hurry it along by increasing the number of outstanding async writes (i.e. make vdev_queue_max_async_writes() return a larger number).
Illumos issue: 4753 increase number of outstanding async writes when sync task is waiting
|
#
269416 |
|
02-Aug-2014 |
delphij |
MFC r268855: MFV r268848:
Instead of asserting all zio's be properly aligned, only assert on the logical ones.
Cap uberblocks at 8k, otherwise with ashift=17, there would be only one uberblock.
This fixes a problem that zdb would trip assert on pools with ashift >= 0xe (8k).
While there, also change the code so it only attempt to condense space map unless the uncondensed size consumes greater than zfs_metaslab_condense_block_threshold blocks.
Illumos issue: 4958 zdb trips assert on pools with ashift >= 0xe
|
#
269006 |
|
22-Jul-2014 |
delphij |
MFC r268473: MFV r268455:
Use reserved space for ZFS administrative commands.
|
#
268650 |
|
15-Jul-2014 |
delphij |
MFC r268079: MFV r267566:
4390 i/o errors when deleting filesystem/zvol can lead to space map corruption
|
#
268649 |
|
15-Jul-2014 |
delphij |
MFC r268075: MFV r267565:
4757 ZFS embedded-data block pointers ("zero block compression") 4913 zfs release should not be subject to space checks
|
#
263407 |
|
19-Mar-2014 |
delphij |
MFC r260183: MFV r260154 + 260182:
4369 implement zfs bookmarks 4368 zfs send filesystems from readonly pools
Illumos/illumos-gate@78f171005391b928aaf1642b3206c534ed644332
|
#
263397 |
|
19-Mar-2014 |
delphij |
MFC r260150: MFV r259170:
4370 avoid transmitting holes during zfs send
4371 DMU code clean up
illumos/illumos-gate@43466aae47bfcd2ad9bf501faec8e75c08095e4f
NOTE: Make sure the boot code is updated if a zpool upgrade is done on boot zpool.
|
#
263390 |
|
19-Mar-2014 |
delphij |
MFC r259813 + r259813: MFV r258374:
4171 clean up spa_feature_*() interfaces
4172 implement extensible_dataset feature for use by other zpool features
illumos/illumos-gate@2acef22db7808606888f8f92715629ff3ba555b9
|
#
262179 |
|
18-Feb-2014 |
avg |
MFC r259052: Expose spa_asize_inflation
|
#
262120 |
|
17-Feb-2014 |
avg |
MFC r260185: MFV r260155: 4391 panic system rather than corrupting pool if we hit bug 4390
|
#
262093 |
|
17-Feb-2014 |
avg |
MFC r258717: MFV r258371,r258372: 4101 metaslab_debug should allow for fine-grained control
|
#
260763 |
|
16-Jan-2014 |
avg |
MFC r258632,258704: MFV r255255: 4045 zfs write throttle & i/o scheduler performance work
Sponsored by: HybridCluster [merge]
|
#
260731 |
|
16-Jan-2014 |
avg |
MFC r258638,258642: expose zfs_flags as debug.zfs_flags r/w tunable and sysctl
Sponsored by: HybridCluster
|
#
260337 |
|
05-Jan-2014 |
mav |
MFC r258137: Introduce allocation cache to store LZ4 compression contexts without kicking VM subsystem twice for every written record.
Tests on 24-core system show double reduction of CPU time spent on copying single large well-compressed file.
This patch is not really needed on illumos (while not harm either) since their memory allocator by default uses caching for all requests up to 128K.
|