MFC r319605
The variable "breakout" is used like a Boolean, so actually define it as
one.

MFC r314272: call vm_lowmem hook in uma_reclaim_worker

MFC r313730: try to fix RACCT_RSS accounting

MFC r303982
Correct errors and clean up the comments on the active queue scan.

Eliminate some unnecessary blank lines.

MFC r303244, r303399
De-pluralize "queues" in the pagedaemon code.

MFC r303773
Correct a spelling error.

MFC r291408:
In vm_pageout_grow_cache(), do not re-try the inactive queue when
active queue scan initiated write, to avoid infinite loop.

MFC r290920:
Raise OOM when pagedaemon is unable to produce a free page in several
back-to-back passes.

MFC r290917:
Provide the OOM-specific vm_pageout_oom_pagecount() function which
estimates the amount of reclamaible memory which could be stolen if
the process is killed.

MFC r290915:
Do not skip a process which has inhibited thread due to the swap-out,
in the OOM selection loop.

MFC r289496:
Modify the 'unchanged' calculation bu dereferencing the marker tailq
pointers, which is known to belong to the queue.

MFC r288025
Correct a non-fatal error in vm_pageout_worker(). vm_pageout_worker()
should not assume that vm_pages_needed will remain set while it sleeps.
Other threads can clear vm_pages_needed by performing a sufficient
number of vm_page_free() calls, e.g., process termination. The effect
of this error was that vm_pageout_worker() would free and/or launder
pages when, in fact, there was no shortage of free pages.

Rewrite a nearby comment to describe all of the possible cases and not
just the most common case. The problem being that the comment made
the most common case seem like the only case.

MFC r285282
The intention of r254304 was to scan the active queue continuously.
However, I've observed the active queue scan stopping when there are
frequent free page shortages and the inactive queue is steadily refilled
by other mechanisms, such as the sequential access heuristic in vm_fault()
or madvise(2). To remedy this problem, record the time of the last active
queue scan, and always scan a number of pages proportional to the time
since the last scan, regardless of whether that last scan was a
timeout-triggered ("pass == 0") or free-page-shortage-triggered ("pass >
0") scan.

Also, on a timeout-triggered scan, allow a full scan of the active queue
when the system is short of inactive pages.

MFC r287121
Testing whether a page is dirty does not require the page lock.

MFC r284654
Avoid pmap_is_modified() on pages that can't be mapped.

MFC r286970:

Prevent ticks rollover from preventing vm_lowmem event

Currently vm_pageout_scan() uses a ticks-based scheme to rate-limit
the number of times that the vm_lowmem event will happen. However
if no events happen for long enough for ticks to roll over, this
leaves us in a long window in which vm_lowmem events will not
happen.

Replace the use of ticks with time_t to prevent rollover from ever
being an issue.

Reviewed by: ian
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D3439

MFC r282213:

Add kern.racct.enable tunable and RACCT_DISABLED config option.
The point of this is to be able to add RACCT (with RACCT_DISABLED)
to GENERIC, to avoid having to rebuild the kernel to use rctl(8).

MFC r282901:

Build GENERIC with RACCT/RCTL support by default. Note that it still
needs to be enabled by adding "kern.racct.enable=1" to /boot/loader.conf.

Note those two are MFC-ed together, because the latter one changes the
name of RACCT_DISABLED option to RACCT_DEFAULT_TO_DISABLED. Should have
committed the renaming separately...

Relnotes: yes
Sponsored by: The FreeBSD Foundation

MFC r282690:
Call uma_reclaim() from the additional pagedaemon thread to reclaim kmem
arena address space.

MFC r277646:
Avoid calling vmspace_free() while owning the process lock.

MFC r270759:
Refactor ZFS ARC reclaim logic to be more VM cooperative

MFC r270861:
Ensure that ZFS ARC free memory checks include cached pages

MFC r272483:
Refactor ZFS ARC reclaim checks and limits

Sponsored by: Multiplay

MFC r272071:
Fix ticks wrap issue of lowmem test in vm_pageout_scan

Approved by: re (kib)
Sponsored by: Multiplay

This is a direct commit to account for the renaming of 'cnt' to 'vm_cnt'
in HEAD but not stable/10.

MFC r270666
Back in the days when the kernel was single threaded, testing
"vm_paging_target() > 0" was a reasonable way of determining if the
inactive queue scan met its target. However, now that other threads
can be allocating pages while the inactive queue scan is running, it's
an unreliable method. The effect of it being unreliable is that we
can start swapping out processes when we didn't intend to.

This issue has existed since the kernel was multithreaded, but the
changes to the inactive queue target in 10.0-RELEASE have made its
effects visible.

This change introduces a more direct method for determining if the
inactive queue scan met its target that is not affected by the actions
of other threads.

MFC r265418
Prior to r254304, a separate function, vm_pageout_page_stats(), was used
to periodically update the reference status of the active pages. This
function was called, instead of vm_pageout_scan(), when memory was not
scarce. The objective was to provide up to date reference status for
active pages in case memory did become scarce and active pages needed to
be deactivated.

The active page queue scan performed by vm_pageout_page_stats() was
virtually identical to that performed by vm_pageout_scan(), and so r254304
eliminated vm_pageout_page_stats(). Instead, vm_pageout_scan() is
called with the parameter "pass" set to zero. The intention was that when
pass is zero, vm_pageout_scan() would only scan the active queue.
However, the variable page_shortage can still be greater than zero when
memory is not scarce and vm_pageout_scan() is called with pass equal to
zero. Consequently, the inactive queue may be scanned and dirty pages
laundered even though that was not intended by r254304. This revision
fixes that.

MFC r260567
Correctly update the count of stuck pages, "addl_page_shortage", in
vm_pageout_scan(). There were missing increments in two less common
cases.

Don't conflate the count of stuck pages and the pageout deficit provided
by vm_page_alloc{,_contig}().

Handle held pages consistently in the inactive queue scan. In the more
common case, we did not move the page to the tail of the queue. Whereas,
in the less common case, we did. There's no particular reason to move
the page in the less common case, so remove it.

Perform the calculation of the page shortage for the active queue scan a
little earlier, before the active queue lock is acquired. The correctness
of this calculation doesn't depend on the active queue lock being held.

Eliminate a redundant variable, "pcount". Use the more descriptive
variable, "maxscan", in its place.

Apply a few nearby style fixes, e.g., eliminate stray whitespace and
excess parentheses.

MFC r259908:
For ia64, use pmap_remove_pages() and not pmap_remove().

MFC r259893:

In sys/vm/vm_pageout.c, since vm_pageout_worker() takes a void * as
argument, cast the incoming 0 argument to void *, to silence a warning
from clang 3.4 ("expression which evaluates to zero treated as a null
pointer constant of type 'void *' [-Wnon-literal-null-conversion]").

MFC r257899:
If filesystem declares that it supports shared locking for writes, use
shared vnode lock for VOP_PUTPAGES() as well.

MFC r288025
Correct a non-fatal error in vm_pageout_worker(). vm_pageout_worker()
should not assume that vm_pages_needed will remain set while it sleeps.
Other threads can clear vm_pages_needed by performing a sufficient
number of vm_page_free() calls, e.g., process termination. The effect
of this error was that vm_pageout_worker() would free and/or launder
pages when, in fact, there was no shortage of free pages.

Rewrite a nearby comment to describe all of the possible cases and not
just the most common case. The problem being that the comment made
the most common case seem like the only case.

MFC r285282
The intention of r254304 was to scan the active queue continuously.
However, I've observed the active queue scan stopping when there are
frequent free page shortages and the inactive queue is steadily refilled
by other mechanisms, such as the sequential access heuristic in vm_fault()
or madvise(2). To remedy this problem, record the time of the last active
queue scan, and always scan a number of pages proportional to the time
since the last scan, regardless of whether that last scan was a
timeout-triggered ("pass == 0") or free-page-shortage-triggered ("pass >
0") scan.

Also, on a timeout-triggered scan, allow a full scan of the active queue
when the system is short of inactive pages.

MFC r287121
Testing whether a page is dirty does not require the page lock.

MFC r284654
Avoid pmap_is_modified() on pages that can't be mapped.

MFC r286970:

Prevent ticks rollover from preventing vm_lowmem event

Currently vm_pageout_scan() uses a ticks-based scheme to rate-limit
the number of times that the vm_lowmem event will happen. However
if no events happen for long enough for ticks to roll over, this
leaves us in a long window in which vm_lowmem events will not
happen.

Replace the use of ticks with time_t to prevent rollover from ever
being an issue.

Reviewed by: ian
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D3439

MFC r282213:

Add kern.racct.enable tunable and RACCT_DISABLED config option.
The point of this is to be able to add RACCT (with RACCT_DISABLED)
to GENERIC, to avoid having to rebuild the kernel to use rctl(8).

MFC r282901:

Build GENERIC with RACCT/RCTL support by default. Note that it still
needs to be enabled by adding "kern.racct.enable=1" to /boot/loader.conf.

Note those two are MFC-ed together, because the latter one changes the
name of RACCT_DISABLED option to RACCT_DEFAULT_TO_DISABLED. Should have
committed the renaming separately...

Relnotes: yes
Sponsored by: The FreeBSD Foundation

MFC r282690:
Call uma_reclaim() from the additional pagedaemon thread to reclaim kmem
arena address space.

MFC r277646:
Avoid calling vmspace_free() while owning the process lock.

MFC r270759:
Refactor ZFS ARC reclaim logic to be more VM cooperative

MFC r270861:
Ensure that ZFS ARC free memory checks include cached pages

MFC r272483:
Refactor ZFS ARC reclaim checks and limits

Sponsored by: Multiplay

MFC r272071:
Fix ticks wrap issue of lowmem test in vm_pageout_scan

Approved by: re (kib)
Sponsored by: Multiplay

This is a direct commit to account for the renaming of 'cnt' to 'vm_cnt'
in HEAD but not stable/10.

MFC r270666
Back in the days when the kernel was single threaded, testing
"vm_paging_target() > 0" was a reasonable way of determining if the
inactive queue scan met its target. However, now that other threads
can be allocating pages while the inactive queue scan is running, it's
an unreliable method. The effect of it being unreliable is that we
can start swapping out processes when we didn't intend to.

This issue has existed since the kernel was multithreaded, but the
changes to the inactive queue target in 10.0-RELEASE have made its
effects visible.

This change introduces a more direct method for determining if the
inactive queue scan met its target that is not affected by the actions
of other threads.

MFC r265418
Prior to r254304, a separate function, vm_pageout_page_stats(), was used
to periodically update the reference status of the active pages. This
function was called, instead of vm_pageout_scan(), when memory was not
scarce. The objective was to provide up to date reference status for
active pages in case memory did become scarce and active pages needed to
be deactivated.

The active page queue scan performed by vm_pageout_page_stats() was
virtually identical to that performed by vm_pageout_scan(), and so r254304
eliminated vm_pageout_page_stats(). Instead, vm_pageout_scan() is
called with the parameter "pass" set to zero. The intention was that when
pass is zero, vm_pageout_scan() would only scan the active queue.
However, the variable page_shortage can still be greater than zero when
memory is not scarce and vm_pageout_scan() is called with pass equal to
zero. Consequently, the inactive queue may be scanned and dirty pages
laundered even though that was not intended by r254304. This revision
fixes that.

MFC r260567
Correctly update the count of stuck pages, "addl_page_shortage", in
vm_pageout_scan(). There were missing increments in two less common
cases.

Don't conflate the count of stuck pages and the pageout deficit provided
by vm_page_alloc{,_contig}().

Handle held pages consistently in the inactive queue scan. In the more
common case, we did not move the page to the tail of the queue. Whereas,
in the less common case, we did. There's no particular reason to move
the page in the less common case, so remove it.

Perform the calculation of the page shortage for the active queue scan a
little earlier, before the active queue lock is acquired. The correctness
of this calculation doesn't depend on the active queue lock being held.

Eliminate a redundant variable, "pcount". Use the more descriptive
variable, "maxscan", in its place.

Apply a few nearby style fixes, e.g., eliminate stray whitespace and
excess parentheses.

MFC r259908:
For ia64, use pmap_remove_pages() and not pmap_remove().

MFC r259893:

In sys/vm/vm_pageout.c, since vm_pageout_worker() takes a void * as
argument, cast the incoming 0 argument to void *, to silence a warning
from clang 3.4 ("expression which evaluates to zero treated as a null
pointer constant of type 'void *' [-Wnon-literal-null-conversion]").

MFC r257899:
If filesystem declares that it supports shared locking for writes, use
shared vnode lock for VOP_PUTPAGES() as well.