#
259065 |
|
07-Dec-2013 |
gjb |
- Copy stable/10 (r259064) to releng/10.0 as part of the 10.0-RELEASE cycle. - Update __FreeBSD_version [1] - Set branch name to -RC1
[1] 10.0-CURRENT __FreeBSD_version value ended at '55', so start releng/10.0 at '100' so the branch is started with a value ending in zero.
Approved by: re (implicit) Sponsored by: The FreeBSD Foundation |
#
256281 |
|
10-Oct-2013 |
gjb |
Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.
Approved by: re (implicit) Sponsored by: The FreeBSD Foundation
|
#
255724 |
|
20-Sep-2013 |
alc |
The pmap function pmap_clear_reference() is no longer used. Remove it.
pmap_clear_reference() has had exactly one caller in the kernel for several years, more precisely, since FreeBSD 8. Now, that call no longer exists.
Approved by: re (kib) Sponsored by: EMC / Isilon Storage Division
|
#
254667 |
|
22-Aug-2013 |
kib |
Revert r254501. Instead, reuse the type stability of the struct pmap which is the part of struct vmspace, allocated from UMA_ZONE_NOFREE zone. Initialize the pmap lock in the vmspace zone init function, and remove pmap lock initialization and destruction from pmap_pinit() and pmap_release().
Suggested and reviewed by: alc (previous version) Tested by: pho Sponsored by: The FreeBSD Foundation
|
#
254138 |
|
09-Aug-2013 |
attilio |
The soft and hard busy mechanism rely on the vm object lock to work. Unify the 2 concept into a real, minimal, sxlock where the shared acquisition represent the soft busy and the exclusive acquisition represent the hard busy. The old VPO_WANTED mechanism becames the hard-path for this new lock and it becomes per-page rather than per-object. The vm_object lock becames an interlock for this functionality: it can be held in both read or write mode. However, if the vm_object lock is held in read mode while acquiring or releasing the busy state, the thread owner cannot make any assumption on the busy state unless it is also busying it.
Also: - Add a new flag to directly shared busy pages while vm_page_alloc and vm_page_grab are being executed. This will be very helpful once these functions happen under a read object lock. - Move the swapping sleep into its own per-object flag
The KPI is heavilly changed this is why the version is bumped. It is very likely that some VM ports users will need to change their own code.
Sponsored by: EMC / Isilon storage division Discussed with: alc Reviewed by: jeff, kib Tested by: gavin, bapt (older version) Tested by: pho, scottl
|
#
254025 |
|
07-Aug-2013 |
jeff |
Replace kernel virtual address space allocation with vmem. This provides transparent layering and better fragmentation.
- Normalize functions that allocate memory to use kmem_* - Those that allocate address space are named kva_* - Those that operate on maps are named kmap_* - Implement recursive allocation handling for kmem_arena in vmem.
Reviewed by: alc Tested by: pho Sponsored by: EMC / Isilon Storage Division
|
#
253978 |
|
06-Aug-2013 |
jhibbits |
Remove an unnecessary panic. The PVO's PTE entry and the PTEG's PTE entry may not match, if the PVO's PTE is invalid.
|
#
253976 |
|
05-Aug-2013 |
jhibbits |
Evict pages from the PTEG when it's full and trying to insert a new PTE, rather than panicking.
Reviewed by: nwhitehorn MFC after: 3 weeks
|
#
250884 |
|
21-May-2013 |
attilio |
o Relax locking assertions for vm_page_find_least() o Relax locking assertions for pmap_enter_object() and add them also to architectures that currently don't have any o Introduce VM_OBJECT_LOCK_DOWNGRADE() which is basically a downgrade operation on the per-object rwlock o Use all the mechanisms above to make vm_map_pmap_enter() to work mostl of the times only with readlocks.
Sponsored by: EMC / Isilon storage division Reviewed by: alc
|
#
250747 |
|
17-May-2013 |
alc |
Relax the object locking assertion in pmap_enter_locked().
Reviewed by: attilio Sponsored by: EMC / Isilon Storage Division
|
#
249918 |
|
26-Apr-2013 |
jhibbits |
Remove a comment that shouldn't have gone in.
X-MFC-with: r249864
|
#
249864 |
|
24-Apr-2013 |
jhibbits |
Introduce kernel coredumps to ppc32 AIM. Leeched from the booke code.
MFC after: 2 weeks
|
#
248280 |
|
14-Mar-2013 |
kib |
Add pmap function pmap_copy_pages(), which copies the content of the pages around, taking array of vm_page_t both for source and destination. Starting offsets and total transfer size are specified.
The function implements optimal algorithm for copying using the platform-specific optimizations. For instance, on the architectures were the direct map is available, no transient mappings are created, for i386 the per-cpu ephemeral page frame is used. The code was typically borrowed from the pmap_copy_page() for the same architecture.
Only i386/amd64, powerpc aim and arm/arm-v6 implementations were tested at the time of commit. High-level code, not committed yet to the tree, ensures that the use of the function is only allowed after explicit enablement.
For sparc64, the existing code has known issues and a stab is added instead, to allow the kernel linking.
Sponsored by: The FreeBSD Foundation Tested by: pho (i386, amd64), scottl (amd64), ian (arm and arm-v6) MFC after: 2 weeks
|
#
248084 |
|
09-Mar-2013 |
attilio |
Switch the vm_object mutex to be a rwlock. This will enable in the future further optimizations where the vm_object lock will be held in read mode most of the time the page cache resident pool of pages are accessed for reading purposes.
The change is mostly mechanical but few notes are reported: * The KPI changes as follow: - VM_OBJECT_LOCK() -> VM_OBJECT_WLOCK() - VM_OBJECT_TRYLOCK() -> VM_OBJECT_TRYWLOCK() - VM_OBJECT_UNLOCK() -> VM_OBJECT_WUNLOCK() - VM_OBJECT_LOCK_ASSERT(MA_OWNED) -> VM_OBJECT_ASSERT_WLOCKED() (in order to avoid visibility of implementation details) - The read-mode operations are added: VM_OBJECT_RLOCK(), VM_OBJECT_TRYRLOCK(), VM_OBJECT_RUNLOCK(), VM_OBJECT_ASSERT_RLOCKED(), VM_OBJECT_ASSERT_LOCKED() * The vm/vm_pager.h namespace pollution avoidance (forcing requiring sys/mutex.h in consumers directly to cater its inlining functions using VM_OBJECT_LOCK()) imposes that all the vm/vm_pager.h consumers now must include also sys/rwlock.h. * zfs requires a quite convoluted fix to include FreeBSD rwlocks into the compat layer because the name clash between FreeBSD and solaris versions must be avoided. At this purpose zfs redefines the vm_object locking functions directly, isolating the FreeBSD components in specific compat stubs.
The KPI results heavilly broken by this commit. Thirdy part ports must be updated accordingly (I can think off-hand of VirtualBox, for example).
Sponsored by: EMC / Isilon storage division Reviewed by: jeff Reviewed by: pjd (ZFS specific review) Discussed with: alc Tested by: pho
|
#
247400 |
|
27-Feb-2013 |
attilio |
Merge from vmobj-rwlock: VM_OBJECT_LOCKED() macro is only used to implement a custom version of lock assertions right now (which likely spread out thanks to copy and paste). Remove it and implement actual assertions.
Sponsored by: EMC / Isilon storage division Reviewed by: alc Tested by: pho
|
#
247297 |
|
25-Feb-2013 |
attilio |
Merge from vmobj-rwlock branch: Remove unused inclusion of vm/vm_pager.h and vm/vnode_pager.h.
Sponsored by: EMC / Isilon storage division Tested by: pho Reviewed by: alc
|
#
243370 |
|
21-Nov-2012 |
adrian |
Setup BAT0 and BAT1 on the Wii.
This is the missing piece for FreeBSD/Wii, but there's still a lot of work ahead. We have to reset the MMU in locore before continuing the boot process because we don't know how the boot loaders might have setup the BATs. We also disable the PCI BAT because there's no PCI bus on the Wii.
Thanks to Nathan Whitehorn and Peter Grenhan for their help.
Submitted by: Margarida Gouveia
|
#
242534 |
|
03-Nov-2012 |
attilio |
Rework the known rwlock to benefit about staying on their own cache line in order to avoid manual frobbing but using struct rwlock_padalign.
Reviewed by: alc, jimharris
|
#
241020 |
|
28-Sep-2012 |
alc |
Eliminate a stale comment. It describes another use case for the pmap in Mach that doesn't exist in FreeBSD.
|
#
238357 |
|
10-Jul-2012 |
alc |
Avoid recursion on the pvh global lock in the aim oea pmap.
Correct the return type of the pmap_ts_referenced() implementations.
Reported by: jhibbits [1] Tested by: andreast
|
#
238159 |
|
06-Jul-2012 |
alc |
Replace all uses of the vm page queues lock by a r/w lock that is private to this pmap.
Tested by: andreast, jhibbits
|
#
235936 |
|
24-May-2012 |
raj |
Fix physical address type to vm_paddr_t.
|
#
235689 |
|
20-May-2012 |
nwhitehorn |
Replace the list of PVOs owned by each PMAP with an RB tree. This simplifies range operations like pmap_remove() and pmap_protect() as well as allowing simple operations like pmap_extract() not to involve any global state. This substantially reduces lock coverages for the global table lock and improves concurrency.
|
#
234156 |
|
11-Apr-2012 |
nwhitehorn |
We don't need kcopy() in any of the remaining places it is used, so remove it.
MFC after: 2 weeks
|
#
234149 |
|
11-Apr-2012 |
nwhitehorn |
Fix error in r233949. Synchronizing icaches on uncacheable pages turns out not to be a good idea, and of course the PV entry list for a page is never empty after the page has been mapped.
|
#
233949 |
|
06-Apr-2012 |
nwhitehorn |
Reduce the frequency that the PowerPC/AIM pmaps invalidate instruction caches, by invalidating kernel icaches only when needed and not flushing user caches for shared pages.
Suggested by: kib MFC after: 2 weeks
|
#
232980 |
|
14-Mar-2012 |
nwhitehorn |
Use LIST_FOREACH_SAFE() instead of LIST_FOREACH() in pmap_remove(), since the point of this loop is to remove elements. This worked by accident before.
MFC after: 2 days
|
#
228609 |
|
16-Dec-2011 |
nwhitehorn |
Allow this to work on embedded systems without Open Firmware by making lack of a /chosen non-fatal, and manually removing memory in use by the kernel from the physical memory map.
Submitted by: rpaulo
|
#
228412 |
|
11-Dec-2011 |
nwhitehorn |
Keep track of PVO entries in each pmap, which allows much faster pmap_remove() for large sparse requests. This can prevent pmap_remove() operations on 64-bit process destruction or swapout that would take several hundred times the lifetime of the universe to complete. This behavior is largely indistinguishable from a hang.
|
#
227627 |
|
17-Nov-2011 |
nwhitehorn |
Add an extra invariant here which was useful on 64-bit CPUs.
|
#
225418 |
|
06-Sep-2011 |
kib |
Split the vm_page flags PG_WRITEABLE and PG_REFERENCED into atomic flags field. Updates to the atomic flags are performed using the atomic ops on the containing word, do not require any vm lock to be held, and are non-blocking. The vm_page_aflag_set(9) and vm_page_aflag_clear(9) functions are provided to modify afalgs.
Document the changes to flags field to only require the page lock.
Introduce vm_page_reference(9) function to provide a stable KPI and KBI for filesystems like tmpfs and zfs which need to mark a page as referenced.
Reviewed by: alc, attilio Tested by: marius, flo (sparc64); andreast (powerpc, powerpc64) Approved by: re (bz)
|
#
224746 |
|
09-Aug-2011 |
kib |
- Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flag to VPO_UNMANAGED (and also making the flag protected by the vm object lock, instead of vm page queue lock). - Mark the fake pages with both PG_FICTITIOUS (as it is now) and VPO_UNMANAGED. As a consequence, pmap code now can use use just VPO_UNMANAGED to decide whether the page is unmanaged.
Reviewed by: alc Tested by: pho (x86, previous version), marius (sparc64), marcel (arm, ia64, powerpc), ray (mips) Sponsored by: The FreeBSD Foundation Approved by: re (bz)
|
#
223758 |
|
04-Jul-2011 |
attilio |
With retirement of cpumask_t and usage of cpuset_t for representing a mask of CPUs, pc_other_cpus and pc_cpumask become highly inefficient.
Remove them and replace their usage with custom pc_cpuid magic (as, atm, pc_cpumask can be easilly represented by (1 << pc_cpuid) and pc_other_cpus by (all_cpus & ~(1 << pc_cpuid))).
This change is not targeted for MFC because of struct pcpu members removal and dependency by cpumask_t retirement.
MD review by: marcel, marius, alc Tested by: pluknet MD testing by: marcel, marius, gonzo, andreast
|
#
222813 |
|
07-Jun-2011 |
attilio |
etire the cpumask_t type and replace it with cpuset_t usage.
This is intended to fix the bug where cpu mask objects are capped to 32. MAXCPU, then, can now arbitrarely bumped to whatever value. Anyway, as long as several structures in the kernel are statically allocated and sized as MAXCPU, it is suggested to keep it as low as possible for the time being.
Technical notes on this commit itself: - More functions to handle with cpuset_t objects are introduced. The most notable are cpusetobj_ffs() (which calculates a ffs(3) for a cpuset_t object), cpusetobj_strprint() (which prepares a string representing a cpuset_t object) and cpusetobj_strscan() (which creates a valid cpuset_t starting from a string representation). - pc_cpumask and pc_other_cpus are target to be removed soon. With the moving from cpumask_t to cpuset_t they are now inefficient and not really useful. Anyway, for the time being, please note that access to pcpu datas is protected by sched_pin() in order to avoid migrating the CPU while reading more than one (possible) word - Please note that size of cpuset_t objects may differ between kernel and userland. While this is not directly related to the patch itself, it is good to understand that concept and possibly use the patch as a reference on how to deal with cpuset_t objects in userland, when accessing kernland members. - KTR_CPUMASK is changed and now is represented through a string, to be set as the example reported in NOTES.
Please additively note that no MAXCPU is bumped in this patch, but private testing has been done until to MAXCPU=128 on a real 8x8x2(htt) machine (amd64).
Please note that the FreeBSD version is not yet bumped because of the upcoming pcpu changes. However, note that this patch is not targeted for MFC.
People to thank for the time spent on this patch: - sbruno, pluknet and Nicholas Esborn (nick AT desert DOT net) tested several revision of the patches and really helped in improving stability of this work. - marius fixed several bugs in the sparc64 implementation and reviewed patches related to ktr. - jeff and jhb discussed the basic approach followed. - kib and marcel made targeted review on some specific part of the patch. - marius, art, nwhitehorn and andreast reviewed MD specific part of the patch. - marius, andreast, gonzo, nwhitehorn and jceel tested MD specific implementations of the patch. - Other people have made contributions on other patches that have been already committed and have been listed separately.
Companies that should be mentioned for having participated at several degrees: - Yahoo! for having offered the machines used for testing on big count of CPUs. - The FreeBSD Foundation for having sponsored my devsummit attendance, which has been instrumental. - Sandvine for having offered offices and infrastructure during development.
(I really hope I didn't forget anyone, if it happened I apologize in advance).
|
#
222614 |
|
02-Jun-2011 |
nwhitehorn |
Remove some dead code: unnecessary isyncs and memory sorting, which are handled in mtmsr() and mem_regions(), respectively.
|
#
220639 |
|
14-Apr-2011 |
andreast |
The macro MOEA_PVO_CHECK is empty and not used. It is a left over from the NetBSD import. Remove the definition and all its occurrences.
Approved by: nwhitehorn (mentor)
|
#
217688 |
|
21-Jan-2011 |
pluknet |
Make MSGBUF_SIZE kernel option a loader tunable kern.msgbufsize.
Submitted by: perryh pluto.rain.com (previous version) Reviewed by: jhb Approved by: kib (mentor) Tested by: universe
|
#
216563 |
|
19-Dec-2010 |
nwhitehorn |
Garbage-collect unused variable.
|
#
216174 |
|
04-Dec-2010 |
nwhitehorn |
Add an abstraction layer to the 64-bit AIM MMU's page table manipulation logic to support modifying the page table through a hypervisor. This uses KOBJ inheritance to provide subclasses of the base 64-bit AIM MMU class with additional methods for page table manipulation.
Many thanks to Peter Grehan for suggesting this design and implementing the MMU KOBJ inheritance mechanism.
|
#
215163 |
|
12-Nov-2010 |
nwhitehorn |
Remove use of a separate ofw_pmap on 32-bit CPUs. Many Open Firmware mappings need to end up in the kernel anyway since the kernel begins executing in OF context. Separating them adds needless complexity, especially since the powerpc64 and mmu_oea64 code gave up on it a long time ago.
As a side effect, the PPC ofw_machdep code is no longer AIM-specific, so move it to powerpc/ofw.
|
#
214617 |
|
01-Nov-2010 |
alc |
Implement pmap_is_prefaultable().
Reviewed by: nwhitehorn
|
#
214601 |
|
31-Oct-2010 |
nwhitehorn |
Add some missing parentheses so that moea_bat_mapped() actually works.
Submitted by: alc MFC after: 3 days
|
#
213407 |
|
04-Oct-2010 |
nwhitehorn |
Follow exactly the steps in architecture manual for correctly invalidating TLB entries instead of trying to cut corners.
|
#
213335 |
|
01-Oct-2010 |
nwhitehorn |
Fix pmap_page_set_memattr() behavior in the presence of fictitious pages by just caching the mode for later use by pmap_enter(), following amd64. While here, correct some mismerges from mmu_oea64 -> mmu_oea and clean up some dead code found while fixing the fictitious page behavior.
|
#
213307 |
|
30-Sep-2010 |
nwhitehorn |
Add support for memory attributes (pmap_mapdev_attr() and friends) on PowerPC/AIM. This is currently stubbed out on Book-E, since I have no idea how to implement it there.
|
#
212627 |
|
14-Sep-2010 |
grehan |
Introduce inheritance into the PowerPC MMU kobj interface.
include/mmuvar.h - Change the MMU_DEF macro to also create the class definition as well as define the DATA_SET. Add a macro, MMU_DEF_INHERIT, which has an extra parameter specifying the MMU class to inherit methods from. Update the comments at the start of the header file to describe the new macros.
booke/pmap.c aim/mmu_oea.c aim/mmu_oea64.c - Collapse mmu_def_t declaration into updated MMU_DEF macro
The MMU_DEF_INHERIT macro will be used in the PS3 MMU implementation to allow it to inherit the stock powerpc64 MMU methods.
Reviewed by: nwhitehorn
|
#
212322 |
|
08-Sep-2010 |
nwhitehorn |
Fix a typo in the original import of this code from NetBSD that caused the wrong element of the VSID bitmap array to be examined after a collision, leading to reallocation of in-use VSIDs under some circumstances, with attendant memory corruption. Also add an assert to check for this kind of problem in the future.
MFC after: 4 days
|
#
212278 |
|
06-Sep-2010 |
nwhitehorn |
Fix the same race condition on 32-bit AIM CPUs that was fixed for 64-bit ones in r211967 involving VSID allocation.
|
#
211861 |
|
27-Aug-2010 |
nwhitehorn |
pmap_mapdev() does not appear to actually need GIANT to be held here, and asserting that is held breaks drm.
MFC after: 2 weeks
|
#
209975 |
|
13-Jul-2010 |
nwhitehorn |
MFppc64:
Kernel sources for 64-bit PowerPC, along with build-system changes to keep 32-bit kernels compiling (build system changes for 64-bit kernels are coming later). Existing 32-bit PowerPC kernel configurations must be updated after this change to specify their architecture.
|
#
209369 |
|
20-Jun-2010 |
nwhitehorn |
Temporarily disable instruction relocation while setting up the kernel's IBAT entry in early boot in order to prevent possible faults from races between the instruction cache and the MMU.
PR: powerpc/148003 MFC after: 3 days
|
#
209048 |
|
11-Jun-2010 |
alc |
Relax one of the new assertions in pmap_enter() a little. Specifically, allow pmap_enter() to be performed on an unmanaged page that doesn't have VPO_BUSY set. Having VPO_BUSY set really only matters for managed pages. (See, for example, pmap_remove_write().)
|
#
208990 |
|
10-Jun-2010 |
alc |
Reduce the scope of the page queues lock and the number of PG_REFERENCED changes in vm_pageout_object_deactivate_pages(). Simplify this function's inner loop using TAILQ_FOREACH(), and shorten some of its overly long lines. Update a stale comment.
Assert that PG_REFERENCED may be cleared only if the object containing the page is locked. Add a comment documenting this.
Assert that a caller to vm_page_requeue() holds the page queues lock, and assert that the page is on a page queue.
Push down the page queues lock into pmap_ts_referenced() and pmap_page_exists_quick(). (As of now, there are no longer any pmap functions that expect to be called with the page queues lock held.)
Neither pmap_ts_referenced() nor pmap_page_exists_quick() should ever be passed an unmanaged page. Assert this rather than returning "0" and "FALSE" respectively.
ARM:
Simplify pmap_page_exists_quick() by switching to TAILQ_FOREACH().
Push down the page queues lock inside of pmap_clearbit(), simplifying pmap_clear_modify(), pmap_clear_reference(), and pmap_remove_write(). Additionally, this allows for avoiding the acquisition of the page queues lock in some cases.
PowerPC/AIM:
moea*_page_exits_quick() and moea*_page_wired_mappings() will never be called before pmap initialization is complete. Therefore, the check for moea_initialized can be eliminated.
Push down the page queues lock inside of moea*_clear_bit(), simplifying moea*_clear_modify() and moea*_clear_reference().
The last parameter to moea*_clear_bit() is never used. Eliminate it.
PowerPC/BookE:
Simplify mmu_booke_page_exists_quick()'s control flow.
Reviewed by: kib@
|
#
208847 |
|
05-Jun-2010 |
nwhitehorn |
Correct a harmless typo introduced when copying code from mmu_oea64.
Submitted by: alc MFC after: 8.1-RELEASE
|
#
208810 |
|
05-Jun-2010 |
alc |
Don't set PG_WRITEABLE in pmap_enter() unless the page is managed.
|
#
208574 |
|
26-May-2010 |
alc |
Push down page queues lock acquisition in pmap_enter_object() and pmap_is_referenced(). Eliminate the corresponding page queues lock acquisitions from vm_map_pmap_enter() and mincore(), respectively. In mincore(), this allows some additional cases to complete without ever acquiring the page queues lock.
Assert that the page is managed in pmap_is_referenced().
On powerpc/aim, push down the page queues lock acquisition from moea*_is_modified() and moea*_is_referenced() into moea*_query_bit(). Again, this will allow some additional cases to complete without ever acquiring the page queues lock.
Reorder a few statements in vm_page_dontneed() so that a race can't lead to an old reference persisting. This scenario is described in detail by a comment.
Correct a spelling error in vm_page_dontneed().
Assert that the object is locked in vm_page_clear_dirty(), and restrict the page queues lock assertion to just those cases in which the page is currently writeable.
Add object locking to vnode_pager_generic_putpages(). This was the one and only place where vm_page_clear_dirty() was being called without the object being locked.
Eliminate an unnecessary vm_page_lock() around vnode_pager_setsize()'s call to vm_page_clear_dirty().
Change vnode_pager_generic_putpages() to the modern-style of function definition. Also, change the name of one of the parameters to follow virtual memory system naming conventions.
Reviewed by: kib
|
#
208504 |
|
24-May-2010 |
alc |
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore().
Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page.
Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information.
Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock.
Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed().
Reviewed by: kib (an earlier version)
|
#
208175 |
|
16-May-2010 |
alc |
On entry to pmap_enter(), assert that the page is busy. While I'm here, make the style of assertion used by pmap_enter() consistent across all architectures.
On entry to pmap_remove_write(), assert that the page is neither unmanaged nor fictitious, since we cannot remove write access to either kind of page.
With the push down of the page queues lock, pmap_remove_write() cannot condition its behavior on the state of the PG_WRITEABLE flag if the page is busy. Assert that the object containing the page is locked. This allows us to know that the page will neither become busy nor will PG_WRITEABLE be set on it while pmap_remove_write() is running.
Correct a long-standing bug in vm_page_cowsetup(). We cannot possibly do copy-on-write-based zero-copy transmit on unmanaged or fictitious pages, so don't even try. Previously, the call to pmap_remove_write() would have failed silently.
|
#
207796 |
|
08-May-2010 |
alc |
Push down the page queues into vm_page_cache(), vm_page_try_to_cache(), and vm_page_try_to_free(). Consequently, push down the page queues lock into pmap_enter_quick(), pmap_page_wired_mapped(), pmap_remove_all(), and pmap_remove_write().
Push down the page queues lock into Xen's pmap_page_is_mapped(). (I overlooked the Xen pmap in r207702.)
Switch to a per-processor counter for the total number of pages cached.
|
#
207410 |
|
29-Apr-2010 |
kmacy |
On Alan's advice, rather than do a wholesale conversion on a single architecture from page queue lock to a hashed array of page locks (based on a patch by Jeff Roberson), I've implemented page lock support in the MI code and have only moved vm_page's hold_count out from under page queue mutex to page lock. This changes pmap_extract_and_hold on all pmaps.
Supported by: Bitgravity Inc.
Discussed with: alc, jeffr, and kib
|
#
207155 |
|
24-Apr-2010 |
alc |
Resurrect pmap_is_referenced() and use it in mincore(). Essentially, pmap_ts_referenced() is not always appropriate for checking whether or not pages have been referenced because it clears any reference bits that it encounters. For example, in mincore(), clearing the reference bits has two negative consequences. First, it throws off the activity count calculations performed by the page daemon. Specifically, a page on which mincore() has called pmap_ts_referenced() looks less active to the page daemon than it should. Consequently, the page could be deactivated prematurely by the page daemon. Arguably, this problem could be fixed by having mincore() duplicate the activity count calculation on the page. However, there is a second problem for which that is not a solution. In order to clear a reference on a 4KB page, it may be necessary to demote a 2/4MB page mapping. Thus, a mincore() by one process can have the side effect of demoting a superpage mapping within another process!
|
#
204128 |
|
20-Feb-2010 |
nwhitehorn |
Reduce KVA pressure on OEA64 systems running in bridge mode by mapping UMA segments at their physical addresses instead of into KVA. This emulates the direct mapping behavior of OEA32 in an ad-hoc way. To make this work properly required sharing the entire kernel PMAP with Open Firmware, so ofw_pmap is transformed into a stub on 64-bit CPUs.
Also implement some more tweaks to get more mileage out of our limited amount of KVA, principally by extending KVA into segment 16 until the beginning of the first OFW mapping.
Reported by: linimon
|
#
204042 |
|
18-Feb-2010 |
nwhitehorn |
Fix a bug where pages being removed from memory entirely no longer have PVOs, and so the modified state of the page can no longer be communicated to the VM layer, causing pages not to be flushed to swap when needed, in turn causing memory corruption. Also make several correctness adjustments to I-Cache synchronization and TLB invalidation for 64-bit Book-S CPUs.
Obtained from: projects/ppc64 Discussed with: grehan MFC after: 2 weeks
|
#
201758 |
|
07-Jan-2010 |
mbr |
Remove extraneous semicolons, no functional changes.
Submitted by: Marc Balmer <marc@msys.ch> MFC after: 1 week
|
#
198341 |
|
21-Oct-2009 |
marcel |
o Introduce vm_sync_icache() for making the I-cache coherent with the memory or D-cache, depending on the semantics of the platform. vm_sync_icache() is basically a wrapper around pmap_sync_icache(), that translates the vm_map_t argumument to pmap_t. o Introduce pmap_sync_icache() to all PMAP implementation. For powerpc it replaces the pmap_page_executable() function, added to solve the I-cache problem in uiomove_fromphys(). o In proc_rwmem() call vm_sync_icache() when writing to a page that has execute permissions. This assures that when breakpoints are written, the I-cache will be coherent and the process will actually hit the breakpoint. o This also fixes the Book-E PMAP implementation that was missing necessary locking while trying to deal with the I-cache coherency in pmap_enter() (read: mmu_booke_enter_locked).
The key property of this change is that the I-cache is made coherent *after* writes have been done. Doing it in the PMAP layer when adding or changing a mapping means that the I-cache is made coherent *before* any writes happen. The difference is key when the I-cache prefetches.
|
#
194784 |
|
23-Jun-2009 |
jeff |
Implement a facility for dynamic per-cpu variables. - Modules and kernel code alike may use DPCPU_DEFINE(), DPCPU_GET(), DPCPU_SET(), etc. akin to the statically defined PCPU_*. Requires only one extra instruction more than PCPU_* and is virtually the same as __thread for builtin and much faster for shared objects. DPCPU variables can be initialized when defined. - Modules are supported by relocating the module's per-cpu linker set over space reserved in the kernel. Modules may fail to load if there is insufficient space available. - Track space available for modules with a one-off extent allocator. Free may block for memory to allocate space for an extent.
Reviewed by: jhb, rwatson, kan, sam, grehan, marius, marcel, stas
|
#
192067 |
|
13-May-2009 |
nwhitehorn |
Factor out platform dependent things unrelated to device drivers into a new platform module. These are probed in early boot, and have the responsibility of determining the layout of physical memory, determining the CPU timebase frequency, and handling the zoo of SMP mechanisms found on PowerPC.
Reviewed by: marcel, raj Book-E parts by: raj
|
#
190681 |
|
03-Apr-2009 |
nwhitehorn |
Add support for 64-bit PowerPC CPUs operating in the 64-bit bridge mode provided, for example, on the PowerPC 970 (G5), as well as on related CPUs like the POWER3 and POWER4.
This also adds support for various built-in hardware found on Apple G5 hardware (e.g. the IBM CPC925 northbridge).
Reviewed by: grehan
|
#
189675 |
|
11-Mar-2009 |
nwhitehorn |
Change the PVO zone for fictitious pages to the unmanaged PVO zone, to match the unmanaged flag set in the PVO attributes. Without doing this, pmap_remove() could try to remove fictitious pages (like those created by mmap of physical memory) from the wrong UMA zone, causing a panic.
Reported by: Justin Hibbits MFC after: 1 week
|
#
183290 |
|
23-Sep-2008 |
nwhitehorn |
In preparation for PowerPC G5 support, allow PVO objects to contain page table entries for both the 32-bit and 64-bit AIM MMUs.
|
#
183094 |
|
16-Sep-2008 |
marcel |
o When not making a translation cache-inhibit and guarded (PTE_I|PTE_G) make it memory-coherency enforced (PTE_M). This is required for SMP to work. o Serialize tlbie operations and implement the tlbie operation in a function called tlbie(). Hardware can end up in a live-lock if between the tlbsync and subsequent sync on one processor another processor executes a tlbie or tlbsync. o Eliminate the following defines: TLBIE, TLBSYNC, SYNC and EIEIO Use either inline assembly statements or inline functions defined in <machine/cpufunc.h>
|
#
183060 |
|
15-Sep-2008 |
marcel |
Remove the tracing from the AP startup. The AP is known to start and the tracing can interfere with AP startup. Instead, use the available space in the reset vector for the initial stack.
|
#
182573 |
|
31-Aug-2008 |
marcel |
Remove redundant KTR statements.
|
#
179254 |
|
23-May-2008 |
marcel |
Invalidate the TLB in pmap_cpu_bootstrap(), so that it also happens on the APs.
|
#
178629 |
|
28-Apr-2008 |
marcel |
The first argment of mtdbatu or mtibatu is part of the encoding. It needs to be constant, so eliminate the loop and "hand-unroll".
|
#
178628 |
|
27-Apr-2008 |
marcel |
MFp4: SMP support
|
#
178265 |
|
16-Apr-2008 |
marcel |
Simplify the pmap_zero_page family of functions by making use of the fact that we have a 1:1 mapping by virtue of the BATs. Eliminate the now unused moea_rkva_alloc(), moea_pa_map() and moea_pa_unmap() functions.
Pointed out by: grehan.
|
#
178261 |
|
16-Apr-2008 |
marcel |
Allocate a stack (with optional guard pages) for thread0 and switch to it before calling mi_startup().
|
#
174599 |
|
14-Dec-2007 |
marcel |
Forced commit to record that this file was repocopied from src/sys/powerpc/powerpc and modified for its new location.
|
#
173708 |
|
17-Nov-2007 |
alc |
Prevent the leakage of wired pages in the following circumstances: First, a file is mmap(2)ed and then mlock(2)ed. Later, it is truncated. Under "normal" circumstances, i.e., when the file is not mlock(2)ed, the pages beyond the EOF are unmapped and freed. However, when the file is mlock(2)ed, the pages beyond the EOF are unmapped but not freed because they have a non-zero wire count. This can be a mistake. Specifically, it is a mistake if the sole reason why the pages are wired is because of wired, managed mappings. Previously, unmapping the pages destroys these wired, managed mappings, but does not reduce the pages' wire count. Consequently, when the file is unmapped, the pages are not unwired because the wired mapping has been destroyed. Moreover, when the vm object is finally destroyed, the pages are leaked because they are still wired. The fix is to reduce the pages' wired count by the number of wired, managed mappings destroyed. To do this, I introduce a new pmap function pmap_page_wired_mappings() that returns the number of managed mappings to the given physical page that are wired, and I use this function in vm_object_page_remove().
Reviewed by: tegge MFC after: 6 weeks
|
#
165362 |
|
19-Dec-2006 |
grehan |
Remove bogus increment of re-hashed PTEG index. This snuck in with r1.12 of pmap.c, and is potentially the cause of hangs reported on machines with a small amount of memory. On machines with sufficient RAM, and without a lot of processes running, this situation would probably never occur.
Testing is still incomplete, but it is obviously wrong so remove the offending code now.
The issue of what to do when both the primary and secondary hash overflow is still open.
Reported by: Dan Kresja at windriver dot com, via alc
|
#
164895 |
|
05-Dec-2006 |
grehan |
Fix gdb issue where the i-cache was not being updated when a breakpoint was written into a user's address space. The fix is to modify uiomove_fromphys to sync the icache when an executable user-space page is written into.
Alan Cox suggested that there should probably be a higher-level interface to this in the ptrace code, but agreed that this is an OK short-term solution.
Files changed:
pmap.h - declaration of pmap_page_executable() pmap_dispatch.c - pass through the page_executable call to the mmu object mmu_oea.c - implement the page_executable method by examining the PTE_EXEC field in the vm_page_t uio_machdep.c - in uiomove_fromphys(), if the op was a UIO_WRITE to user-space, and if the page is executable, sync the icache since this is at the least a breakpoint-write from gdb.
Reported by: marcel Tested by: marcel, grehan on g3+g4 Discussed with: alc MFC after: 2 weeks
|
#
164765 |
|
30-Nov-2006 |
grehan |
Don't use vm_page_flag_set() if installing bootstrap page-table entries since the vm page mutex's aren't yet initialized. Fixes boot-time panic.
Reported by: Dario Freni saturnero at freesbie dot org
|
#
164229 |
|
12-Nov-2006 |
alc |
Make pmap_enter() responsible for setting PG_WRITEABLE instead of its caller. (As a beneficial side-effect, a high-contention acquisition of the page queues lock in vm_fault() is eliminated.)
|
#
160889 |
|
01-Aug-2006 |
alc |
Complete the transition from pmap_page_protect() to pmap_remove_write(). Originally, I had adopted sparc64's name, pmap_clear_write(), for the function that is now pmap_remove_write(). However, this function is more like pmap_remove_all() than like pmap_clear_modify() or pmap_clear_reference(), hence, the name change.
The higher-level rationale behind this change is described in src/sys/amd64/amd64/pmap.c revision 1.567. The short version is that I'm trying to clean up and fix our support for execute access.
Reviewed by: marcel@ (ia64)
|
#
160235 |
|
10-Jul-2006 |
alc |
Add synchronization to moea_zero_page() and moea_zero_page_area().
Remove the acquisition and release of Giant from moea_zero_page_idle().
Tested by: grehan@
|
#
160068 |
|
01-Jul-2006 |
alc |
Eliminate the acquisition and release of Giant from moea_extract_and_hold() and moea_protect().
Tested by: grehan@ and rink@
|
#
159928 |
|
25-Jun-2006 |
alc |
Synchronize accesses to the PTEG table.
Add many lock assertions.
Tested by: grehan@
|
#
159705 |
|
17-Jun-2006 |
rink |
Prevent 'mutex not owned' panic on boot if INVARIANTS is in the kernel. This makes the GENERIC kernel boot on ppc.
Reviewed by: grehan Approved by: imp (mentor) MFC after: 1 week
dCVS: ----------------------------------------------------------------------
|
#
159627 |
|
14-Jun-2006 |
ups |
Remove mpte optimization from pmap_enter_quick(). There is a race with the current locking scheme and removing it should have no measurable performance impact. This fixes page faults leading to panics in pmap_enter_quick_locked() on amd64/i386.
Reviewed by: alc,jhb,peter,ps
|
#
159324 |
|
06-Jun-2006 |
alc |
Correct a typo in the previous revision.
|
#
159303 |
|
05-Jun-2006 |
alc |
Introduce the function pmap_enter_object(). It maps a sequence of resident pages from the same object. Use it in vm_map_pmap_enter() to reduce the locking overhead of premapping objects.
Reviewed by: tegge@
|
#
152180 |
|
08-Nov-2005 |
grehan |
Name change from pmap_* to moea_* to fit into the new order of mmu implementation.
This code handles the 32-bit 'OEA' MMU found on G2/G3/G4 PPC cores.
|
#
152177 |
|
08-Nov-2005 |
grehan |
Forced commit to note that this file has been repo-copied from pmap.c
|
#
149958 |
|
10-Sep-2005 |
grehan |
Fix boot-time hang/panic on G3 systems when modifying IBAT0 in pmap_bootstrap by using the sync;isync big hammer to make sure all prior operations have completed.
Reported by: Nathan Whitehorn <nathan at uchicago edu> MFC after: 2 days
|
#
149768 |
|
03-Sep-2005 |
alc |
Pass a value of type vm_prot_t to pmap_enter_quick() so that it determine whether the mapping should permit execute access.
|
#
147217 |
|
10-Jun-2005 |
alc |
Introduce a procedure, pmap_page_init(), that initializes the vm_page's machine-dependent fields. Use this function in vm_pageq_add_new_page() so that the vm_page's machine-dependent and machine-independent fields are initialized at the same time.
Remove code from pmap_init() for initializing the vm_page's machine-dependent fields.
Remove stale comments from pmap_init().
Eliminate the Boolean variable pmap_initialized from the alpha, amd64, i386, and ia64 pmap implementations. Its use is no longer required because of the above changes and earlier changes that result in physical memory that is being mapped at initialization time being mapped without pv entries.
Tested by: cognet, kensmith, marcel
|
#
143234 |
|
07-Mar-2005 |
grehan |
Replaced previous hw.physmem extraction with des's mods to getenv_ulong() - much simpler.
Pointed out by: des
|
#
143200 |
|
06-Mar-2005 |
grehan |
Allow user to undersize memory with hw.physmem loader variable.
Obtained from: i386/machdep.c:getmemsize()
|
#
142416 |
|
25-Feb-2005 |
grehan |
Add PVO_FAKE flag to pvo entries for PG_FICTITIOUS mappings, to avoid trying to reverse-map a device physical address to the vm_page array and walking into non-existent vm weeds.
found by: Xorg server exiting
|
#
140538 |
|
20-Jan-2005 |
grehan |
Fix (accidental?) lock order reversal in pmap_remove. Found when a process that has mmap'd device mem exits.
|
#
139825 |
|
07-Jan-2005 |
imp |
/* -> /*- for license, minor formatting changes
|
#
139401 |
|
29-Dec-2004 |
grehan |
Correctly initialise the 2nd kernel segment, and don't forget to actually install it in the segment register. This may fix some of the weird panics seen when kernel VM is heavily used.
|
#
139241 |
|
23-Dec-2004 |
alc |
Modify pmap_enter_quick() so that it expects the page queues to be locked on entry and it assumes the responsibility for releasing the page queues lock if it must sleep.
Remove a bogus comment from pmap_enter_quick().
Using the first change, modify vm_map_pmap_enter() so that the page queues lock is acquired and released once, rather than each time that a page is mapped.
|
#
138897 |
|
15-Dec-2004 |
alc |
In the common case, pmap_enter_quick() completes without sleeping. In such cases, the busying of the page and the unlocking of the containing object by vm_map_pmap_enter() and vm_fault_prefault() is unnecessary overhead. To eliminate this overhead, this change modifies pmap_enter_quick() so that it expects the object to be locked on entry and it assumes the responsibility for busying the page and unlocking the object if it must sleep. Note: alpha, amd64, i386 and ia64 are the only implementations optimized by this change; arm, powerpc, and sparc64 still conservatively busy the page and unlock the object within every pmap_enter_quick() call.
Additionally, this change is the first case where we synchronize access to the page's PG_BUSY flag and busy field using the containing object's lock rather than the global page queues lock. (Modifications to the page's PG_BUSY flag and busy field have asserted both locks for several weeks, enabling an incremental transition.)
|
#
135172 |
|
13-Sep-2004 |
alc |
Lock the kernel pmap in pmap_kenter().
Tested by: gallatin@
|
#
134535 |
|
30-Aug-2004 |
alc |
- Introduce a lock for synchronizing access to the pvo and pteg tables. - In pmap_enter(), only the acquisition and release of the page queues lock needs to check the bootstrap flag.
Tested by: gallatin@
|
#
134453 |
|
28-Aug-2004 |
alc |
Eliminate unnecessary indirection.
|
#
134329 |
|
26-Aug-2004 |
alc |
Add pmap locking to many of the functions.
Many thanks to Andrew Gallatin for resolving a powerpc-specific initialization problem in my original patch.
Tested by: gallatin@
|
#
133862 |
|
16-Aug-2004 |
marius |
Instead of "OpenFirmware", "openfirmware", etc. use the official spelling "Open Firmware" from IEEE 1275 and OpenFirmware.org (no pun intended).
Ok'ed by: tmm
|
#
133855 |
|
16-Aug-2004 |
ssouhlal |
Add /dev/mem and /dev/kmem to powerpc.
Approved by: grehan (mentor)
|
#
133166 |
|
05-Aug-2004 |
grehan |
In pmap_page_protect, clear the vm page's PG_WRITEABLE flag if downgrading to read-only. Found by triggering the KASSERT in vm_pageout_flush().
|
#
133143 |
|
04-Aug-2004 |
alc |
- Push down the acquisition and release of Giant into pmap_enter_quick() on those architectures without pmap locking. - Eliminate the acquisition and release of Giant in vm_map_pmap_enter().
|
#
132899 |
|
30-Jul-2004 |
alc |
- Push down the acquisition and release of Giant into pmap_protect() on those architectures without pmap locking. - Eliminate the acquisition and release of Giant from vm_map_protect().
(Translation: mprotect(2) runs to completion without touching Giant on alpha, amd64, i386 and ia64.)
|
#
132666 |
|
26-Jul-2004 |
alc |
Implement the protection check required by the pmap_extract_and_hold() specification.
Reviewed and tested by: grehan@
|
#
132220 |
|
15-Jul-2004 |
alc |
Push down the acquisition and release of the page queues lock into pmap_protect() and pmap_remove(). In general, they require the lock in order to modify a page's pv list or flags. In some cases, however, pmap_protect() can avoid acquiring the lock.
|
#
132011 |
|
12-Jul-2004 |
alc |
pmap_remove_pages() must not remove wired mappings. Since pmap_remove_pages() is an optimization, its implementation is optional.
Discussed with: grehan
|
#
131808 |
|
08-Jul-2004 |
grehan |
G4 requires isync after 256Mb ibat/dbat update, G3 requires isync after each bat update. Otherwise, pmap_bootstrap causes an ISI exception. A fall-out of loader BAT removal.
|
#
131658 |
|
05-Jul-2004 |
alc |
Correct pmap_extract()'s return type. It should be vm_paddr_t, not vm_offset_t.
|
#
131401 |
|
01-Jul-2004 |
grehan |
Modify loop test when cycling through phys_avail array. It's possible for an OpenFirmware implementation to have a single memory region (hello PearPC).
|
#
128103 |
|
11-Apr-2004 |
alc |
Remove avail_end. It is not used.
|
#
127875 |
|
05-Apr-2004 |
alc |
Remove avail_start on those platforms that no longer use it. (Only amd64 does anything with it beyond simple initialization.)
|
#
127869 |
|
04-Apr-2004 |
alc |
Remove unused arguments from pmap_init().
|
#
126728 |
|
07-Mar-2004 |
alc |
Retire pmap_pinit2(). Alpha was the last platform that used it. However, ever since alpha/alpha/pmap.c revision 1.81 introduced the list allpmaps, there has been no reason for having this function on Alpha. Briefly, when pmap_growkernel() relied upon the list of all processes to find and update the various pmaps to reflect a growth in the kernel's valid address space, pmap_init2() served to avoid a race between pmap initialization and pmap_growkernel(). Specifically, pmap_pinit2() was responsible for initializing the kernel portions of the pmap and pmap_pinit2() was called after the process structure contained a pointer to the new pmap for use by pmap_growkernel(). Thus, an update to the kernel's address space might be applied to the new pmap unnecessarily, but an update would never be lost.
|
#
126478 |
|
02-Mar-2004 |
grehan |
Increase kernel VA from 256Mb to 512Mb by shifting the segment used for user copyinout down to 12, and keeping segments 13/14 for kernel VA.
It would be nice to have more available, but segments lower than this are reserved for either memory or 1:1 mapped device i/o, and seg 15 is OpenFirmware ROM. Also, the effort to keep OpenFirmware available for callbacks limits the use of VA-mapped segments. Fortunately UMA_MD_SMALL_ALLOC takes away a lot of VM pressure.
Obtained from: NetBSD
|
#
125687 |
|
11-Feb-2004 |
grehan |
Cleaned up param.h:
- culled long-dead #define's - segment register defs moved to sr.h - NPMAPS moved to pmap.h - KERNBASE moved to vmparam.h - removed include of <machine/cpu.h> and fixed src files that relied on this.
Modifying segment register code no longer causes gcc rebuilds :-)
|
#
125442 |
|
04-Feb-2004 |
grehan |
Remove pmap_pvo_allocf zone alloc function. It was a way of using the direct-mapping of physmem to force PTE data structures to be physically addressable so the interrupt-time real-mode DSI trap handler could perform PTE spills. However, the memory may have been > 256Mb, which would have caused a BAT spill and double-interrupt.
The new trap code no longer handles PTE spills, so the requirement that these pages be direct-mapped no longer applies. The irony is UMA_MD_SMALL_ALLOC will return direct mappings for these structs :-)
|
#
125185 |
|
28-Jan-2004 |
grehan |
When UMA_MD_SMALL_ALLOC is defined, pmap_kextract will be called for direct-mapped addresses. Assume that any address less than KVA is one of these and return it. Also assert that an address is KVA does have a valid mapping - callers of pmap_kextract don't check the return value, since they assume that they have a valid virtual address.
|
#
123560 |
|
16-Dec-2003 |
grehan |
Disable the per-vm_page PTE cache. This was not being invalidated correctly, resulting in the dreaded "vm_pageout_flush: partially invalid page" panic. The caching issue will be revisited in the future, but opt for safety over performance in the meantime.
Tested by: gallatin
|
#
123354 |
|
09-Dec-2003 |
gallatin |
pmap_query_bit() should return false if the bit is not set.
Reviewed by: grehan
|
#
120722 |
|
03-Oct-2003 |
alc |
Migrate pmap_prefault() into the machine-independent virtual memory layer.
A small helper function pmap_is_prefaultable() is added. This function encapsulate the few lines of pmap_prefault() that actually vary from machine to machine. Note: pmap_is_prefaultable() and pmap_mincore() have much in common. Going forward, it's worth considering their merger.
|
#
120336 |
|
22-Sep-2003 |
grehan |
Soften assert in pmap_remove_all. Introduct pmap_extract_and_hold.
Stolen from: sparc64
|
#
118365 |
|
02-Aug-2003 |
alc |
Use kmem_alloc_nofault() rather than kmem_alloc_pageable() in pmap_mapdev(). See revision 1.140 of kern/sys_pipe.c for a detailed rationale.
Submitted by: tegge
|
#
118244 |
|
31-Jul-2003 |
bmilekic |
Make sure that when the PV ENTRY zone is created in pmap, that it's created not only with UMA_ZONE_VM but also with UMA_ZONE_NOFREE. In the i386 case in particular, the pmap code would hook a special page allocation routine that allocated from kernel_map and not kmem_map, and so when/if the pageout daemon drained the zones, it could actually push out slabs from the PV ENTRY zone but call UMA's default page_free, which resulted in pages allocated from kernel_map being freed to kmem_map; bad. kmem_free() ignores the return value of the vm_map_delete and just returns. I'm not sure what the exact repercussions could be, but it doesn't look good.
In the PAE case on i386, we also set-up a zone in pmap, so be conservative for now and make that zone also ZONE_NOFREE and ZONE_VM. Do this for the pmap zones for the other archs too, although in some cases it may not be entirely necessarily. We'd rather be safe than sorry at this point.
Perhaps all UMA_ZONE_VM zones should by default be also UMA_ZONE_NOFREE?
May fix some of silby's crashes on the PV ENTRY zone.
|
#
118239 |
|
30-Jul-2003 |
peter |
Deal with 'options KSTACK_PAGES' being a global option.
|
#
118100 |
|
27-Jul-2003 |
alc |
Make pmap_pvo_allocf() callable without Giant.
|
#
117206 |
|
03-Jul-2003 |
alc |
Background: pmap_object_init_pt() premaps the pages of a object in order to avoid the overhead of later page faults. In general, it implements two cases: one for vnode-backed objects and one for device-backed objects. Only the device-backed case is really machine-dependent, belonging in the pmap.
This commit moves the vnode-backed case into the (relatively) new function vm_map_pmap_enter(). On amd64 and i386, this commit only amounts to code rearrangement. On alpha and ia64, the new machine independent (MI) implementation of the vnode case is smaller and more efficient than their pmap-based implementations. (The MI implementation takes advantage of the fact that objects in -CURRENT are ordered collections of pages.) On sparc64, pmap_object_init_pt() hadn't (yet) been implemented.
|
#
117045 |
|
29-Jun-2003 |
alc |
- Export pmap_enter_quick() to the MI VM. This will permit the implementation of a largely MI pmap_object_init_pt() for vnode-backed objects. pmap_enter_quick() is implemented via pmap_enter() on sparc64 and powerpc. - Correct a mismatch between pmap_object_init_pt()'s prototype and its various implementations. (I plan to keep pmap_object_init_pt() as the MD hook for device-backed objects on i386 and amd64.) - Correct an error in ia64's pmap_enter_quick() and adjust its interface to match the other versions. Discussed with: marcel
|
#
116355 |
|
14-Jun-2003 |
alc |
Migrate the thread stack management functions from the machine-dependent to the machine-independent parts of the VM. At the same time, this introduces vm object locking for the non-i386 platforms.
Two details:
1. KSTACK_GUARD has been removed in favor of KSTACK_GUARD_PAGES. The different machine-dependent implementations used various combinations of KSTACK_GUARD and KSTACK_GUARD_PAGES. To disable guard page, set KSTACK_GUARD_PAGES to 0.
2. Remove the (unnecessary) clearing of PG_ZERO in vm_thread_new. In 5.x, (but not 4.x,) PG_ZERO can only be set if VM_ALLOC_ZERO is passed to vm_page_alloc() or vm_page_grab().
|
#
116328 |
|
14-Jun-2003 |
alc |
Move the *_new_altkstack() and *_dispose_altkstack() functions out of the various pmap implementations into the machine-independent vm. They were all identical.
|
#
113038 |
|
03-Apr-2003 |
obrien |
Use __FBSDID rather than rcsid[].
|
#
111462 |
|
25-Feb-2003 |
mux |
Cleanup of the d_mmap_t interface.
- Get rid of the useless atop() / pmap_phys_address() detour. The device mmap handlers must now give back the physical address without atop()'ing it. - Don't borrow the physical address of the mapping in the returned int. Now we properly pass a vm_offset_t * and expect it to be filled by the mmap handler when the mapping was successful. The mmap handler must now return 0 when successful, any other value is considered as an error. Previously, returning -1 was the only way to fail. This change thus accidentally fixes some devices which were bogusly returning errno constants which would have been considered as addresses by the device pager. - Garbage collect the poorly named pmap_phys_address() now that it's no longer used. - Convert all the d_mmap_t consumers to the new API.
I'm still not sure wheter we need a __FreeBSD_version bump for this, since and we didn't guarantee API/ABI stability until 5.1-RELEASE.
Discussed with: alc, phk, jake Reviewed by: peter Compile-tested on: LINT (i386), GENERIC (alpha and sparc64) Runtime-tested on: i386
|
#
110172 |
|
01-Feb-2003 |
grehan |
- add pmap_pagedaemon_waken variable - remove dead code and fix warnings in pmap_zero_page/zero_page_area - implement pmap_clear_reference pmap_ts_referenced pmap_page_exists_quick pmap_remove_all - align pmap_qenter/qremove closer with i386 code - fix vm_page locking in pmap_new_thread (from benno) - add new parameter to pmap_clear_bit to return original pte value
Approved by: benno
|
#
109920 |
|
27-Jan-2003 |
benno |
Back out some changes that snuck in with the last commit.
Pointy hat to: benno
|
#
109919 |
|
27-Jan-2003 |
benno |
Flesh out bus_dmamap_sync.
|
#
106838 |
|
13-Nov-2002 |
alc |
Move pmap_collect() out of the machine-dependent code, rename it to reflect its new location, and add page queue and flag locking.
Notes: (1) alpha, i386, and ia64 had identical implementations of pmap_collect() in terms of machine-independent interfaces; (2) sparc64 doesn't require it; (3) powerpc had it as a TODO.
|
#
104434 |
|
03-Oct-2002 |
grehan |
- fix zero-sized stack alloc from previous commit. a default is now selected ala sparc64 - KSEIII routines implemented (taken from i386/sparc64)
Approved by: Benno
|
#
104354 |
|
02-Oct-2002 |
scottl |
Some kernel threads try to do significant work, and the default KSTACK_PAGES doesn't give them enough stack to do much before blowing away the pcb. This adds MI and MD code to allow the allocation of an alternate kstack who's size can be speficied when calling kthread_create. Passing the value 0 prevents the alternate kstack from being created. Note that the ia64 MD code is missing for now, and PowerPC was only partially written due to the pmap.c being incomplete there. Though this patch does not modify anything to make use of the alternate kstack, acpi and usb are good candidates.
Reviewed by: jake, peter, jhb
|
#
103604 |
|
19-Sep-2002 |
grehan |
- use BAT registers to map device space and physical memory - remove test in pmap_activate that prevented vmspace sharing (v/rfork) - always sync icache in pmap_enter until problems are sorted - fix incorrect use of regions in pmap_kenter - bring in pmap_release from NetBSD - fix overwrite of bootstrap flag in pmap_pvo_enter
Approved by: benno
|
#
102399 |
|
25-Aug-2002 |
alc |
o Retire pmap_pageable(). It's an advisory routine that none of our platforms implements.
|
#
101346 |
|
04-Aug-2002 |
alc |
o Don't set PG_MAPPED or PG_WRITEABLE when a page is mapped using pmap_kenter() or pmap_qenter(). o Use VM_ALLOC_WIRED in pmap_new_thread().
|
#
100319 |
|
18-Jul-2002 |
benno |
Remove the statically allocated array that holds OpenFirmware memory mappings during pmap_bootstrap. Instead, temporarily help ourselves to some memory from phys_avail since we won't need it post-boostrap.
|
#
99666 |
|
09-Jul-2002 |
benno |
Add an implementation for pmap_zero_page_area.
|
#
99571 |
|
08-Jul-2002 |
peter |
Add a special page zero entry point intended to be called via the single threaded VM pagezero kthread outside of Giant. For some platforms, this is really easy since it can just use the direct mapped region. For others, IPI sending is involved or there are other issues, so grab Giant when needed.
We still have preemption issues to deal with, but Alan Cox has an interesting suggestion on how to minimize the problem on x86.
Use Luigi's hack for preserving the (lack of) priority.
Turn the idle zeroing back on since it can now actually do something useful outside of Giant in many cases.
|
#
99559 |
|
07-Jul-2002 |
peter |
Collect all the (now equivalent) pmap_new_proc/pmap_dispose_proc/ pmap_swapin_proc/pmap_swapout_proc functions from the MD pmap code and use a single equivalent MI version. There are other cleanups needed still.
While here, use the UMA zone hooks to keep a cache of preinitialized proc structures handy, just like the thread system does. This eliminates one dependency on 'struct proc' being persistent even after being freed. There are some comments about things that can be factored out into ctor/dtor functions if it is worth it. For now they are mostly just doing statistics to get a feel of how it is working.
|
#
99557 |
|
07-Jul-2002 |
peter |
Update for post-kse3 pmap kthread allocation changes
|
#
99038 |
|
29-Jun-2002 |
benno |
Add pmap_mapdev and pmap_unmapdev.
|
#
99037 |
|
29-Jun-2002 |
benno |
- Initialise battable to cover I/O spaces. - Statically size the bpvo entries to avoid conflicts between bpvo allocation and the vm allocator. - Shift pmap_init2 code into pmap_init. - Add UMA_ZONE_VM flag to uma_zcreate.
Submitted by: Peter Grehan <peterg@ptree32.com.au>
|
#
97385 |
|
28-May-2002 |
benno |
Implement pmap_copy and pmap_copy_page.
|
#
97346 |
|
27-May-2002 |
benno |
Get the correct memory regions from OpenFirmware. We were getting the "available" ranges, not the "physical" ranges. Clean up some of the bootstrap code in the process.
Submitted by: Peter Grehan <peterg@ptree32.com.au>
|
#
96353 |
|
10-May-2002 |
benno |
Implement the following functions: - pmap_addr_hint - pmap_change_wiring - pmap_extract - pmap_is_modified
|
#
96334 |
|
10-May-2002 |
benno |
Improve our detection of an attempted duplicate entry. We may be trying to change the page protection bits.
|
#
96250 |
|
09-May-2002 |
benno |
1. Better track the executable status of mappings. 2. Set a pcpu variable to the real address of the active pmap (used when exiting from traps.
Obtained from: NetBSD (1)
|
#
95710 |
|
29-Apr-2002 |
peter |
Tidy up some loose ends. i386/ia64/alpha - catch up to sparc64/ppc: - replace pmap_kernel() with refs to kernel_pmap - change kernel_pmap pointer to (&kernel_pmap_store) (this is a speedup since ld can set these at compile/link time) all platforms (as suggested by jake): - gc unused pmap_reference - gc unused pmap_destroy - gc unused struct pmap.pm_count (we never used pm_count - we track address space sharing at the vmspace)
|
#
94839 |
|
16-Apr-2002 |
benno |
Correct a comment.
|
#
94838 |
|
16-Apr-2002 |
benno |
Implement the following functions: - pmap_kextract - pmap_object_init_pt - pmap_protect - pmap_remove_pages
I'm pretty sure pmap_remove_pages is at least somewhat bogus.
|
#
94837 |
|
16-Apr-2002 |
benno |
Remove some dead code.
|
#
94836 |
|
16-Apr-2002 |
benno |
Use mtsrin() instead of inline asm.
|
#
94835 |
|
16-Apr-2002 |
benno |
Change the value of PMAP_BOOTSTRAP so we don't stomp on the PTE index value.
|
#
94777 |
|
15-Apr-2002 |
peter |
Pass vm_page_t instead of physical addresses to pmap_zero_page[_area]() and pmap_copy_page(). This gets rid of a couple more physical addresses in upper layers, with the eventual aim of supporting PAE and dealing with the physical addressing mostly within pmap. (We will need either 64 bit physical addresses or page indexes, possibly both depending on the circumstances. Leaving this to pmap itself gives more flexibilitly.)
Reviewed by: jake Tested on: i386, ia64 and (I believe) sparc64. (my alpha was hosed)
|
#
94753 |
|
15-Apr-2002 |
benno |
Turn some CTR's into CTR0's.
|
#
92847 |
|
20-Mar-2002 |
jeff |
Remove references to vm_zone.h and switch over to the new uma API.
|
#
92757 |
|
20-Mar-2002 |
benno |
Increment pmap_pvo_count in the right place.
|
#
92654 |
|
19-Mar-2002 |
jeff |
This is the first part of the new kernel memory allocator. This replaces malloc(9) and vm_zone with a slab like allocator.
Reviewed by: arch@
|
#
92521 |
|
17-Mar-2002 |
benno |
Changes and fixes in preparation for UMA:
- Bootstrap pvo entries are now allocated by stealing pages. - Just return if we're pmap_enter'ing a mapping that's already there. Don't remove it and re-enter it.
|
#
92067 |
|
11-Mar-2002 |
benno |
Correct a typo. (* that should've been &)
|
#
91802 |
|
07-Mar-2002 |
benno |
Copy the "implementation" of pmap_prefault from sparc64.
|
#
91793 |
|
07-Mar-2002 |
benno |
Calculate physmem.
|
#
91483 |
|
28-Feb-2002 |
benno |
- Modify pmap_activate so it only marks the pmap as active. - Add a pmap_deactivate function.
|
#
91456 |
|
28-Feb-2002 |
benno |
Implement the following functions: - pmap_remove - pmap_kremove - pmap_qremove
|
#
91455 |
|
28-Feb-2002 |
benno |
Remove most of the usage of critical_enter/exit.
I put these in to match the use of spl*() in the NetBSD code I was basing this on, but it appears to cause problems.
I'm doing this in a separate commit so as to be able to refer back if locking becomes an issue at a later stage.
|
#
91403 |
|
27-Feb-2002 |
silby |
Fix a horribly suboptimal algorithm in the vm_daemon.
In order to determine what to page out, the vm_daemon checks reference bits on all pages belonging to all processes. Unfortunately, the algorithm used reacted badly with shared pages; each shared page would be checked once per process sharing it; this caused an O(N^2) growth of tlb invalidations. The algorithm has been changed so that each page will be checked only 16 times.
Prior to this change, a fork/sleepbomb of 1300 processes could cause the vm_daemon to take over 60 seconds to complete, effectively freezing the system for that time period. With this change in place, the vm_daemon completes in less than a second. Any system with hundreds of processes sharing pages should benefit from this change.
Note that the vm_daemon is only run when the system is under extreme memory pressure. It is likely that many people with loaded systems saw no symptoms of this problem until they reached the point where swapping began.
Special thanks go to dillon, peter, and Chuck Cranor, who helped me get up to speed with vm internals.
PR: 33542, 20393 Reviewed by: dillon MFC after: 1 week
|
#
91131 |
|
23-Feb-2002 |
benno |
Don't call critical_enter()/critical_exit() around calls to pmap_pvo_enter() as it does it's own handling of critical sections.
|
#
90643 |
|
13-Feb-2002 |
benno |
Complete rework of the PowerPC pmap and a number of other bits in the early boot sequence.
The new pmap.c is based on NetBSD's newer pmap.c (for the mpc6xx processors) which is 70% faster than the older code that the original pmap.c was based on. It has also been based on the framework established by jake's initial sparc64 pmap.c.
There is no change to how far the kernel gets (it makes it to the mountroot prompt in psim) but the new pmap code is a lot cleaner.
Obtained from: NetBSD (pmap code)
|
#
85201 |
|
19-Oct-2001 |
mp |
Fix includes based on recent changes to lock.h, mutex.h and ktr.h.
|
#
84921 |
|
14-Oct-2001 |
benno |
Implement pmap_mapdev.
|
#
83730 |
|
20-Sep-2001 |
mp |
Add missing include file.
|
#
83683 |
|
19-Sep-2001 |
mp |
Use BATL/BATU macros instead of hardcoded hex constants.
|
#
83682 |
|
19-Sep-2001 |
mp |
Update PowerPC MD code to compile and do initial bootstrap based on recent changes (KSE and VM requiring physmem to be setup).
Reviewed by: benno, jhb, julian
|
#
83366 |
|
12-Sep-2001 |
julian |
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process.
Sorry john! (your next MFC will be a doosie!)
Reviewed by: peter@freebsd.org, dillon@freebsd.org
X-MFC after: ha ha ha ha
|
#
82634 |
|
31-Aug-2001 |
peter |
Similar to changes on i386/alpha/etc pmap.c; converge on a similar look/feel on pmap_new_proc() with some cosmetic style changes.
|
#
80431 |
|
26-Jul-2001 |
peter |
Make PMAP_SHPGPERPROC tunable. One shouldn't need to recompile a kernel for this, since it is easy to run into with large systems with lots of shared mmap space.
Obtained from: yahoo
|
#
78882 |
|
27-Jun-2001 |
benno |
Forced commit due to mishap in the previous commit.
Add in an implementation of pmap_new_proc.
|
#
78880 |
|
27-Jun-2001 |
benno |
Fix comment breakage.
|
#
77957 |
|
10-Jun-2001 |
benno |
Bring in NetBSD code used in the PowerPC port.
Reviewed by: obrien, dfr Obtained from: NetBSD
|