MFC: r267961, r309361, r322710, r323286, r326378, r326383, r326407

Sync. hwpstate with head.

r267961 (hselasky, partial):

Remove a redundant TUNABLE statement.

r309361 (danfe):

- Mention mismatching numbers in MSR vs. ACPI _PSS count warning.
- Rephrase unsupported AMD CPUs message and wrap as an overly long line.
- Improve readability when reporting resulted P-state transition (debug).

r322710, r323286 (cem):

- Add support for family 17h pstate info from MSRs.
- Yield CPU awaiting frequency change.

r326378, r326383, r326407:

- Fix some style(9) nits.
- Add a tunable "debug.hwpstate_verify" to check P-state after changing it
and turn it off by default.

MFC 308005: Add powerd(8) support for several families of AMD CPUs.

Use the same logic to calculate the nominal CPU frequency from the P-state
MSRs on family 0x12, 0x15, and 0x16 CPUs as is used for family 0x10.
Family 0x14 was included in the original patch in the PR but I left that
out as the BIOS writer's guide for family 0x14 CPUs show a different layout
for the relevant MSR and include a different formulate for calculating the
frequency.

While here, simplify a few expressions and print out the family of
unsupported CPUs in hex rather than decimal.

PR: 212020

MFC r303891, r303892:
sys: replace comma with semicolon when pertinent.

Uses of commas instead of a semicolons can easily go undetected. The comma
can serve as a statement separator but this shouldn't be abused when
statements are meant to be standalone.

MFC r259197:
Do not DELAY() for P-state transition unless we want to see the result.

Intel manual says: "If a transition is already in progress, transition to
a new value will subsequently take effect. Reads of IA32_PERF_CTL determine
the last targeted operating point." So seems it should be fine to just
trigger wanted transition and go. Linux does the same.

MFC r257769 to stable/10

Fix powerd/states on AMD cpus. Resolves issues with system reporting:
hwpstate0: set freq failed, err 6

Tested on FX-8150 and others.

PR: kern/167018
Submitted by: avg@
Approved by: re (gjb)

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Adding a detach method to p4tcc driver.

PR: 118739
Submitted by: Dan Lukes <dan@obluda.cz> (earlier version)
Reviewed by: jhb
Approved by: sbruno (mentor)
MFC after: 1 week

This isn't functionally identical. In some cases a hint to disable
unit 0 would in fact disable all units.

This reverts r241856

Approved by: cperciva (implicit)

Now that device disabling is generic, remove extraneous code from the
device drivers that used to provide this feature.

Reviewed by: des
Approved by: cperciva
MFC after: 1 week

Fix apparent logic reversal in setting the 'auto_mode' flag.

MFC after: 2 weeks

Use ACPI-supplied CPU frequencies instead of estimated ones as we are about
to use other values from the same table anyway.

MFC after: 3 days

Use atomic load & store for TSC frequency. It may be overkill for amd64 but
safer for i386 because it can be easily over 4 GHz now. More worse, it can
be easily changed by user with 'machdep.tsc_freq' tunable (directly) or
cpufreq(4) (indirectly). Note it is intentionally not used in performance
critical paths to avoid performance regression (but we should, in theory).
Alternatively, we may add "virtual TSC" with lower frequency if maximum
frequency overflows 32 bits (and ignore possible incoherency as we do now).

Set C1 "I/O then Halt" capability bit for Intel EIST. Some broken BIOSes
refuse to load external SSDTs if this bit is unset for _PDC. It seems Linux
and OpenSolaris did the same long ago.

MFC after: 1 week

hwpstate: use CPU_FOREACH when binding to all available processors

Also, add a comment mentioning _PSD - on some systems it's enough to
put one logical CPU into a particular P-state to make other CPUs in
the same domain to enter that P-state.

Also, call sched_unbind() after the loop - sched_bind() automatically
rebinds from previous CPU to a new one, and the new arrangement of code
is safer against early loop exit.

Plus one minor style nit.

MFC after: 10 days

make it possible to actually enable hwpstate_verbose

Either via the tunable or the sysctl.

MFC after: 3 days

Few whitespace cleanups and comments tunings.

Submitted by: arundel

Core i5, same as previously Core2Duo, found to not set P-state for single
core lower then set on other cores. Do not try to test P-states on attach
on SMP systems. It is hopeless now and will just pollute verbose logs.
If needed, check still can be forced via loader tunable.

Introduce the new kernel sub-tree x86 which should contain all the code
shared and generalized between our current amd64, i386 and pc98.

This is just an initial step that should lead to a more complete effort.
For the moment, a very simple porting of cpufreq modules, BIOS calls and
the whole MD specific ISA bus part is added to the sub-tree but ideally
a lot of code might be added and more shared support should grow.

Sponsored by: Sandvine Incorporated
Reviewed by: emaste, kib, jhb, imp
Discussed on: arch
MFC: 3 weeks

Previous solution appeared to be unsufficient. After additional testing
I have found that it is not only desktop CPUs problem. but mobile also.
Probably AP on laptops just started initially at lower frequency, hiding
the problem.

Disable frequency validation by default, for systems with more then one CPU,
until we can implement it properly. It looks like making more harm now then
benefits. Add 'hw.est.strict' loader tunable to control it.

Now my iXsystems Invincibook is able to run at 800MHz lowest frequency,
instead of 1200MHz before, when 800MHz was incorrectly reported invalid.

Retry only once, if BIOS is completely broken and gives zero freqs.

Desktop Core2Duo/Core2Quad CPUs are unable to control frequency of single
CPU core, only pair of them. As result, both cores are running on highest
one of requested frequencies, and that is reported by status register.
Such behavior confuses frequency validation logic, as it runs on only
one core, as SMP is not yet launched, making EIST completely unusable.

To workaround this, add check for validation result. If we haven't found
at least two usable frequencies, then probably we are looking bad and have
to trust data provided by BIOS as-is.

Consolidate CPUID to CPU family/model macros for amd64 and i386 to reduce
unnecessary #ifdef's for shared code between them.

Import ACPICA 20090521.

Remove unused variables.

Found with: Coverity Prevent(tm)
CID: 4285, 4286

Extend comment in copyright notice as requested by author.

Submitted by: G.Otsuji

Add support for Phenom (Family 10h) to cpufreq.
Its a newer version provided by the author than in the PR.

PR: kern/128575
Submitted by: Gen Otsuji annona2 [at] gmail.com

Include a missing header file.

Reported by: thompsa, "build universe"

Replace couple of strcmp(cpu_vendor, "foo") with cpu_vendor_id for i386
and hide i386-specific code under #ifdef.

Add Centaur/IDT/VIA vendor ID for Nano family, which has long mode support.

Remove an unused variable.

Found with: Coverity Prevent(tm)
CID: 3677

Introduce cpu_vendor_id and replace a lot of strcmp(cpu_vendor, "...").

Reviewed by: jhb, peter (early amd64 version)

Add Pentium D cores into the list that can't handle 12.5% and 25%
throttle. My SMP kernel hangs when one of those is selected by
powerd. Errata AA21 here:

ftp://download.intel.com/design/PentiumXE/specupdt/31030717.pdf

MFC after: 2 weeks

Do not use PowerNow! if FID or VID is missing.

Use power management information for AMD CPUs from identcpu.c.

Add a proper detach method to the est(4) driver using cpufreq_unregister().

MFC after: 1 week

Fail detach if cpufreq_unregister() fails.

MFC after: 1 week

Disable the code to generate a simple table from the status MSR by default.
This can be enabled by setting the 'hw.est.msr_info' tunable to 1.

If we are unable to obtain a frequency list from either ACPI or the static
tables, then attempt to build a simple list containing just the high and
low frequencies based on the current CPU frequency calculated during boot
and the contents of the MSR.

MFC after: 1 month

Attach the cpufreq child devices with specific orders to enforce relative
priority of some of the drivers that manage the same state (e.g. ichss0
vs est0). Specifically, powernow, est, and p4tcc are added at order 10,
ichss at order 20, and smist at order 30. Previously, some laptops were
seeing both ichss0 and est0 attaching and stomping on each other.

XXX: This isn't quite ideal, but works with the existing hacks, I think
what we really want instead is a single "speedstep0" device for CPUs
that the ichss, est, and smist drivers probe (but with differing
priorities).

MFC after: 1 week

After probing the available frequency settings, restore the CPU to run at
whatever frequency it started at instead of always picking the highest
frequency. The first version of this driver attempted to do this, but it
set the speed to the first frequency in the list rather than the value it
had saved.

MFC after: 1 week
Discussed with: rpaulo, phk

Remove unused variable saved_id16.

Pointy hat to: me
Pointed out by: jhb
MFC after: 1 week

A cautionary XXX comment about seemingly bogus errata checks.

Increase time we wait for things to settle to 1 millisecond,
10 microseconds is too short.

Always set the cpu to the highest frequency so that we get through
boot and don't handicap cpus where powerd(8) is not used.

Revert last commit and stop committing before morning tea.

Increase time we wait for things to settle to 1 millisecond,
10 microseconds is too short.

Always set the cpu to the highest frequency so that we get through
boot and don't handicap cpus where powerd(8) is not used.

Use correct bitmask for identifying chip family.

- Don't execute cpuid to fetch the features. We already have the features
present in cpu_feature2. Also, use CPUID2_EST rather than a magic
number.
- Don't free the ACPI settings list in detach if we are going to fail the
request. Otherwise an attempt to kldunload est would free the array
but the driver would keep trying to use it.

MFC after: 1 week

In est_acpi_info(), initialize count before passing its pointer to
CPUFREQ_DRV_SETTINGS(). The value of count on input is used to
prefent overflow of the settings buffer passed into CPUFREQ_DRV_SETTINGS().

This corrects the "est: CPU supports Enhanced Speedstep, but is not recognized."
error on my system.

MFC after: 1 week

Validate the id16 values gathered from ACPI (previously a TODO item).
Style changes by me and njl.

Approved by: njl (mentor)
Reviewed by: njl (mentor)
Submitted by: Takeharu KATO <takeharu1219 at ybb.ne.jp>
PR: 119350
MFC after: 1 week

Use bus_dma to get a page in the first 4 GB. Since the physical address
of the magic string is passed in a 32-bit register, we can't use high
memory in the PAE case. This also eliminates a use of vtophys().

Tested by: Jeff Shimbo <jts767 / gmail.com>
MFC after: 1 week

o introduce a flags 'errata' for HW bugs onto the softc.
o remove errata_a0 and introduce the corresponding flags into 'errata'.
o introduce a new errata for K8, namely some platform might set the
PENDING_BIT but aren't able to unset it, also don't loop forever
waiting PENDING_BIT being cleared.
o try to introduce a workaround for the PENDING_BIT stuck problem,
o support now half multipliers for K8.

Tested by: Abdullah Al-Marrie

Approved by: njl

Since DELAY() was moved, most <machine/clock.h> #includes have been
unnecessary.

Add support for the VIA C7-M processor family.

Remove an unnecessary check of the table's bus clock. CPUs that
support this feature export only the high/low settings via the MSR,
packed into 32 bits.

Hardware from: Centaur Technologies
MFC after: 1 week

Add frequency-voltage tables for Intel 778, 758, 773, 753, and 733J
processors.

Obtained from: Intel Datasheet 302189-008

Like acpi_throttle, set frequency to 100% in attach. Some BIOSen may set
this value lower, making the system quite slow after booting.

Print cpu_vendor and the MSR value if we don't support this processor
even though we're not asking people to contact us.

Requested by: njl

Remove the instruction to "contact the maintainer" for unrecognized
CPUs. Intel refuses to give me the information I need, and getting
more emails about this doesn't help.

Add a driver for SMI-based SpeedStep. The hardware supports two frequency
settings and is an older version of the same design used for ICH SpeedStep.
It is only known to be available on PIIX4 chipsets.

Many thanks to Bruno Ducrot for writing the driver and Jon Noack for
testing.

Submitted by: Bruno Ducrot

Properly terminate the table generated from ACPI info. The cpufreq
settings are length-counted while the EST table is null-terminated.
This fixes extra garbage states being reported with ACPI probing.

Advertise p4tcc via acpi_get_features() _PDC support.

Add support for _PDC/_OSC by advertising that we support direct access to
the PERF_CTL/STS MSRs via the new acpi_get_features() method. This should
allow newer systems to use SpeedStep.

Remove check of numpst to allow more K8 variants to attach. The other
checks, including cpuid_is_k7(), will catch CPUs that really don't support
this method.

Submitted by: Bruno Ducrot
Tested by: Jari Kirma (kirma cs.hut.fi)

Add the powernow driver, which handles AMD Mobile Athlon PowerNow! (k7)
and AMD Cool&Quiet PowerNow! (k8) cpufreq control. This driver is enabled
for both i386 and amd64 architectures. It has both acpi and legacy BIOS
attachments. Thanks to Bruno Ducrot for writing this driver and Jung-uk
Kim for testing.

Submitted by: Bruno Ducrot (ducrot:poupinou.org)

Add support for probing EST settings from ACPI. This should handle more
modern CPUs that have multiple VID#s that aren't detectable via public
methods. We use the control value from acpi_perf as the id16 for setting
a given frequency.

Make a pass through all drivers checking specs for desired behavior on
SMP systems. It appears all drivers except ichss should attach to each
CPU and that settings should be performed on each CPU. Add comments about
this. Also, add a guard for p4tcc's identify method being called more than
once.

Correct an off-by-one error in the number of settings est announces.
The extraneous "0" state was not fatal but useless.

Import a rewrite of p4tcc for the cpufreq(4) framework. This includes
a bugfix of clearing the On-Demand flag when going back to 100%. It
has been tested and works on an IBM R32. Note original work done by
Ted Unangst and sobomax@.

Support disabling individual cpufreq drivers with hints, e.g.,
hint.ichss.0.disabled="1"

Add the Enhanced SpeedStep driver (EST). Currently, this driver only works
on the previous generation of Pentium-M processors (Banias). Support for
Dothan and later processors involves working with acpi_perf(4) to extract
information about supported states. This driver should work on MP systems
including HTT. It is experimental and may have a few bugs but has been
tested to not crash at least.

Thanks to Colin Percival for his initial work on this driver.

Fix the problem with incorrect throttling level reported immediately after
reboot. Safter the reboot the TCC is usually in the Automatic mode, in which
reading current performance level is likely to produce bogus results make sure
to switch it to the On-Demand mode and set to some known performance level.
Unfortunately there is no reliable way to check that TCC is in the Automatic
mode. Reading bit 4 of ACPI Thermal Monitor Control Register produces 0
regardless of the current mode.

MFC after: 1 week

Start all license/copyright notice comments with /*-, per tradition

o Fix whitespace bug introduced in the previous commit.

Submitted by: ru

o Simplify p4tcc_power_profile().

Submitted by: maxim

o Extend boot output: print out mimimum/maximum performance value and number
of performance steps available;

o similarly to Enhanced SpeedStep driver, export list of all available steps
via hw.p4tcc.cpuperf_levels sysctl.

o Typo: Ternal -> Thermal.

- Move performance-controlling sysctls into hw.p4tcc.* tree;

Suggested by: nate

- get rid of "magick" values in code and make sysctl's reflecting reality
on processor versions which have one or another frequency "forbidden"
due to errata.

MFC after: 2 weeks

Add new CPU_ENABLE_TCC option, from NOTES:

CPU_ENABLE_TCC enables Thermal Control Circuitry (TCC) found in some
Pentium(tm) 4 and (possibly) later CPUs. When enabled and detected,
TCC allows to restrict power consumption by using machdep.cpuperf*
sysctls. This operates independently of SpeedStep and is useful on
systems where other mechanisms such as apm(4) or acpi(4) don't work.

Given the fact that many, even modern, notebooks don't work properly
with Intel ACPI, this is indeed very useful option for notebook owners.

Obtained from: OpenBSD
MFC after: 2 weeks