XHCI: Properly handle "Length invalid" and erroring TRBs on the endpoint ring.

Should improve #18432 and other tickets.

Change-Id: Iaafe2d9d61bc0514e4dd6283b9e75496d5e2d44a
Reviewed-on: https://review.haiku-os.org/c/haiku/+/7341
Tested-by: Commit checker robot <no-reply+buildbot@haiku-os.org>
Reviewed-by: waddlesplash <waddlesplash@gmail.com>

xhci: report link status for USB 3 ports

The values come from XHCI table 5-27.

PS_PLS_MASK is the same as PORT_STATUS_SS_LINK_STATE.

Change-Id: Ie54abcda0c33738fe7dc5caa4e4f3abae3b40adc
Reviewed-on: https://review.haiku-os.org/c/haiku/+/6488
Tested-by: Commit checker robot <no-reply+buildbot@haiku-os.org>
Reviewed-by: waddlesplash <waddlesplash@gmail.com>

XHCI: Fix endpoint flags bits macros.

There can be up to 32 endpoints, so the correct mask is 0x1f.
Checked against FreeBSD.

xhci: Check endpoint state before stop

* As per the spec, the only exit from an endpoint in a halt
state is to issue a restart command. (restart puts the endpoint
into a stop state.. it's named poorly)

Change-Id: I4a1a556374ad7f564a562d995d1a2e29230a50c3
Reviewed-on: https://review.haiku-os.org/c/haiku/+/3959
Reviewed-by: Alex von Gluck IV <kallisti5@unixzen.com>

XHCI: Rename ERST_EHB to ERDP_BUSY for clarity's sake.

No functional change intended.

XHCI: Remove unused and unnecessary data structure definitions.

XHCI: Implement isochronous transfers support (!!).

Only out-bound isochronous transfers are tested right now, but in
theory inbound ones "should" work just fine, too. The frame calculation
has a bunch of TODOs around it, because I'm probably not doing it
entirely correctly...

But hey, I made this commit while listening to some glitchy (audio
only in left channel, strange pops occurring intermittently, but
otherwise it sounds 100% correct) music over my USB audio device!
(I'm willing to blame the above problems on our rather buggy
usb_audio driver.)

XHCI: Overhaul endpoint ring logic.

The changes in this commit are large but also subtle, so
an explanation of how they came to be seems to make sense:

Earlier today I tried booting Haiku on QEMU under USB3 ... and
discovered that it didn't work, with the "Operation timed out"
message from in usb_disk upon failing a data transfer. Indeed,
turning on tracing showed no event was being posted for the
transfer. So, I downloaded QEMU's source code, turned on XHCI
debugging, and began tracing what was going on.

Eventually I determined by adding more and more printfs into QEMU's
XHCI implementation that what was occuring was that it was evaluating
a Link TRB, hitting an empty TRB, and then deciding that the TD (aka.,
the "run" of TRBs) was not ready to be consumed, and bailing.

In fact, that very condition (a link TRB leading to an empty TRB)
is precisely what _LinkDescriptorForPipe did before this commit.
We allocate only a small (8 before this commit, 16 + 1 after this
commit) TRB ring for each *endpoint*, and larger ones on a per-*transfer*
basis; and just write Link TRBs onto the Endpoint ring pointing to
the transfer TD, and then one at the end of each transfer TD leading
back to the endpoint ring.

The reason this occured inside usb_disk, and not earlier (e.g.
during descriptor fetching), is that QEMU has special logic around
determining transfer lengths of control transfers which made it
not perform the "TRB valid?" check after evaluating the Link TRB.
So, being implementation-defined behavior, I am guessing that
this same problem was also the cause of boot failures on real
hardware.

This also means that the problem was essentially a race condition,
as if we posted another transfer to the ring before it evaluated
the TRB, it would always work.

The solution of course is to put some valid TRB at the end of
every transfer on the Endpoint ring. A "no-op" would have done
the job (well, maybe not, it appears QEMU does not implement "no-op"
TRBs for some reason), but there was another feature of XHCI
that we did not take advantage of: Event Data TRBs. These provide
the "total transferred length" as well as the status, instead of
the "remaining length" of the final TRB.

This of course required refactoring the use of the CHAIN bit
and the IOC bit (namely, more or less all TRBs save the Event Data
get the CHAIN bit set, and none save Event Data get the IOC bit.)

There was also an update to the XHCI spec since I've last committed here,
so the new comments are in reference to the "XHCI 1.2" spec. (I'll
eventually find time to update the old ones.)

Fixes booting from USB3 on QEMU, and most likely also on
bare metal, where it at least got to usb_disk (it does
not seem to fix the case where usb_disk does not even start.) Whew!

XHCI: Rename fields of xhci_trb to match their actual purposes.

FreeBSD has the names we used before this commit; OpenBSD has names
more similar to the ones used in this commit (though following their
conventions of course.)

This makes the code significantly more readable.

XHCI: Do all endianness conversion of TRBs in LinkDescriptorForPipe.

XHCI: Miscellaneous tweaks and cleanups.

* Check "endpoint == NULL" instead of the ID for this pipe; as
_InsertEndpointForPipe will have already done the check for us.
* Store the endpoint's ID (our internal ID) in the structure, and
then move the doorbell-ring into _LinkDescriptorForPipe. Now all
variables named "id" are actually that, and not the endpoint number
(which is "id + 1".)
* Return actual statuses in NotifyPipeChange, among other tweaks.
* All things which set the usb_request_data structure also set data,
so we can just use ReadDescriptor to do copies like normal, simplifying
the finish-transfers code.

No (major) functional change intended.

XHCI: Completely rewrite Transfer Descriptor and related logic.

This is essentially a complete rewrite of the way TDs work, and a
significant change to how "normal" requests are submitted. In summation:
* There is now no such thing as a "descriptor chain". This was a concept
mostly carried over from EHCI and previous controllers that does not
really map to XHCI properly, as one "transfer descriptor" can contain
as many TRBs as are necessary to complete the transfer.

What we were really doing before was setting up multiple TRB sequences
owned by our xhci_td structure, and then linking them together with the
CHAIN bit set on each TRB, which in XHCI terms is *one* descriptor,
not multiple ones as we had it, and we were allocating the same amount
of memory anyway, so we might as well do it all in one structure.

So now xhci_td does not have a fixed size of TRBs, but rather a dynamic
one, and thus also a dynamic number of buffers. As part of this refactor,
xhci_td is now a data structure for our own use only, and so it is allocated
from the normal heap instead of as physical memory, and TRBs separately.
* Removing the distinction between "descriptor" and "descriptor chain" greatly
simplifies quite a lot of logic related to transfer handling, especially
in WriteDescriptor and ReadDescriptor, which no longer need to handle
writing to buffers across xhci_td boundaries.
* There is now a proper split between "trb_count" and "trb_used". The former
is the actual number of TRBs allocated in the data structure; the latter
is the number presently used in said data structure. This clarifies
quite a lot of code.

As part of this refactor, a number of other related issues were also cleaned up:
* TRB buffer sizes are now hard-coded to be 4x the Max Packet Size for the
given pipe. Previously they were 64KB, which is much larger than the spec
suggests. As we now size them exactly this way, we can set the endpoint's
Average TRB Length to this value also, which allows the controller to
better schedule transfers for us. This logic can probably be cleaned up
further in the future, even.
* We now write the "TD Size" field of Normal transfers (i.e. packet count,
not TRB count) properly, as it is now much easier to compute based on
our new TRB sizing logic.
* We now record the Max Packet Size for the endpoint in the endpoint
structure. (This will probably be more useful when we are dealing
with isochronous transfers later on.)
* Write the last cycle bit in _LinkDescriptorForPipe after everything else
has been written. This probably does not affect anything too seriously,
but it is technically more correct.
* Added section & page references to the specification in various comments.
* Added more error checking where applicable.

Some things still to be done that I noticed while working on this change:
* When we use a single large buffer and manually segment it rather than
calling AllocateChunk() for each one, there is a massive performance
gain (I saw 50MB/s -> 95MB/s in testing on some USB devices.) However,
we can't do this unconditionally, as the stack doesn't allocate physical
chunks larger than 32 * B_PAGE_SIZE, which is a problem for some filesystems
which can read/write in chunks of 1MB. I'll implement this in a later commit.
* Setting the IOC bit on the last TRB in a transfer is acceptable to find
out when the transfer is finished, but it isn't enough to get a proper
tally of how much data was transfered. This is what Event Status TRBs
are for; we should implement them.
* _LinkDescriptorForPipe has an overflow condition. I'll fix that in the
next commit.

Tested on a ThinkPad E550 (Intel Broadwell) with usb_disk and usb_hid.

xhci: the command ring must be stopped when setting its address.

* send stop and cancel commands.

Signed-off-by: Jérôme Duval <jerome.duval@gmail.com>

xhci: add some capability definitions.

* scratchpad buffer shouldn't cross the page boundary,
the maximum with the rest bytes in the page is 2048 / 8.

XHCI: fix tracing

- The XHCI version register is at offset 2, but we can't do a 32-bit
read at that address. Instead read at address 0 and shift. Fix a freeze
on my system (this register is currently used only for tracing, but I
noticed my USB controller implements version 0.96 and not 1.0 of the
spec, so maybe this will have to change). All other registers in the
capability area are already properly aligned.
- Usual printf type mismatch error in xhci.cpp.

xhci: Clean up command register defines

* Ensure their naming match Intel xhci datasheets

xhci: Check and report interface version

* We could also check to make sure it isn't 0xffff and
bail if it is... not really a solution but helps
prevent kdl's if our mapped memory is corrupted.
* That seems like more of a hack though, i'd like to
solve the real problem.

xhci: 1.1 spec actually specifies 1023 maximum scratch pads.

* While my test XHCI machine only requested 36, the spec for 1.1
actually states the controller can now "request up to 1023"

xhci: 1.1 spec bumped scratchpads to 36. Set to sane 48 max

xhci: Tab cleanup of hw defines. No functional change

XHCI: Even more fixes.

* Fix Endpoint Context Initialisation (Refer xHCI v1.1 - 6.2.3)
* Fix Interval Calculation (Refer xHCI v1.1 - 6.2.3.6 , USB 2.0 - 9.6.6 page 271)
* Fix MaxBurst, MaxPacketSize Calculation (Refer xHCI v1.1 - 6.2.3.5, USB 2.0 - 9.6.6 page 271)
* Fix MaxESITPayload Calculation (Refer xHCI v1.1 - 4.14.2)
* Remove Link TRBs as they were never being used
* Increase Number of TRBs per endpoint (to utilise the whole area allocated for Device TRBs)
* Fix usage of XHCI_MAX_ENDPOINTS (most of the checks were failing at corner cases)
* Some coding style fixes.

Signed-off-by: Augustin Cavalier <waddlesplash@gmail.com>
Before this patch, writes to USB disks on XHCI in VirtualBox (which emulates
an Intel C210) stalled or failed. After this patch, they apparently work,
although I got mixed results - a BFS disk seemed to work perfectly, a FAT32
one also seemed to work OK but after a reboot there was data corruption. USB
mouse is still as busted as ever.

busses/usb: Trim trailing whitespace.

XHCI USB: Add port routing for Intel WildcatPoint LP USB.

* from FreeBSD.

XHCI USB: Add port routing for Intel WildcaPoint USB device.

* from FreeBSD.

XHCI USB: Add port routing for Intel BayTrail USB device.

* from FreeBSD.

XHCI USB: Some more fixes.

* Fix device speed identification.
* Fix Max Packet Size for Full-Speed devices.
* Fix IRQ rate.
* Update slot context for LS/FS devices connected to non-root HS hub.
* Fix typo.

Signed-off-by: Jérôme Duval <jerome.duval@gmail.com>

XHCI USB: Fixes.

* Add support for hubs in AllocateDevice().
* Prevent page fault in FinishTransfers().
* Set fCapabilityLength
* Correct in BIOS ownership code
* Fix context errors in _InsertEndpointForPipe().
* Update constants according to latest Specification (v1.1)
* Fix SMI code (reference
http://lkml.iu.edu/hypermail/linux/kernel/1204.2/02460.html).
* Fix Memory/Device-Slot leaks.
* Fix area allocation for TRBs.
* Fix for Intel Lynx Point and Panther Point chipsets. Also move init
of xhci before ehci, to switch USB 2.0 ports before the ehci module
discovers them.

Signed-off-by: Jérôme Duval <jerome.duval@gmail.com>

Work in progress on xHCI bus driver

* added a thread to handle events, locking wasn't easy in an interrupt
handler
* the td struct can now track several buffers instead of just one.
* use Transfer::Data*() instead of Vector*() for the time being
until support for fragmented transfers is done
* added CreateDescriptorChain, WriteDescriptorChain and ReadDescriptorChain,
chained tds not working yet though.
* added a mutex lock per enabled endpoint, lock when touching the endpoint
transfer ring.
* correctly configure interval and average trb length for endpoint contexts.
* interrupt transfers seem to work on real hardware
* xhci qemu driver doesn't advance ring dequeue pointers on link trbs, thus
accessing freed trbs that could already be reused, leading to crash.

Work in progress on xHCI bus driver, testing on emulation.

* Initial testing with QEmu emulation works with interrupt/bulk,
testing on real hardware still not working. Small bulk requests work
OK, not bigger ones (for instance it can't load the MediaPlayer
application at 800KB).
* QEmu doesn't support byte read in the MMIO space, switch to double word.
* added SubmitNormalRequest() for interrupt and bulk endpoints.
* AllocateDevice(): fixed the physical address of the link TRB pointing
to the beginning of the ring.
* _InsertEndpointForPipe(): setup context for non-default endpoints
on creation. This should get the device to the configured state.
* _UnlinkDescriptorForPipe(): the next pointer wasn't null after unlinking.
* HandleTransferComplete(): add descriptor to a list to be processed by the
finisher thread.
* FinishThread(): process finished transfers and call the callback,
then free the TRB.

Work in progress on xHCI bus driver.

* added more traces, coding style
* added xhci_td, xhci_endpoint structs.
* SubmitRequest: this links a 3-stage TD control request into
the endpoint ring.
* _InsertEndpointForPipe: when a pipe is created, configure the
endpoint and set up the controller cookie.
* CreateDescriptor and FreeDescriptor: allocates memory for a TD.
* _LinkDescriptorForPipe: link a TD in an endpoint ring.
* _UnlinkDescriptorForPipe: unlink a TD
* AllocateDevice:
slot number zero isn't valid.
stick the slot number to the device struct.
configure the default endpoint before calling SetAddress.
address is found in the device context instead of the input context.
device address passed to the USB stack has to begin with 1.
* Ring:
check for invalid slot/endpoint combination.
ring the slot doorbell instead of the doorbell 0.
* Root hub: fixed USB version.
* HandleTransferComplete: finds out the TD which submitted a
transfer.

Work in progress on XHCI bus driver.

* USB Core:
Hub: pass more correct speed, port and address for USB3 devices.
Transfer: doesn't care for bandwidth computation for super speed.
BusManager: Make AllocateDevice() and FreeDevice() virtual, XHCI needs to setup
a slot and an address for the device.
Add USB_SPEED_WIRELESS usb_speed definition.
Add USB2.0 erratum definition and some USB3 definitions
* XHCI:
Find outs which port supports SuperSpeed or not.
Override AllocateDevice() to configure slot and address for a connected device.
Override FreeDevice() to free ressources associated to a slot.
Add context struct definitions.

XHCI improvements

* remove the xhci cmd complete thread
* creation of the scratchpad area
* wait for running status after starting the controller
* increase delays on controller reset (250ms instead of 100ms)
* use of SpinLocker from AutoLock.h
* add a DoCommand() method to queue a command, ring and wait for the event
response
* add a method for each command
* XHCI_PORTSC macro was off 1 port
* add definitions for TRB types, Completion Codes.

Patch by Jian Chiang as part of his GSoc Project (coding style fixes by myself):
* xhci controller start operation
* command ring and event ring initialization
* No-Op Command test and real xhci irq handle
* xhci root hub support
* add Super Speed enumeration and xhci_rh.cpp into jamfile


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@42511 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added skeleton XHCI driver by Jian Chiang, based on the specs and the EHCI driver.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@41142 a95241bf-73f2-0310-859d-f6bbb57e9c96

XHCI USB: Add port routing for Intel WildcaPoint USB device.

* from FreeBSD.

XHCI USB: Add port routing for Intel BayTrail USB device.

* from FreeBSD.

XHCI USB: Some more fixes.

* Fix device speed identification.
* Fix Max Packet Size for Full-Speed devices.
* Fix IRQ rate.
* Update slot context for LS/FS devices connected to non-root HS hub.
* Fix typo.

Signed-off-by: Jérôme Duval <jerome.duval@gmail.com>

XHCI USB: Fixes.

* Add support for hubs in AllocateDevice().
* Prevent page fault in FinishTransfers().
* Set fCapabilityLength
* Correct in BIOS ownership code
* Fix context errors in _InsertEndpointForPipe().
* Update constants according to latest Specification (v1.1)
* Fix SMI code (reference
http://lkml.iu.edu/hypermail/linux/kernel/1204.2/02460.html).
* Fix Memory/Device-Slot leaks.
* Fix area allocation for TRBs.
* Fix for Intel Lynx Point and Panther Point chipsets. Also move init
of xhci before ehci, to switch USB 2.0 ports before the ehci module
discovers them.

Signed-off-by: Jérôme Duval <jerome.duval@gmail.com>

Work in progress on xHCI bus driver

* added a thread to handle events, locking wasn't easy in an interrupt
handler
* the td struct can now track several buffers instead of just one.
* use Transfer::Data*() instead of Vector*() for the time being
until support for fragmented transfers is done
* added CreateDescriptorChain, WriteDescriptorChain and ReadDescriptorChain,
chained tds not working yet though.
* added a mutex lock per enabled endpoint, lock when touching the endpoint
transfer ring.
* correctly configure interval and average trb length for endpoint contexts.
* interrupt transfers seem to work on real hardware
* xhci qemu driver doesn't advance ring dequeue pointers on link trbs, thus
accessing freed trbs that could already be reused, leading to crash.

Work in progress on xHCI bus driver, testing on emulation.

* Initial testing with QEmu emulation works with interrupt/bulk,
testing on real hardware still not working. Small bulk requests work
OK, not bigger ones (for instance it can't load the MediaPlayer
application at 800KB).
* QEmu doesn't support byte read in the MMIO space, switch to double word.
* added SubmitNormalRequest() for interrupt and bulk endpoints.
* AllocateDevice(): fixed the physical address of the link TRB pointing
to the beginning of the ring.
* _InsertEndpointForPipe(): setup context for non-default endpoints
on creation. This should get the device to the configured state.
* _UnlinkDescriptorForPipe(): the next pointer wasn't null after unlinking.
* HandleTransferComplete(): add descriptor to a list to be processed by the
finisher thread.
* FinishThread(): process finished transfers and call the callback,
then free the TRB.

Work in progress on xHCI bus driver.

* added more traces, coding style
* added xhci_td, xhci_endpoint structs.
* SubmitRequest: this links a 3-stage TD control request into
the endpoint ring.
* _InsertEndpointForPipe: when a pipe is created, configure the
endpoint and set up the controller cookie.
* CreateDescriptor and FreeDescriptor: allocates memory for a TD.
* _LinkDescriptorForPipe: link a TD in an endpoint ring.
* _UnlinkDescriptorForPipe: unlink a TD
* AllocateDevice:
slot number zero isn't valid.
stick the slot number to the device struct.
configure the default endpoint before calling SetAddress.
address is found in the device context instead of the input context.
device address passed to the USB stack has to begin with 1.
* Ring:
check for invalid slot/endpoint combination.
ring the slot doorbell instead of the doorbell 0.
* Root hub: fixed USB version.
* HandleTransferComplete: finds out the TD which submitted a
transfer.

Work in progress on XHCI bus driver.

* USB Core:
Hub: pass more correct speed, port and address for USB3 devices.
Transfer: doesn't care for bandwidth computation for super speed.
BusManager: Make AllocateDevice() and FreeDevice() virtual, XHCI needs to setup
a slot and an address for the device.
Add USB_SPEED_WIRELESS usb_speed definition.
Add USB2.0 erratum definition and some USB3 definitions
* XHCI:
Find outs which port supports SuperSpeed or not.
Override AllocateDevice() to configure slot and address for a connected device.
Override FreeDevice() to free ressources associated to a slot.
Add context struct definitions.

XHCI improvements

* remove the xhci cmd complete thread
* creation of the scratchpad area
* wait for running status after starting the controller
* increase delays on controller reset (250ms instead of 100ms)
* use of SpinLocker from AutoLock.h
* add a DoCommand() method to queue a command, ring and wait for the event
response
* add a method for each command
* XHCI_PORTSC macro was off 1 port
* add definitions for TRB types, Completion Codes.

Patch by Jian Chiang as part of his GSoc Project (coding style fixes by myself):
* xhci controller start operation
* command ring and event ring initialization
* No-Op Command test and real xhci irq handle
* xhci root hub support
* add Super Speed enumeration and xhci_rh.cpp into jamfile


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@42511 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added skeleton XHCI driver by Jian Chiang, based on the specs and the EHCI driver.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@41142 a95241bf-73f2-0310-859d-f6bbb57e9c96