133965Sjdp.. SPDX-License-Identifier: GPL-2.0 2218822Sdim.. include:: <isonum.txt> 338889Sjdp 433965Sjdp=========================================== 533965SjdpThe PCI Express Port Bus Driver Guide HOWTO 633965Sjdp=========================================== 733965Sjdp 833965Sjdp:Author: Tom L Nguyen tom.l.nguyen@intel.com 11/03/2004 933965Sjdp:Copyright: |copy| 2004 Intel Corporation 1033965Sjdp 1133965SjdpAbout this guide 1233965Sjdp================ 1333965Sjdp 1433965SjdpThis guide describes the basics of the PCI Express Port Bus driver 1533965Sjdpand provides information on how to enable the service drivers to 1633965Sjdpregister/unregister with the PCI Express Port Bus Driver. 1733965Sjdp 1833965Sjdp 19218822SdimWhat is the PCI Express Port Bus Driver 20218822Sdim======================================= 2133965Sjdp 2233965SjdpA PCI Express Port is a logical PCI-PCI Bridge structure. There 2333965Sjdpare two types of PCI Express Port: the Root Port and the Switch 2433965SjdpPort. The Root Port originates a PCI Express link from a PCI Express 2533965SjdpRoot Complex and the Switch Port connects PCI Express links to 2633965Sjdpinternal logical PCI buses. The Switch Port, which has its secondary 2733965Sjdpbus representing the switch's internal routing logic, is called the 2877298Sobrienswitch's Upstream Port. The switch's Downstream Port is bridging from 2933965Sjdpswitch's internal routing bus to a bus representing the downstream 30218822SdimPCI Express link from the PCI Express Switch. 31218822Sdim 32218822SdimA PCI Express Port can provide up to four distinct functions, 33218822Sdimreferred to in this document as services, depending on its port type. 34218822SdimPCI Express Port's services include native hotplug support (HP), 3533965Sjdppower management event support (PME), advanced error reporting 3677298Sobriensupport (AER), and virtual channel support (VC). These services may 37218822Sdimbe handled by a single complex driver or be individually distributed 3833965Sjdpand handled by corresponding service drivers. 3978828Sobrien 40104834SobrienWhy use the PCI Express Port Bus Driver? 4178828Sobrien======================================== 4278828Sobrien 4333965SjdpIn existing Linux kernels, the Linux Device Driver Model allows a 4433965Sjdpphysical device to be handled by only a single driver. The PCI 4533965SjdpExpress Port is a PCI-PCI Bridge device with multiple distinct 4633965Sjdpservices. To maintain a clean and simple solution each service 4733965Sjdpmay have its own software service driver. In this case several 4833965Sjdpservice drivers will compete for a single PCI-PCI Bridge device. 4933965SjdpFor example, if the PCI Express Root Port native hotplug service 5033965Sjdpdriver is loaded first, it claims a PCI-PCI Bridge Root Port. The 5133965Sjdpkernel therefore does not load other service drivers for that Root 5233965SjdpPort. In other words, it is impossible to have multiple service 5333965Sjdpdrivers load and run on a PCI-PCI Bridge device simultaneously 5433965Sjdpusing the current driver model. 5533965Sjdp 5633965SjdpTo enable multiple service drivers running simultaneously requires 5733965Sjdphaving a PCI Express Port Bus driver, which manages all populated 5833965SjdpPCI Express Ports and distributes all provided service requests 5933965Sjdpto the corresponding service drivers as required. Some key 6033965Sjdpadvantages of using the PCI Express Port Bus driver are listed below: 6133965Sjdp 6233965Sjdp - Allow multiple service drivers to run simultaneously on 6333965Sjdp a PCI-PCI Bridge Port device. 6433965Sjdp 6533965Sjdp - Allow service drivers implemented in an independent 6633965Sjdp staged approach. 6733965Sjdp 6833965Sjdp - Allow one service driver to run on multiple PCI-PCI Bridge 6933965Sjdp Port devices. 7033965Sjdp 7133965Sjdp - Manage and distribute resources of a PCI-PCI Bridge Port 7233965Sjdp device to requested service drivers. 7333965Sjdp 7433965SjdpConfiguring the PCI Express Port Bus Driver vs. Service Drivers 7533965Sjdp=============================================================== 7633965Sjdp 7733965SjdpIncluding the PCI Express Port Bus Driver Support into the Kernel 7833965Sjdp----------------------------------------------------------------- 7933965Sjdp 8033965SjdpIncluding the PCI Express Port Bus driver depends on whether the PCI 8133965SjdpExpress support is included in the kernel config. The kernel will 8233965Sjdpautomatically include the PCI Express Port Bus driver as a kernel 8333965Sjdpdriver when the PCI Express support is enabled in the kernel. 8433965Sjdp 85218822SdimEnabling Service Driver Support 8633965Sjdp------------------------------- 8733965Sjdp 8833965SjdpPCI device drivers are implemented based on Linux Device Driver Model. 8933965SjdpAll service drivers are PCI device drivers. As discussed above, it is 90218822Sdimimpossible to load any service driver once the kernel has loaded the 91218822SdimPCI Express Port Bus Driver. To meet the PCI Express Port Bus Driver 9233965SjdpModel requires some minimal changes on existing service drivers that 9333965Sjdpimposes no impact on the functionality of existing service drivers. 9433965Sjdp 9533965SjdpA service driver is required to use the two APIs shown below to 9633965Sjdpregister its service with the PCI Express Port Bus driver (see 9733965Sjdpsection 5.2.1 & 5.2.2). It is important that a service driver 9833965Sjdpinitializes the pcie_port_service_driver data structure, included in 9933965Sjdpheader file /include/linux/pcieport_if.h, before calling these APIs. 10033965SjdpFailure to do so will result an identity mismatch, which prevents 10133965Sjdpthe PCI Express Port Bus driver from loading a service driver. 10277298Sobrien 10377298Sobrienpcie_port_service_register 10477298Sobrien~~~~~~~~~~~~~~~~~~~~~~~~~~ 10533965Sjdp:: 10633965Sjdp 10733965Sjdp int pcie_port_service_register(struct pcie_port_service_driver *new) 10833965Sjdp 10933965SjdpThis API replaces the Linux Driver Model's pci_register_driver API. A 11077298Sobrienservice driver should always calls pcie_port_service_register at 11177298Sobrienmodule init. Note that after service driver being loaded, calls 11233965Sjdpsuch as pci_enable_device(dev) and pci_set_master(dev) are no longer 11377298Sobriennecessary since these calls are executed by the PCI Port Bus driver. 11433965Sjdp 11533965Sjdppcie_port_service_unregister 11677298Sobrien~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 11777298Sobrien:: 11833965Sjdp 119218822Sdim void pcie_port_service_unregister(struct pcie_port_service_driver *new) 12033965Sjdp 12133965Sjdppcie_port_service_unregister replaces the Linux Driver Model's 12233965Sjdppci_unregister_driver. It's always called by service driver when a 12333965Sjdpmodule exits. 12433965Sjdp 12533965SjdpSample Code 12633965Sjdp~~~~~~~~~~~ 12733965Sjdp 12833965SjdpBelow is sample service driver code to initialize the port service 12977298Sobriendriver data structure. 13077298Sobrien:: 13133965Sjdp 13233965Sjdp static struct pcie_port_service_id service_id[] = { { 13333965Sjdp .vendor = PCI_ANY_ID, 13433965Sjdp .device = PCI_ANY_ID, 13533965Sjdp .port_type = PCIE_RC_PORT, 13633965Sjdp .service_type = PCIE_PORT_SERVICE_AER, 13733965Sjdp }, { /* end: all zeroes */ } 13833965Sjdp }; 139218822Sdim 14033965Sjdp static struct pcie_port_service_driver root_aerdrv = { 14160484Sobrien .name = (char *)device_name, 14277298Sobrien .id_table = &service_id[0], 14377298Sobrien 14433965Sjdp .probe = aerdrv_load, 14533965Sjdp .remove = aerdrv_unload, 14633965Sjdp 14733965Sjdp .suspend = aerdrv_suspend, 14833965Sjdp .resume = aerdrv_resume, 14933965Sjdp }; 15033965Sjdp 15177298SobrienBelow is a sample code for registering/unregistering a service 15277298Sobriendriver. 15377298Sobrien:: 15433965Sjdp 15533965Sjdp static int __init aerdrv_service_init(void) 15633965Sjdp { 15733965Sjdp int retval = 0; 15833965Sjdp 15977298Sobrien retval = pcie_port_service_register(&root_aerdrv); 16077298Sobrien if (!retval) { 16133965Sjdp /* 162218822Sdim * FIX ME 163218822Sdim */ 164218822Sdim } 16533965Sjdp return retval; 16677298Sobrien } 16777298Sobrien 16833965Sjdp static void __exit aerdrv_service_exit(void) 16933965Sjdp { 17077298Sobrien pcie_port_service_unregister(&root_aerdrv); 17133965Sjdp } 17233965Sjdp 17333965Sjdp module_init(aerdrv_service_init); 17433965Sjdp module_exit(aerdrv_service_exit); 17533965Sjdp 17633965SjdpPossible Resource Conflicts 17733965Sjdp=========================== 17833965Sjdp 17933965SjdpSince all service drivers of a PCI-PCI Bridge Port device are 18033965Sjdpallowed to run simultaneously, below lists a few of possible resource 18133965Sjdpconflicts with proposed solutions. 18233965Sjdp 18333965SjdpMSI and MSI-X Vector Resource 18433965Sjdp----------------------------- 18533965Sjdp 18633965SjdpOnce MSI or MSI-X interrupts are enabled on a device, it stays in this 18733965Sjdpmode until they are disabled again. Since service drivers of the same 18833965SjdpPCI-PCI Bridge port share the same physical device, if an individual 18977298Sobrienservice driver enables or disables MSI/MSI-X mode it may result 19033965Sjdpunpredictable behavior. 19133965Sjdp 19233965SjdpTo avoid this situation all service drivers are not permitted to 19333965Sjdpswitch interrupt mode on its device. The PCI Express Port Bus driver 19433965Sjdpis responsible for determining the interrupt mode and this should be 19533965Sjdptransparent to service drivers. Service drivers need to know only 19633965Sjdpthe vector IRQ assigned to the field irq of struct pcie_device, which 19733965Sjdpis passed in when the PCI Express Port Bus driver probes each service 19833965Sjdpdriver. Service drivers should use (struct pcie_device*)dev->irq to 19933965Sjdpcall request_irq/free_irq. In addition, the interrupt mode is stored 20033965Sjdpin the field interrupt_mode of struct pcie_device. 20133965Sjdp 20233965SjdpPCI Memory/IO Mapped Regions 20333965Sjdp---------------------------- 20433965Sjdp 20533965SjdpService drivers for PCI Express Power Management (PME), Advanced 20633965SjdpError Reporting (AER), Hot-Plug (HP) and Virtual Channel (VC) access 20733965SjdpPCI configuration space on the PCI Express port. In all cases the 20833965Sjdpregisters accessed are independent of each other. This patch assumes 20933965Sjdpthat all service drivers will be well behaved and not overwrite 21033965Sjdpother service driver's configuration settings. 21133965Sjdp 21233965SjdpPCI Config Registers 21333965Sjdp-------------------- 21433965Sjdp 21533965SjdpEach service driver runs its PCI config operations on its own 21633965Sjdpcapability structure except the PCI Express capability structure, 21733965Sjdpthat is shared between many drivers including the service drivers. 21833965SjdpRMW Capability accessors (pcie_capability_clear_and_set_word(), 21933965Sjdppcie_capability_set_word(), and pcie_capability_clear_word()) protect 22033965Sjdpa selected set of PCI Express Capability Registers (Link Control 22133965SjdpRegister and Root Control Register). Any change to those registers 22233965Sjdpshould be performed using RMW accessors to avoid problems due to 22333965Sjdpconcurrent updates. For the up-to-date list of protected registers, 22433965Sjdpsee pcie_capability_clear_and_set_word(). 22533965Sjdp