1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
|
@page page_device_nvme_dm NVMe device model (DM)
# NVMe under `RT_USING_DM`
**`RT_USING_NVME`** builds the protocol core (**`nvme.c`**) and optionally **`nvme-pci.c`** (**`RT_NVME_PCI`**). Controllers register with **`rt_nvme_controller_register`**; each namespace becomes a **`rt_blk_disk`** for the block layer.
Controller API and queue ops: @ref page_device_nvme.
Sources: **`components/drivers/nvme/nvme.c`**, **`nvme-pci.c`**, **`components/drivers/include/drivers/nvme.h`**.
---
## Kconfig / build
| Option | Depends on | Role |
| --- | --- | --- |
| **`RT_USING_NVME`** | **`RT_USING_DM`**, **`RT_USING_BLK`**, **`RT_USING_DMA`** | Core stack |
| **`RT_USING_NVME_IO_QUEUE`** | **`RT_USING_NVME`** | Number of I/O queues (2/4/8 by priority profile) |
| **`RT_NVME_PCI`** | **`RT_USING_NVME`**, **`RT_USING_PCI`** | PCI function driver (default **y**) |
| **`SOC_DM_NVME_DIR`** | BSP | Extra non-PCI controllers |
**`RT_USING_NVME_QUEUE`** in **`nvme.h`** is **`1 + RT_USING_NVME_IO_QUEUE * RT_CPUS_NR`** (admin + per-CPU IO slots).
---
## PCI transport (`nvme-pci.c`)
### Matching
```c
static const struct rt_pci_device_id pci_nvme_ids[] = {
{ RT_PCI_DEVICE_ID(PCI_VENDOR_ID_REDHAT, 0x0010) },
{ RT_PCI_DEVICE_CLASS(PCIS_STORAGE_EXPRESS, ~0) },
};
```
Any **NVMe class** PCI function binds **`nvme-pci`** driver.
### Probe flow
```
pci_nvme_probe(pdev)
|
| rt_pci_iomap(pdev, 0) → nvme->regs
| ops = quirk->ops ?: pci_nvme_std_ops ("PCI")
| MSI-X: rt_pci_msix_enable → nvme->irqs[i]
| else: single pdev->irq, rt_pci_irq_unmask
| rt_pci_set_master(pdev)
v
rt_nvme_controller_register(nvme)
|
v
pdev->parent.user_data = pci_nvme
```
### Remove / shutdown
1. **`rt_nvme_controller_unregister`**
2. MSI-X disable or INTx mask
3. **`rt_pci_clear_master`**, **`rt_iounmap(regs)`**, free **`pci_nvme_controller`**
### Quirks
**`pdev->id->data`** may point to **`struct pci_nvme_quirk`** with alternate **`rt_nvme_ops`** (vendor MMIO). Default uses standard doorbell path in **`nvme.c`**.
---
## IRQ and NUMA affinity
| Mode | Behavior |
| --- | --- |
| **MSI-X** | Up to **`RT_USING_NVME_QUEUE`** vectors; linear index mapping |
| **INTx** | **`irqs_nr = 1`**, shared line — rely on **`complete_cmd`** to disambiguate if needed |
**`INIT_SECONDARY_CPU_EXPORT(nvme_queue_affinify_fixup)`** (in **`nvme.c`**):
- Runs when secondary CPUs start
- For each controller I/O queue index **`i ≡ cpuid (mod RT_CPUS_NR)`**, sets IRQ affinity to that CPU via **`rt_pic_irq_set_affinity`**
- Pairs with @ref page_device_numa when **`numa=on`** — keep queue IRQ near memory node
Assign **`nvme->irqs[]`** before **`rt_nvme_controller_register`** — core wires ISR per queue during setup.
---
## DMA
Queue rings allocated with **`rt_dma_alloc(nvme->dev, ...)`** and **`nvme_queue_dma_flags()`** (coherent / device attributes per **`dev`**).
Identify buffers use **`rt_malloc_align(..., nvme->page_size)`** after page size is negotiated from **`CAP.MPSMIN/MAX`**.
---
## Block device exposure
After Identify NS:
```c
ndev->parent.ops = &nvme_blk_ops;
ndev->parent.parallel_io = RT_TRUE;
rt_dm_dev_set_name(&ndev->parent.parent, "%sn%u", nvme->name, nsid);
rt_hw_blk_disk_register(&ndev->parent);
```
Partitions use standard **`RT_BLK_PARTITION_MAX`** — see disk/partitions docs if enabled in BSP.
---
## Non-PCI controllers
1. Map **`regs`**, implement **`struct rt_nvme_ops`** ( **`setup_queue`/`submit_cmd`/`complete_cmd`** as needed).
2. Fill **`irqs[]`** for admin + IO queues.
3. Set **`nvme->dev`** to the **`rt_device`** used for DMA mapping.
4. **`rt_nvme_controller_register`**.
5. Add Kconfig under **`SOC_DM_NVME_DIR`** and platform/PCIe-alt probe.
---
## Checklist (PCI BSP)
1. Enable **`RT_USING_DM`**, **`RT_USING_PCI`**, **`RT_USING_BLK`**, **`RT_USING_DMA`**, **`RT_USING_NVME`**, **`RT_NVME_PCI`**.
2. ECAM/host bridge probes before NVMe function.
3. Ensure **`rt_pci_set_master`** path runs (done in **`pci_nvme_probe`**).
4. Prefer **MSI-X** in firmware/DT if available.
5. For multi-node servers: **`numa=on`** + CPU/memory **`numa-node-id`** in DT (@ref page_device_numa).
6. Verify namespace appears as **`nvme0n1`** (etc.) and partition mount if used.
---
## Pitfalls
| Issue | Mitigation |
| --- | --- |
| **BAR not mapped** | **`rt_pci_iomap`** failure → probe abort |
| **MSI-X vector shortage** | Falls back to INTx — may need **`complete_cmd`** |
| **`CSTS == -1`** | Usually bad BAR/endianness — register rejects early |
| **Inverted `setup_queue` success** | @ref page_device_nvme — match **`nvme.c`** polarity |
| **Secondary CPU IRQ affinity** | Ensure **`nvme_queue_affinify_fixup`** runs (SMP BSP) |
---
## See also
- @ref page_device_nvme
- @ref page_device_pci
- @ref page_device_numa
- @ref page_device_dma
- `components/drivers/nvme/Kconfig`
|
