cxl: docs - add self-referencing cross-links

device to multiple heads which may be connected to one or more discrete

hosts.  An example of this would be a simple memory-pool which may be

statically configured (prior to boot) to expose portions of its memory

to Linux via the CEDT ACPI table.

to Linux via :doc:`CEDT <../platform/acpi/cedt>`.

MHMLD

~~~~~

There can be multiple switches under an RP. There can be multiple RPs under

a CXL Host Bridge (HB). There can be multiple HBs under a CXL Fixed Memory

Window Structure (CFMWS).

Window Structure (CFMWS) in the :doc:`CEDT <../platform/acpi/cedt>`.

An example hierarchy::

The next step is to take the min() of the per host bridge bandwidth and the

bandwidth from the Generic Port (GP). The bandwidths for the GP are retrieved

via ACPI tables SRAT/HMAT. The minimum bandwidth are aggregated under the same

ACPI0017 device to form a new xarray.

via ACPI tables (:doc:`SRAT <../platform/acpi/srat>` and

:doc:`HMAT <../platform/acpi/hmat>`). The minimum bandwidth are aggregated

under the same ACPI0017 device to form a new xarray.

Finally, the cxl_region_update_bandwidth() is called and the aggregated

bandwidth from all the members of the last xarray is updated for the

The Root contains links to:

* `Host Bridge Ports` defined by ACPI CEDT CHBS.

* `Host Bridge Ports` defined by CHBS in the :doc:`CEDT<../platform/acpi/cedt>`

* `Downstream Ports` typically connected to `Host Bridge Ports`.

* `Root Decoders` defined by ACPI CEDT CFMWS.

* `Root Decoders` defined by CFMWS the :doc:`CEDT<../platform/acpi/cedt>`

::

and may be one of many `logical devices` presented by a memory device. It

is still considered a type of `port` in the fabric.

An `endpoint` contains `endpoint decoders` available for use and the

*Coherent Device Attribute Table* (CDAT) used to describe the capabilities

of the device. ::

An `endpoint` contains `endpoint decoders` and the device's Coherent Device

Attribute Table (which describes the device's capabilities). ::

  # ls /sys/bus/cxl/devices/endpoint5

    CDAT        decoders_committed  modalias      uevent

Root Decoder

~~~~~~~~~~~~

A `Root Decoder` is logical construct of the physical address and interleave

configurations present in the ACPI CEDT CFMWS.  Linux presents this information

as a decoder present in the `CXL Root`.  We consider this a `Root Decoder`,

though technically it exists on the boundary of the CXL specification and

platform-specific CXL root implementations.

configurations present in the CFMWS field of the :doc:`CEDT

<../platform/acpi/cedt>`.

Linux presents this information as a decoder present in the `CXL Root`.  We

consider this a `Root Decoder`, though technically it exists on the boundary

of the CXL specification and platform-specific CXL root implementations.

Linux considers these logical decoders a type of `Routing Decoder`, and is the

first decoder in the CXL fabric to receive a memory access from the platform's

memory controllers.

`Root Decoders` are created during :code:`cxl_acpi_probe`.  One root decoder

is created per CFMWS entry in the ACPI CEDT.

is created per CFMWS entry in the :doc:`CEDT <../platform/acpi/cedt>`.

The :code:`target_list` parameter is filled by the CFMWS target fields. Targets

of a root decoder are `Host Bridges`, which means interleave done at the root

Such interleaves must be configured by the platform and described in the ACPI

CEDT CFMWS, as the target CXL host bridge UIDs in the CFMWS must match the CXL

host bridge UIDs in the ACPI CEDT CHBS and ACPI DSDT.

host bridge UIDs in the CHBS field of the :doc:`CEDT

<../platform/acpi/cedt>` and the UID field of CXL Host Bridges defined in

the :doc:`DSDT <../platform/acpi/dsdt>`.

Interleave settings in a root decoder describe how to interleave accesses among

the *immediate downstream targets*, not the entire interleave set.

At Root

~~~~~~~

Root decoder interleave is defined by the ACPI CEDT CFMWS.  The CEDT

may actually define multiple CFMWS configurations to describe the same

physical capacity - with the intent to allow users to decide at runtime

whether to online memory as interleaved or non-interleaved. ::

Root decoder interleave is defined by CFMWS field of the :doc:`CEDT

<../platform/acpi/cedt>`.  The CEDT may actually define multiple CFMWS

configurations to describe the same physical capacity, with the intent to allow

users to decide at runtime whether to online memory as interleaved or

non-interleaved. ::

             Subtable Type : 01 [CXL Fixed Memory Window Structure]

       Window base address : 0000000100000000

representations of the devices.

During Linux Early Boot stage (functions in the kernel that have the __init

decorator), the system takes the resources created by EFI/BIOS (ACPI tables)

and turns them into resources that the kernel can consume.

decorator), the system takes the resources created by EFI/BIOS

(:doc:`ACPI tables <../platform/acpi>`) and turns them into resources that the

kernel can consume.

BIOS, Build and Boot Options

NUMA Node Reservation

=====================

Linux refers to the proximity domains (:code:`PXM`) defined in the SRAT to

create NUMA nodes in :code:`acpi_numa_init`. Typically, there is a 1:1 relation

between :code:`PXM` and NUMA node IDs.

Linux refers to the proximity domains (:code:`PXM`) defined in the :doc:`SRAT

<../platform/acpi/srat>` to create NUMA nodes in :code:`acpi_numa_init`.

Typically, there is a 1:1 relation between :code:`PXM` and NUMA node IDs.

SRAT is the only ACPI defined way of defining Proximity Domains. Linux chooses

to, at most, map those 1:1 with NUMA nodes. CEDT adds a description of SPA

ranges which Linux may wish to map to one or more NUMA nodes.

The SRAT is the only ACPI defined way of defining Proximity Domains. Linux

chooses to, at most, map those 1:1 with NUMA nodes.

:doc:`CEDT <../platform/acpi/cedt>` adds a description of SPA ranges which

Linux may map to one or more NUMA nodes.

If there are CXL ranges in the CFMWS but not in SRAT, then a fake :code:`PXM`

is created (as of v6.15). In the future, Linux may reject CFMWS not described

The relevant code exists in: :code:`linux/drivers/acpi/numa/srat.c`.

See the Example Platform Configurations section for more information.

See :doc:`Example Platform Configurations <../platform/example-configs>`

for more info.

Memory Tiers Creation

=====================

  /sys/devices/virtual/memory_tiering/memory_tierN/nodelist

  0-1

If nodes are grouped which have clear difference in performance, check the HMAT

and CDAT information for the CXL nodes.  All nodes default to the DRAM tier,

unless HMAT/CDAT information is reported to the memory_tier component via

`access_coordinates`.

If nodes are grouped which have clear difference in performance, check the

:doc:`HMAT <../platform/acpi/hmat>` and CDAT information for the CXL nodes. All

nodes default to the DRAM tier, unless HMAT/CDAT information is reported to the

memory_tier component via `access_coordinates`.

For more, see :doc:`CXL access coordinates documentation

<../linux/access-coordinates>`.

Contiguous Memory Allocation

============================

Much of what this section is concerned with is ACPI Table production and

static memory map configuration. More detail on these tables can be found

under Platform Configuration -> ACPI Table Reference.

at :doc:`ACPI Tables <acpi>`.

.. note::

   Platform Vendors should read carefully, as this sections has recommendations

Interleave and Configuration Flexibility

----------------------------------------

If providing cross-host-bridge interleave, a CFMWS entry in the CEDT must be

presented with target host-bridges for the interleaved device sets (there may

be multiple behind each host bridge).

If providing cross-host-bridge interleave, a CFMWS entry in the :doc:`CEDT

<acpi/cedt>` must be presented with target host-bridges for the interleaved

device sets (there may be multiple behind each host bridge).

If providing intra-host-bridge interleaving, only 1 CFMWS entry in the CEDT is

required for that host bridge - if it covers the entire capacity of the devices

A platform may choose to add all of these, or change the mode based on a BIOS

setting.  For each CFMWS entry, Linux expects descriptions of the described

memory regions in the SRAT to determine the number of NUMA nodes it should

reserve during early boot / init.

memory regions in the :doc:`SRAT <acpi/srat>` to determine the number of

NUMA nodes it should reserve during early boot / init.

As of v6.14, Linux will create a NUMA node for each CEDT CFMWS entry, even if

a matching SRAT entry does not exist; however, this is not guaranteed in the