Commit d104e3d1 authored by Linus Torvalds's avatar Linus Torvalds
Browse files
Pull CXL updates from Dave Jiang:
 "The changes include adding poison injection support, fixing CXL access
  coordinates when onlining CXL memory, and delaing the enumeration of
  downstream switch ports for CXL hierarchy to ensure that the CXL link
  is established at the time of enumeration to address a few issues
  observed on AMD and Intel platforms.

  Misc changes:
   - Use str_plural() instead of open code for emitting strings.
   - Use str_enabled_disabled() instead of ternary operator
   - Fix emit of type resource_size_t argument for
     validate_region_offset()
   - Typo fixup in CXL driver-api documentation
   - Rename CFMWS coherency restriction defines
   - Add convention doc describe dealing with x86 low memory hole
     and CXL

  Poison Inject support:
   - Move hpa_to_spa callback to new reoot decoder ops structure
   - Define a SPA to HPA callback for interleave calculation with
     XOR math
   - Add support for SPA to DPA address translation with XOR
   - Add locked variants of poison inject and clear functions
   - Add inject and clear poison support by region offset

  CXL access coordinates update fix:
   - A comment update for hotplug memory callback prority defines
   - Add node_update_perf_attrs() for updating perf attrs on a node
   - Update cxl_access_coordinates() to use the new node update function
   - Remove hmat_update_target_coordinates() and related code

  CXL delayed downstream port enumeration and initialization:
   - Add helper to detect top of CXL device topology and remove
     open coding
   - Add helper to delete single dport
   - Add a cached copy of target_map to cxl_decoder
   - Refactor decoder setup to reduce cxl_test burden
   - Defer dport allocation for switch ports
   - Add mock version of devm_cxl_add_dport_by_dev() for cxl_test
   - Adjust the mock version of devm_cxl_switch_port_decoders_setup()
     due to cxl core usage
   - Setup target_map for cxl_test decoder initialization
   - Change SSLBIS handler to handle single dport
   - Move port register setup to when first dport appears"

* tag 'cxl-for-6.18' of git://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl: (25 commits)
  cxl: Move port register setup to when first dport appear
  cxl: Change sslbis handler to only handle single dport
  cxl/test: Setup target_map for cxl_test decoder initialization
  cxl/test: Adjust the mock version of devm_cxl_switch_port_decoders_setup()
  cxl/test: Add mock version of devm_cxl_add_dport_by_dev()
  cxl: Defer dport allocation for switch ports
  cxl/test: Refactor decoder setup to reduce cxl_test burden
  cxl: Add a cached copy of target_map to cxl_decoder
  cxl: Add helper to delete dport
  cxl: Add helper to detect top of CXL device topology
  cxl: Documentation/driver-api/cxl: Describe the x86 Low Memory Hole solution
  cxl/acpi: Rename CFMW coherency restrictions
  Documentation/driver-api: Fix typo error in cxl
  acpi/hmat: Remove now unused hmat_update_target_coordinates()
  cxl, acpi/hmat: Update CXL access coordinates directly instead of through HMAT
  drivers/base/node: Add a helper function node_update_perf_attrs()
  mm/memory_hotplug: Update comment for hotplug memory callback priorities
  cxl: Fix emit of type resource_size_t argument for validate_region_offset()
  cxl/region: Add inject and clear poison by region offset
  cxl/core: Add locked variants of the poison inject and clear funcs
  ...
parents 67da125e 46037455
Loading
Loading
Loading
Loading
+87 −0
Original line number Diff line number Diff line
@@ -19,6 +19,20 @@ Description:
		is returned to the user. The inject_poison attribute is only
		visible for devices supporting the capability.

		TEST-ONLY INTERFACE: This interface is intended for testing
		and validation purposes only. It is not a data repair mechanism
		and should never be used on production systems or live data.

		DATA LOSS RISK: For CXL persistent memory (PMEM) devices,
		poison injection can result in permanent data loss. Injected
		poison may render data permanently inaccessible even after
		clearing, as the clear operation writes zeros and does not
		recover original data.

		SYSTEM STABILITY RISK: For volatile memory, poison injection
		can cause kernel crashes, system instability, or unpredictable
		behavior if the poisoned addresses are accessed by running code
		or critical kernel structures.

What:		/sys/kernel/debug/cxl/memX/clear_poison
Date:		April, 2023
@@ -35,6 +49,79 @@ Description:
		The clear_poison attribute is only visible for devices
		supporting the capability.

		TEST-ONLY INTERFACE: This interface is intended for testing
		and validation purposes only. It is not a data repair mechanism
		and should never be used on production systems or live data.

		CLEAR IS NOT DATA RECOVERY: This operation writes zeros to the
		specified address range and removes the address from the poison
		list. It does NOT recover or restore original data that may have
		been present before poison injection. Any original data at the
		cleared address is permanently lost and replaced with zeros.

		CLEAR IS NOT A REPAIR MECHANISM: This interface is for testing
		purposes only and should not be used as a data repair tool.
		Clearing poison is fundamentally different from data recovery
		or error correction.

What:		/sys/kernel/debug/cxl/regionX/inject_poison
Date:		August, 2025
Contact:	linux-cxl@vger.kernel.org
Description:
		(WO) When a Host Physical Address (HPA) is written to this
		attribute, the region driver translates it to a Device
		Physical Address (DPA) and identifies the corresponding
		memdev. It then sends an inject poison command to that memdev
		at the translated DPA. Refer to the memdev ABI entry at:
		/sys/kernel/debug/cxl/memX/inject_poison for the detailed
		behavior. This attribute is only visible if all memdevs
		participating in the region support both inject and clear
		poison commands.

		TEST-ONLY INTERFACE: This interface is intended for testing
		and validation purposes only. It is not a data repair mechanism
		and should never be used on production systems or live data.

		DATA LOSS RISK: For CXL persistent memory (PMEM) devices,
		poison injection can result in permanent data loss. Injected
		poison may render data permanently inaccessible even after
		clearing, as the clear operation writes zeros and does not
		recover original data.

		SYSTEM STABILITY RISK: For volatile memory, poison injection
		can cause kernel crashes, system instability, or unpredictable
		behavior if the poisoned addresses are accessed by running code
		or critical kernel structures.

What:		/sys/kernel/debug/cxl/regionX/clear_poison
Date:		August, 2025
Contact:	linux-cxl@vger.kernel.org
Description:
		(WO) When a Host Physical Address (HPA) is written to this
		attribute, the region driver translates it to a Device
		Physical Address (DPA) and identifies the corresponding
		memdev. It then sends a clear poison command to that memdev
		at the translated DPA. Refer to the memdev ABI entry at:
		/sys/kernel/debug/cxl/memX/clear_poison for the detailed
		behavior. This attribute is only visible if all memdevs
		participating in the region support both inject and clear
		poison commands.

		TEST-ONLY INTERFACE: This interface is intended for testing
		and validation purposes only. It is not a data repair mechanism
		and should never be used on production systems or live data.

		CLEAR IS NOT DATA RECOVERY: This operation writes zeros to the
		specified address range and removes the address from the poison
		list. It does NOT recover or restore original data that may have
		been present before poison injection. Any original data at the
		cleared address is permanently lost and replaced with zeros.

		CLEAR IS NOT A REPAIR MECHANISM: This interface is for testing
		purposes only and should not be used as a data repair tool.
		Clearing poison is fundamentally different from data recovery
		or error correction.

What:		/sys/kernel/debug/cxl/einj_types
Date:		January, 2024
KernelVersion:	v6.9
+135 −0
Original line number Diff line number Diff line
@@ -45,3 +45,138 @@ Detailed Description of the Change
----------------------------------

<Propose spec language that corrects the conflict.>


Resolve conflict between CFMWS, Platform Memory Holes, and Endpoint Decoders
============================================================================

Document
--------

CXL Revision 3.2, Version 1.0

License
-------

SPDX-License Identifier: CC-BY-4.0

Creator/Contributors
--------------------

- Fabio M. De Francesco, Intel
- Dan J. Williams, Intel
- Mahesh Natu, Intel

Summary of the Change
---------------------

According to the current Compute Express Link (CXL) Specifications (Revision
3.2, Version 1.0), the CXL Fixed Memory Window Structure (CFMWS) describes zero
or more Host Physical Address (HPA) windows associated with each CXL Host
Bridge. Each window represents a contiguous HPA range that may be interleaved
across one or more targets, including CXL Host Bridges. Each window has a set
of restrictions that govern its usage. It is the Operating System-directed
configuration and Power Management (OSPM) responsibility to utilize each window
for the specified use.

Table 9-22 of the current CXL Specifications states that the Window Size field
contains the total number of consecutive bytes of HPA this window describes.
This value must be a multiple of the Number of Interleave Ways (NIW) * 256 MB.

Platform Firmware (BIOS) might reserve physical addresses below 4 GB where a
memory gap such as the Low Memory Hole for PCIe MMIO may exist. In such cases,
the CFMWS Range Size may not adhere to the NIW * 256 MB rule.

The HPA represents the actual physical memory address space that the CXL devices
can decode and respond to, while the System Physical Address (SPA), a related
but distinct concept, represents the system-visible address space that users can
direct transaction to and so it excludes reserved regions.

BIOS publishes CFMWS to communicate the active SPA ranges that, on platforms
with LMH's, map to a strict subset of the HPA. The SPA range trims out the hole,
resulting in lost capacity in the Endpoints with no SPA to map to that part of
the HPA range that intersects the hole.

E.g, an x86 platform with two CFMWS and an LMH starting at 2 GB:

 +--------+------------+-------------------+------------------+-------------------+------+
 | Window | CFMWS Base |    CFMWS Size     | HDM Decoder Base |  HDM Decoder Size | Ways |
 +========+============+===================+==================+===================+======+
 |   0    |   0 GB     |       2 GB        |      0 GB        |       3 GB        |  12  |
 +--------+------------+-------------------+------------------+-------------------+------+
 |   1    |   4 GB     | NIW*256MB Aligned |      4 GB        | NIW*256MB Aligned |  12  |
 +--------+------------+-------------------+------------------+-------------------+------+

HDM decoder base and HDM decoder size represent all the 12 Endpoint Decoders of
a 12 ways region and all the intermediate Switch Decoders. They are configured
by the BIOS according to the NIW * 256MB rule, resulting in a HPA range size of
3GB. Instead, the CFMWS Base and CFMWS Size are used to configure the Root
Decoder HPA range that results smaller (2GB) than that of the Switch and
Endpoint Decoders in the hierarchy (3GB).

This creates 2 issues which lead to a failure to construct a region:

1) A mismatch in region size between root and any HDM decoder. The root decoders
   will always be smaller due to the trim.

2) The trim causes the root decoder to violate the (NIW * 256MB) rule.

This change allows a region with a base address of 0GB to bypass these checks to
allow for region creation with the trimmed root decoder address range.

This change does not allow for any other arbitrary region to violate these
checks - it is intended exclusively to enable x86 platforms which map CXL memory
under 4GB.

Despite the HDM decoders covering the PCIE hole HPA region, it is expected that
the platform will never route address accesses to the CXL complex because the
root decoder only covers the trimmed region (which excludes this). This is
outside the ability of Linux to enforce.

On the example platform, only the first 2GB will be potentially usable, but
Linux, aiming to adhere to the current specifications, fails to construct
Regions and attach Endpoint and intermediate Switch Decoders to them.

There are several points of failure that due to the expectation that the Root
Decoder HPA size, that is equal to the CFMWS from which it is configured, has
to be greater or equal to the matching Switch and Endpoint HDM Decoders.

In order to succeed with construction and attachment, Linux must construct a
Region with Root Decoder HPA range size, and then attach to that all the
intermediate Switch Decoders and Endpoint Decoders that belong to the hierarchy
regardless of their range sizes.

Benefits of the Change
----------------------

Without the change, the OSPM wouldn't match intermediate Switch and Endpoint
Decoders with Root Decoders configured with CFMWS HPA sizes that don't align
with the NIW * 256MB constraint, and so it leads to lost memdev capacity.

This change allows the OSPM to construct Regions and attach intermediate Switch
and Endpoint Decoders to them, so that the addressable part of the memory
devices total capacity is made available to the users.

References
----------

Compute Express Link Specification Revision 3.2, Version 1.0
<https://www.computeexpresslink.org/>

Detailed Description of the Change
----------------------------------

The description of the Window Size field in table 9-22 needs to account for
platforms with Low Memory Holes, where SPA ranges might be subsets of the
endpoints HPA. Therefore, it has to be changed to the following:

"The total number of consecutive bytes of HPA this window represents. This value
shall be a multiple of NIW * 256 MB.

On platforms that reserve physical addresses below 4 GB, such as the Low Memory
Hole for PCIe MMIO on x86, an instance of CFMWS whose Base HPA range is 0 might
have a size that doesn't align with the NIW * 256 MB constraint.

Note that the matching intermediate Switch Decoders and the Endpoint Decoders
HPA range sizes must still align to the above-mentioned rule, but the memory
capacity that exceeds the CFMWS window size won't be accessible.".
+1 −1
Original line number Diff line number Diff line
@@ -173,7 +173,7 @@ Accelerator
User Flow Support
-----------------

* [0] Inject & clear poison by HPA
* [2] Inject & clear poison by region offset

Details
=======
+1 −1
Original line number Diff line number Diff line
@@ -202,7 +202,7 @@ future and such a configuration should be avoided.

Memory Holes
------------
If your platform includes memory holes intersparsed between your CXL memory, it
If your platform includes memory holes interspersed between your CXL memory, it
is recommended to utilize multiple decoders to cover these regions of memory,
rather than try to program the decoders to accept the entire range and expect
Linux to manage the overlap.
+0 −34
Original line number Diff line number Diff line
@@ -74,7 +74,6 @@ struct memory_target {
	struct node_cache_attrs cache_attrs;
	u8 gen_port_device_handle[ACPI_SRAT_DEVICE_HANDLE_SIZE];
	bool registered;
	bool ext_updated;	/* externally updated */
};

struct memory_initiator {
@@ -368,35 +367,6 @@ static void hmat_update_target_access(struct memory_target *target,
	}
}

int hmat_update_target_coordinates(int nid, struct access_coordinate *coord,
				   enum access_coordinate_class access)
{
	struct memory_target *target;
	int pxm;

	if (nid == NUMA_NO_NODE)
		return -EINVAL;

	pxm = node_to_pxm(nid);
	guard(mutex)(&target_lock);
	target = find_mem_target(pxm);
	if (!target)
		return -ENODEV;

	hmat_update_target_access(target, ACPI_HMAT_READ_LATENCY,
				  coord->read_latency, access);
	hmat_update_target_access(target, ACPI_HMAT_WRITE_LATENCY,
				  coord->write_latency, access);
	hmat_update_target_access(target, ACPI_HMAT_READ_BANDWIDTH,
				  coord->read_bandwidth, access);
	hmat_update_target_access(target, ACPI_HMAT_WRITE_BANDWIDTH,
				  coord->write_bandwidth, access);
	target->ext_updated = true;

	return 0;
}
EXPORT_SYMBOL_GPL(hmat_update_target_coordinates);

static __init void hmat_add_locality(struct acpi_hmat_locality *hmat_loc)
{
	struct memory_locality *loc;
@@ -773,10 +743,6 @@ static void hmat_update_target_attrs(struct memory_target *target,
	u32 best = 0;
	int i;

	/* Don't update if an external agent has changed the data.  */
	if (target->ext_updated)
		return;

	/* Don't update for generic port if there's no device handle */
	if ((access == NODE_ACCESS_CLASS_GENPORT_SINK_LOCAL ||
	     access == NODE_ACCESS_CLASS_GENPORT_SINK_CPU) &&
Loading