Commit 4f3a6c4d authored by Catalin Marinas's avatar Catalin Marinas
Browse files

Merge branch 'for-next/vcpu-hotplug' into for-next/core 

* for-next/vcpu-hotplug: (21 commits)
  : arm64 support for virtual CPU hotplug (ACPI)
  irqchip/gic-v3: Fix 'broken_rdists' unused warning when !SMP and !ACPI
  arm64: Kconfig: Fix dependencies to enable ACPI_HOTPLUG_CPU
  cpumask: Add enabled cpumask for present CPUs that can be brought online
  arm64: document virtual CPU hotplug's expectations
  arm64: Kconfig: Enable hotplug CPU on arm64 if ACPI_PROCESSOR is enabled.
  arm64: arch_register_cpu() variant to check if an ACPI handle is now available.
  arm64: psci: Ignore DENIED CPUs
  irqchip/gic-v3: Add support for ACPI's disabled but 'online capable' CPUs
  irqchip/gic-v3: Don't return errors from gic_acpi_match_gicc()
  arm64: acpi: Harden get_cpu_for_acpi_id() against missing CPU entry
  arm64: acpi: Move get_cpu_for_acpi_id() to a header
  ACPI: Add post_eject to struct acpi_scan_handler for cpu hotplug
  ACPI: scan: switch to flags for acpi_scan_check_and_detach()
  ACPI: processor: Register deferred CPUs from acpi_processor_get_info()
  ACPI: processor: Add acpi_get_processor_handle() helper
  ACPI: processor: Move checks and availability of acpi_processor earlier
  ACPI: processor: Fix memory leaks in error paths of processor_add()
  ACPI: processor: Return an error if acpi_processor_get_info() fails in processor_add()
  ACPI: processor: Drop duplicated check on _STA (enabled + present)
  cpu: Do not warn on arch_register_cpu() returning -EPROBE_DEFER
  ...
parents 3346c566 08040200

Documentation/ABI/testing/sysfs-devices-system-cpu

+6 −0
Original line number Diff line number Diff line
@@ -694,3 +694,9 @@ Description: 
		(RO) indicates whether or not the kernel directly supports

		modifying the crash elfcorehdr for CPU hot un/plug and/or

		on/offline changes.



What:		/sys/devices/system/cpu/enabled

Date:		Nov 2022

Contact:	Linux kernel mailing list <linux-kernel@vger.kernel.org>

Description:

		(RO) the list of CPUs that can be brought online.

Documentation/arch/arm64/cpu-hotplug.rst

0 → 100644
+79 −0
Original line number Diff line number Diff line 
.. SPDX-License-Identifier: GPL-2.0

.. _cpuhp_index:



====================

CPU Hotplug and ACPI

====================



CPU hotplug in the arm64 world is commonly used to describe the kernel taking

CPUs online/offline using PSCI. This document is about ACPI firmware allowing

CPUs that were not available during boot to be added to the system later.



``possible`` and ``present`` refer to the state of the CPU as seen by linux.





CPU Hotplug on physical systems - CPUs not present at boot

----------------------------------------------------------



Physical systems need to mark a CPU that is ``possible`` but not ``present`` as

being ``present``. An example would be a dual socket machine, where the package

in one of the sockets can be replaced while the system is running.



This is not supported.



In the arm64 world CPUs are not a single device but a slice of the system.

There are no systems that support the physical addition (or removal) of CPUs

while the system is running, and ACPI is not able to sufficiently describe

them.



e.g. New CPUs come with new caches, but the platform's cache toplogy is

described in a static table, the PPTT. How caches are shared between CPUs is

not discoverable, and must be described by firmware.



e.g. The GIC redistributor for each CPU must be accessed by the driver during

boot to discover the system wide supported features. ACPI's MADT GICC

structures can describe a redistributor associated with a disabled CPU, but

can't describe whether the redistributor is accessible, only that it is not

'always on'.



arm64's ACPI tables assume that everything described is ``present``.





CPU Hotplug on virtual systems - CPUs not enabled at boot

---------------------------------------------------------



Virtual systems have the advantage that all the properties the system will

ever have can be described at boot. There are no power-domain considerations

as such devices are emulated.



CPU Hotplug on virtual systems is supported. It is distinct from physical

CPU Hotplug as all resources are described as ``present``, but CPUs may be

marked as disabled by firmware. Only the CPU's online/offline behaviour is

influenced by firmware. An example is where a virtual machine boots with a

single CPU, and additional CPUs are added once a cloud orchestrator deploys

the workload.



For a virtual machine, the VMM (e.g. Qemu) plays the part of firmware.



Virtual hotplug is implemented as a firmware policy affecting which CPUs can be

brought online. Firmware can enforce its policy via PSCI's return codes. e.g.

``DENIED``.



The ACPI tables must describe all the resources of the virtual machine. CPUs

that firmware wishes to disable either from boot (or later) should not be

``enabled`` in the MADT GICC structures, but should have the ``online capable``

bit set, to indicate they can be enabled later. The boot CPU must be marked as

``enabled``.  The 'always on' GICR structure must be used to describe the

redistributors.



CPUs described as ``online capable`` but not ``enabled`` can be set to enabled

by the DSDT's Processor object's _STA method. On virtual systems the _STA method

must always report the CPU as ``present``. Changes to the firmware policy can

be notified to the OS via device-check or eject-request.



CPUs described as ``enabled`` in the static table, should not have their _STA

modified dynamically by firmware. Soft-restart features such as kexec will

re-read the static properties of the system from these static tables, and

may malfunction if these no longer describe the running system. Linux will

re-discover the dynamic properties of the system from the _STA method later

during boot.

Documentation/arch/arm64/index.rst

+1 −0
Original line number Diff line number Diff line
@@ -13,6 +13,7 @@ ARM64 Architecture 
    asymmetric-32bit

    booting

    cpu-feature-registers

    cpu-hotplug

    elf_hwcaps

    hugetlbpage

    kdump

arch/arm64/Kconfig

+1 −0
Original line number Diff line number Diff line
@@ -5,6 +5,7 @@ config ARM64 
	select ACPI_CCA_REQUIRED if ACPI

	select ACPI_GENERIC_GSI if ACPI

	select ACPI_GTDT if ACPI

	select ACPI_HOTPLUG_CPU if ACPI_PROCESSOR && HOTPLUG_CPU

	select ACPI_IORT if ACPI

	select ACPI_REDUCED_HARDWARE_ONLY if ACPI

	select ACPI_MCFG if (ACPI && PCI)

arch/arm64/include/asm/acpi.h

+12 −0
Original line number Diff line number Diff line
@@ -119,6 +119,18 @@ static inline u32 get_acpi_id_for_cpu(unsigned int cpu) 
	return	acpi_cpu_get_madt_gicc(cpu)->uid;

}



static inline int get_cpu_for_acpi_id(u32 uid)

{

	int cpu;



	for (cpu = 0; cpu < nr_cpu_ids; cpu++)

		if (acpi_cpu_get_madt_gicc(cpu) &&

		    uid == get_acpi_id_for_cpu(cpu))

			return cpu;



	return -EINVAL;

}



static inline void arch_fix_phys_package_id(int num, u32 slot) { }

void __init acpi_init_cpus(void);

int apei_claim_sea(struct pt_regs *regs);