Vcpu hotplug arm 5.15.12 #13
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
With ACPI enabled, cpus get identified by the presence of the GICC entry in the MADT Table. Each GICC entry part of MADT presents cpu as enabled or disabled. As of now, the disabled cpus are skipped as physical cpu hotplug is not supported. These remain disabled even after the kernel has booted. To support virtual cpu hotplug(in which case disabled vcpus could be hotplugged even after kernel has booted), QEMU will populate MADT Table with appropriate details of GICC entry for each possible(present+disabled) vcpu. Now, during the init time vcpus will be identified as present or disabled. To achieve this, below changes have been made with respect to the present/possible vcpu handling along with the mentioned reasoning: 1. Identify all possible(present+disabled) vcpus at boot/init time and set their present mask and possible mask. In the existing code, cpus are being marked present quite late within smp_prepare_cpus() function, which gets called in context to the kernel thread. Since the cpu hotplug is not supported, present cpus are always equal to the possible cpus. But with cpu hotplug enabled, this assumption is not true. Hence, present cpus should be marked while MADT GICC entries are bring parsed for each vcpu. 2. Set possible cpus to include disabled. This needs to be done now while parsing MADT GICC entries corresponding to each vcpu as the disabled vcpu info is available only at this point as for hotplug case possible vcpus is not equal to present vcpus. 3. We will store the parsed madt/gicc entry even for the disabled vcpus during init time. This is needed as some modules like PMU registers IRQs for each possible vcpus during init time. Therefore, a valid entry of the MADT GICC should be present for all possible vcpus. 4. Refactoring related to DT/OF is also done to align it with the init changes to support vcpu hotplug. Signed-off-by: Salil Mehta <[email protected]> Signed-off-by: Xiongfeng Wang <[email protected]>
Bound the total number of identified cpus(including disabled cpus) by maximum allowed limit by the kernel. Max value is either specified as part of the kernel parameters 'nr_cpus' or specified during compile time using CONFIG_NR_CPUS. Signed-off-by: Salil Mehta <[email protected]> Signed-off-by: Xiongfeng Wang <[email protected]>
Currently, cpu-operations are only initialized for the cpus which already have logical cpuid to hwid assoication established. And this only happens for the cpus which are present during boot time. To support virtual cpu hotplug, we shall initialize the cpu-operations for all possible(present+disabled) vcpus. This means logical cpuid to hwid/mpidr association might not exists(i.e. might be INVALID_HWID) during init. Later, when the vcpu is actually hotplugged logical cpuid is allocated and associated with the hwid/mpidr. This patch does some refactoring to support above change. Signed-off-by: Salil Mehta <[email protected]> Signed-off-by: Xiongfeng Wang <[email protected]>
To support virtual cpu hotplug, some arch specific hooks must be facilitated. These hooks are called by the generic ACPI cpu hotplug framework during a vcpu hot-(un)plug event handling. The changes required involve: 1. Allocation of the logical cpuid corresponding to the hwid/mpidr 2. Mapping of logical cpuid to hwid/mpidr and marking present 3. Removing vcpu from present mask during hot-unplug 4. For arm64, all possible cpus are registered within topology_init() Hence, we need to override the weak ACPI call of arch_register_cpu() (which returns -ENODEV) and return success. 5. NUMA node mapping set for this vcpu using SRAT Table info during init time will be discarded as the logical cpu-ids used at that time might not be correct. This mapping will be set again using the proximity/node info obtained by evaluating _PXM ACPI method. Note, during hot unplug of vcpu, we do not unmap the association between the logical cpuid and hwid/mpidr. This remains persistent. Signed-off-by: Salil Mehta <[email protected]> Signed-off-by: Xiongfeng Wang <[email protected]>
When hot-remove cpu, the map from cpu to numa will set to NUMA_NO_NODE which will lead to failure as the map is used by others. Thus we need a specific map to descrip the unpluged cpu. Here we introduce a new map to descrip the unpluged cpu map. Signed-off-by: Jianyong Wu <[email protected]>
jongwu
force-pushed
the
vcpu_hotplug_arm_5.15.12
branch
from
dc29f27 to
13c5067
Compare
|
Looking better. Can you create a PR rebased against ch-5.18.8, check that it works and then send upstream? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This patch set enable Vcpu hotplug for arm64, including 5 commit, 4 from Salil Mehta and 1 from Jianyong Wu.
quote from Salil's words:
The last commit fix failure when hot-remove cpu