3.2. Development History

Note

This section provides historical context on the development of NUMA-aware claims, including previous implementations and feedback received, to give a better understanding of the design decisions made in the current implementation.

The initial implementation of single-node claims (by Alejandro Vallejo) introduced node-exact claims, allowing domain builders to claim memory on one NUMA node. It passed a NUMA node in the node bits of the xen_memory_reservation.mem_flags field of the pre-existing claims hypercall XENMEM_claim_pages and, by adding the flag d->claim_node and updating it to the passed node, defined the target of the claim as either the specified NUMA node or global memory.

Feedback and suggestions for multi-node claim sets

The initial implementations of single-node claims received feedback from the community, with multiple suggestions to extend the API to support multi-node claim sets. This feedback highlighted the need for a more flexible and extensible design that could accommodate claims on multiple NUMA nodes.

This design was relatively simple and allowed for a quick implementation of single-node claims, but it had limitations in terms of flexibility and future extensibility.

The v2 series added a hypercall API for multi-node claims, opening the door to future multi-node claim sets and further work in that direction.

The v3 series refactored and improved the implementation, protecting claimed memory against parallel allocations by other domain builders.

Between v3 and v4, Roger Pau Monné and Andrew Cooper developed and merged several critical fixes for Xen’s overall claims implementation. These fixes also allowed Roger to improve the implementation for redeeming claims during domain memory allocation. In turn, this enabled a fully working implementation that protected claimed memory against parallel allocations by other domain builders.

With the v4 series, we submitted the combined work that completed the fixes for protecting claimed memory on NUMA nodes. The review process indicated that supporting multiple claim sets would require a redesign of claim installation and management, which led to this design document.

3.2.1. Acknowledgements

The claim sets design builds on the single-node claims implementation described above and the feedback it generated. The following people should be acknowledged for their contributions:

• Alejandro Vallejo for initiating the single-node NUMA claims series.

• Roger Pau Monné for merging critical fixes and proposing the initial multi-node claim-sets specification that inspired this design.

• Andrew Cooper for integrating and validating the work internally, helping to stabilise and productise the single-node implementation.

• Jan Beulich for providing reviews that led to many improvements.

• Bernhard Kaindl for maintaining the single-node series, initiating the multi-node implementation and authoring this design document.

• Marcus Granado and Edwin Török for contributing design input, providing guidance, debugging and testing of single-node implementations.