Albert Mingkun Yang: Design and Implementation of Temperature-Aware Garbage Collectors

Abstract

Garbage collectors (GCs) were invented as a technique to free programmers from the tedious and error-prone task of manual memory management. Traditionally, tracing collectors start with roots (such as global variables and active stack frames),traverse the entire heap to identify live (reachable) and dead (unreachable) objects, and reclaim memory occupied by dead objects. This simple binary classification of live versus dead objects is sufficient for memory reclamation.

In modern hardware, the use of cache memory is widely adopted to bridge the significant latency gap between CPUs and main memory,by leveraging spatial and temporal locality. Since the majority of applications indeed exhibit spatial and/or temporal locality,this memory hierarchy works very well in practice, effectively hiding the large latency of main memory accesses. Inspired by this observation of heterogeneous access patterns, we extend the binary classification of objects by further dividing live objects into two categories: hot (recently accessed by application threads) and cold (the inverse). We explore how to make GCs temperature-aware, taking advantage of this refinement to improve cache efficiency and achieve better application throughput.

This work is conducted with ZGC, a modern garbage collector in OpenJDK. The main body of this work consists of three components. First, we provide a thorough description of ZGC, the base collector upon which our implementations are built. Additionally, we build a SPIN model to capture the key aspects of of ZGC's design and study various concurrency behaviors in its low-level implementation. We believe this documentation and the SPIN model will be valuable for future research building on ZGC. The second component is a temperature-aware collector that relocates hot objects to improve cache locality and application performance. Our evaluation demonstrates significant performance improvements in applications with stable and recurring access patterns. The third component involves studying the behavior of cold objects and designing a temperature-aware collector that separates hot and cold objects into two subheaps, similar to the partitioning used in generational collectors. The evaluation reveals that many applications exhibit stable cold-hot clustering, allowing each cluster to be confined to its own subheap with only marginal overhead.

In summary, we believe that by treating hot and cold objects non-uniformly, temperature-aware collectors can unlock additional optimization opportunities and warrant further exploration.