Application-aware Thermal Management in 3D DRAM
Deep neural networks (DNNs) are extensively being employed in a vast range of applications such as image classification, object detection, visual recognition, natural language processing, and have shown remarkable performance across different domains. While GPUs and accelerators have been the conventional preference for carrying out DNN training and inference, the industry is actively being pushed towards using general-purpose processing cores for DNN inference. DNNs perform concurrent computations and are memory-intensive programs, generating frequent memory requests to the off-chip memory. The increasing speed gap between the processor and off-chip memory, commonly known as the memory wall problem, creates a bottleneck during the execution of DNN workloads. The emerging memory technologies such as 3D stacked DRAMs provide enormous memory bandwidth, however, vertically integrating DRAM dies introduces severe thermal challenges. DNNs execute in different phases, each exhibiting unique memory access behavior that varies depending on the current layer being processed. These phases display varying sensitivities to architectural parameters, such as memory-level parallelism, memory bandwidth, last level cache (LLC) size, and prefetch settings. Maximizing performance for DNN workloads requires a dynamic thermal management (DTM) policy that cannot be entirely application-agnostic. Our work proposes effective DTM policies for DNNs that leverages application-specific knowledge to make informed thermal management decisions.
Publications:
- Shailja Pandey, Preeti Ranjan Panda: NeuroMap: Efficient Task Mapping of Deep Neural Networks for Dynamic Thermal Management in High Bandwidth Memory. Accepted in International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS), and published in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD), 41(11), November 2022. (Best Paper Candidate)