ISCA 2024 Memory Centric Tutorial

Processing-in-Memory (PIM) is a computing paradigm that aims at overcoming the data movement bottleneck (i.e., the waste of execution cycles and energy resulting from the back-and-forth data movement between memory units and compute units) by making memory (and storage) systems compute-capable.

Explored over several decades since the 1960s, PIM systems are now becoming a reality with the advent of the first commercial products and prototypes.

Several startups (e.g., UPMEM, NeuroBlade, Mythic, Syntiant, Aizip, Axelera, d-Matrix, Gyrfalcon Technology, MemComputing, SEMRON, SureCore, Synthara, TetraMem, EnCharge AI) are already commercializing real PIM hardware, each with its design approach and target applications. Major vendors (e.g., Samsung, SK Hynix, Micron, Alibaba) have presented real PIM chip and system prototypes in the past several years.

Recent PIM products and prototypes place compute units near the memory arrays. New memory interfaces like CXL (Compute Express Link) aid the enablement of compute-capable memories. At the same time, academia and industry are actively exploring other types of PIM by, e.g., exploiting the analog operation of DRAM, SRAM, flash memory, and emerging non-volatile memories, and hybrid PIM architectures that combine processing capabilities of different types and at different parts of the memory/storage hierarchy.

PIM can improve performance and energy efficiency for many modern applications, enabling a commercially viable way of dealing with huge amounts of data bottlenecking our computing systems, which is especially exacerbated by workloads like AI/ML and genomics. In fact, workloads like large language model training and inference can potentially be “killer applications'' for PIM.

However, there are many open questions spanning the entire computing stack and many challenges for widespread adoption. For example, it is critical to (1) develop programming frameworks and tools that can lower the learning curve and ease the adoption of PIM systems, (2) develop methods to identify what type of PIM would be useful for what workload, and (3) design system and security mechanisms that enable PIM in a wider scale. Implications of PIM on all aspects of computing systems and workloads is a challenging and exciting field of study.

This tutorial focuses on the latest advances in PIM technology, spanning both hardware and software, including novel PIM ideas, different tools and frameworks to conduct PIM research, and programming techniques and optimization strategies for PIM kernels. We will (1) provide an introduction to PIM and the taxonomy of PIM systems, (2) give an overview and a rigorous analysis of existing PIM hardware from industry and academia, (3) provide and describe hardware and software infrastructures that can enable new and experienced researchers to conduct research in PIM systems, and (4) shed light on how to improve future PIM systems for emerging memory-bound workloads. The tutorial will also incorporate invited talks from leading industry and academic researchers in PIM systems.

YouTube livestream

• 9:00am-9:20am, Prof. Onur Mutlu / Geraldo F. Oliveira, “Memory-centric Computing: Introduction to PIM as a Paradigm to Overcome the Data Movement Bottleneck.”
  • Workload analysis and system bottlenecks.
  • PIM taxonomy: technology, location, and nature of computation (e.g., PNM (processing-near-memory) and PUM (processing-using-memory).
  • Advances in different types of PIM at different parts of the memory/storage systems.
• 9:30am-9:50am, Prof. Minsoo Rhu, Invited Talk #1.

• Coffee break (10:00am-10:20am)

• 10:30am-10:50am, Dr. Mohammad Sadr, “Processing-Near-Memory: Real PNM Architectures.”
  • Example real-world PNM systems: UPMEM PIM, Samsung HBM-PIM & CXL-PNM, SK Hynix AiM & CMS 2.0, Samsung AxDIMM, Alibaba PNM, Mythic.
• 11:00am-11:20am, Geraldo F. Oliveira, “PUM Systems for Bulk Bitwise Operations.”
  • PUM systems for bulk bitwise operations in simulated and off-the-shelf memory technologies (DRAM, SRAM, and NVM).
• 11:30am-11:50am, Prof. Saugata Ghose, Invited Talk #2.
• 12:00pm-12:20pm, Geraldo F. Oliveira, “Programming Techniques, Infrastructure, and Research Challenges for PIM.”
  • Programming techniques and tools for PIM systems.
  • Infrastructures for doing PIM Research (simulation, real systems, FPGA prototypes).
  • Research challenges and opportunities in PIM systems, with a focus on enabling adoption in the real world.
• 12:20pm-12:30pm, Geraldo F. Olveira, “Closing Remarks.”

• Mutlu, O., Ghose, S., Gómez-Luna, J., and Ausavarungnirun, R. A Modern Primer on Processing in Memory. In Emerging Computing: From Devices to Systems, 2023.
• Gómez-Luna, J., El Hajj, I., Fernandez, I., Giannoula, C., Oliveira, G. F., and Mutlu, O. Benchmarking a New Paradigm: Experimental Analysis and Characterization of a Real Processing-in-Memory System. IEEE Access, 2022.
  • PDF (arXiv)
  • Repository (GitHub)
• Giannoula, C., Fernandez, I., Gómez-Luna, J., Koziris, N., Goumas, G., and Mutlu, O. SparseP: Towards Efficient Sparse Matrix Vector Multiplication on Real Processing-In-Memory Architectures. SIGMETRICS 2022.
  • PDF (arXiv)
  • Repository (GitHub)
• Olgun, A., Gómez-Luna, J., Kanellopoulos, K., Salami, B., Hassan, H., Ergin, O., and Mutlu, O. PiDRAM: A Holistic End-to-end FPGA-based Framework for Processing-in-DRAM. ACM TACO, 2022.
  • PDF (arXiv)
  • Repository (GitHub)
• Oliveira, G. F., Gómez-Luna, J., Orosa, L., Ghose, S., Vijaykumar, N., Fernandez, I., Sadrosadati, M., Mutlu, O. DAMOV: A New Methodology and Benchmark Suite for Evaluating Data Movement Bottlenecks. IEEE Access, 2021.
  • PDF (arXiv)
  • Repository (GitHub)
• Luo, H., Tu, Y. C., Bostancı, F. N., Olgun, A., Ya, A. G., Mutlu, O. Ramulator 2.0: A Modern, Modular, and Extensible DRAM Simulator. IEEE CAL, 2023.
  • PDF (arXiv)
  • Repository (GitHub)
• Olgun, A., Hassan, H., Yağlıkçı, A. G., Tuğrul, Y. C., Orosa, L., Luo, H., Patel, M., Ergin, O., Mutlu, O. DRAM Bender: An Extensible and Versatile FPGA-based Infrastructure to Easily Test State-of-the-art DRAM Chips. IEEE CAD, 2023.
  • PDF (arXiv)
  • Repository (GitHub)
• Oliveira, G. F., Olgun, A., Yaglikci, A. G., Bostanci, N., Gomez-Luna, J., Ghose, S., Mutlu, O., MIMDRAM: An End-to-End Processing-Using-DRAM System for High-Throughput, Energy-Efficient and Programmer-Transparent Multiple-Instruction Multiple-Data Computing, in HPCA, 2024.
  • PDF (arXiv)
  • Repository (GitHub)
• Hajinazar, N., Oliveira, G. F., Gregorio, S., Ferreira, J. D., Ghiasi, N. M., Patel, M., Alser, M., Ghose, S., Gomez-Luna, J., Mutlu. O., SIMDRAM: An End-to-End Framework for Bit-Serial SIMD Computing in DRAM, in ASPLOS, 2021.
  • PDF (arXiv)
  • Full Talk Video
• Seshadri, V., Lee, D., Mullins, T., Hassan, H., Boroumand, A., Kim, J., Kozuch, M. A., Mutlu, O., Gibbons, P. B., Mowry, T. C., Ambit: In-Memory Accelerator for Bulk Bitwise Operations Using Commodity DRAM Technology, in MICRO, 2017.
  • PDF

• Mutlu O., Memory-Centric Computing (IMACAW Keynote Talk at DAC 2023), July 2023:
  • PDF PPT Video
• Processing-in-Memory: A Workload-Driven Perspective (summary paper about recent research in PIM):
  • PDF
• Processing Data Where It Makes Sense: Enabling In-Memory Computation (summary paper about recent research in PIM):
  • PDF
• Processing-in-Memory course (Spring 2022):
  • Course website
• Gómez-Luna, J., and Mutlu, O., Data-Centric Architectures: Fundamentally Improving Performance and Energy (227-0085-37L), ETH Zürich, Fall 2022.
  • Course Website
  • Lecture Playlist
  • PDF (arXiv)