Literature Survey: Processing In Memory

Domain: Quantum Computing & Next-Gen Architecture
Topic Search: processing in memory
Timeframe: 2021 - 2026

This is a curated survey of recent publications focusing on processing in memory. Results are filtered for top-tier journals and prominent conferences.

📚 Curated Peer-Reviewed Publications

1. PAPI: Exploiting Dynamic Parallelism in Large Language Model Decoding with a Processing-In-Memory-Enabled Computing System

Venue: ASPLOS | Year: 2025 | Citations: 56 Authors: Yintao He, Haiyu Mao, Christina Giannoula, Mohammad Sadrosadati, Juan Gómez-Luna et al.

Large language models (LLMs) are widely used for natural language understanding and text generation. An LLM model relies on a time-consuming step called LLM decoding to generate output tokens. Several prior works focus on improving the performance of LLM decoding using parallelism techniques, such as batching and speculative decoding. State-of-the-art LLM decoding has both compute-bound and memory-bound kernels. Some prior works statically identify and map these different kernels to a heterogeneous architecture consisting of both processing-in-memory (PIM) units and computation-centric accelerators (e.g., GPUs). We observe that characteristics of LLM decoding kernels (e.g., whether or not a kernel is memory-bound) can change dynamically due to parameter changes to meet user and/or system demands, making (1) static kernel mapping to PIM units and computation-centric accelerators suboptimal, and (2) one-size-fits-all approach of designing PIM units inefficient due to a large degree of heterogeneity even in memory-bound kernels. In this paper, we aim to accelerate LLM decoding while considering the dynamically changing characteristics of the kernels involved. We propose PAPI (PA rallel Decoding with PI M), a PIM-enabled heterogeneous architecture that exploits dynamic scheduling of compute-bound or memory-bound kernels to suitable hardware units. PAPI has two key mechanisms: (1) online kernel characterization to dynamically schedule kernels to the most suitable hardware units at runtime and (2) a PIM-enabled heterogeneous computing system that harmoniously orchestrates both computation-centric processing units (GPU) and hybrid PIM units with different computing capabilities. Our experimental results on three broadly-used LLMs (i.e., LLaMA-65B, GPT-3 66B, and GPT-3 175B) show that PAPI achieves 1.8× and 11.1× speedups over a state-of-the-art heterogeneous LLM accelerator (i.e., GPU and PIM) and a state-of-the-art PIM-only LLM accelerator, respectively.

2. PIM-STM: Software Transactional Memory for Processing-In-Memory Systems

Venue: ASPLOS | Year: 2024 | Citations: 12 Authors: André Lopes, Daniel Castro 0004, Paolo Romano 0002

Processing-In-Memory (PIM) is a novel approach that augments existing DRAM memory chips with lightweight logic. By allowing to offload computations to the PIM system, this architecture allows for circumventing the data-bottleneck problem that affects many modern workloads. This work tackles the problem of how to build efficient software implementations of the Transactional Memory (TM) abstraction by introducing PIM-STM, a library that provides a range of diverse TM implementations for UPMEM, the first commercial PIM system. Via an extensive study we assess the efficiency of alternative choices in the design space of TM algorithms on this emerging architecture. We further quantify the impact of using different memory tiers of the UPMEM system (having different trade-offs for what concerns latency vs capacity) to store the metadata used by different TM implementations. Finally, we assess the gains achievable in terms of performance and memory efficiency when using PIM-STM to accelerate TM applications originally conceived for conventional CPU-based systems.

3. Optimal Data Allocation for Graph Processing in Processing-in-Memory Systems

Venue: ASP-DAC | Year: 2022 | Citations: 7 Authors: Zerun Li, Xiaoming Chen 0003, Yinhe Han 0001

Graph processing involves lots of irregular memory accesses and increases demands on high memory bandwidth, making it difficult to execute efficiently on compute-centric architectures. Dedicated graph processing accelerators based on the processing-in-memory (PIM) technique have recently been proposed. Despite they achieved higher performance and energy efficiency than conventional architectures, the data allocation problem for communication minimization in PIM systems (e.g., hybrid memory cubes (HMCs)) has still not been well solved. In this paper, we demonstrate that the conventional “graph data allocation = graph partitioning” assumption is not true, and the memory access patterns of graph algorithms should also be taken into account when partitioning graph data for communication minimization. For this purpose, we classify graph algorithms into two representative classes from a memory access pattern point of view and propose different graph data partitioning strategies for them. We then propose two algorithms to optimize the partition-to-HMC mapping to minimize the inter-HMC communication. Evaluations have proved the superiority of our data allocation framework and the data movement energy efficiency is improved by 4.2-5 × on average than the state-of-the-art GraphP approach.

4. Minimizing Communication Conflicts in Network-On-Chip Based Processing-In-Memory Architecture

Venue: DATE | Year: 2023 | Citations: 7 Authors: Hanbo Sun, Tongxin Xie, Zhenhua Zhu 0002, Guohao Dai 0001, Huazhong Yang et al.

Deep Neural Networks (DNNs) have made significant breakthroughs in various fields. However, their enormous computations and parameters seriously hinder their applications. Emerging Processing-In-Memory (PIM) architectures provide extremely high energy efficiency to accelerate DNN computing. Moreover, Network-on-Chip (NoC) based PIM architectures significantly improve the scalability of PIM architectures. However, the contradiction between high communication and limited NoC bandwidth introduces severe communication conflicts. Existing work neglects the impact of communication conflicts. On the one hand, neglecting communication conflicts leads to the lack of precise performance estimations in the mapping process, making it hard to find optimal results. On the other hand, communication conflicts cause low NoC bandwidth utilization in the schedule process. And there is over 70% latency gap in existing work caused by communication conflicts. This paper proposes communication conflict optimized mapping and schedule strategies for NoC-based PIM architectures. The proposed mapping strategy constructs communication conflict graphs to model communication conflicts. Based on this constructed graph, we adopt a Graph Neural Network (GNN) as a precise performance estimator. Our schedule strategy predefines the communication priority and NoC communication behavior tables for target DNN workloads. In this way, it can improve the NoC bandwidth utilization effectively. Compared with existing work, for typical classification DNNs on the CIFAR and ImageNet datasets, the proposed strategies reduce 78% latency and improve the throughput by 3.33× on average with negligible deployment and hardware overhead. Experimental results also show that our strategies decrease the average gap to ideal cases without communication conflicts from 80.7% and 70% to 12.3% and 1.26% for latency and throughput, respectively.

5. A Combined Content Addressable Memory and In-Memory Processing Approach for k-Clique Counting Acceleration

Venue: DAC | Year: 2024 | Citations: 6 Authors: Xidi Ma, Weichen Zhang, Xueyan Wang, Tianyang Yu, Bi Wu 0002 et al.

k-Clique counting problem plays an important role in graph mining which has seen a growing number of applications. However, cur-rent k-Clique counting accelerators cannot meet the performance requirement mainly because they struggle with high data transfer issue incurred by the intensive set intersection operations and the inability of load balancing. In this paper, we propose to solve this problem with a hybrid framework of content addressable memory (CAM) and in-memory processing (PIM). Specifically, we first utilize CAM for binary induced subgraph generation in order to reduce the search space, then we use PIM to implement in-place parallel k-Clique counting through iterative Boolean logic “AND”-like operation. To take full advantage of this combined CAM and PIM framework, we develop dynamic task scheduling strategies that can achieve near optimal load balancing among the PIM arrays. Experimental results demonstrate that, compared with state-of-the-art CPU and GPU platforms, our approach achieves speedups of 167.5× and 28.8×, respectively. Meanwhile, the energy efficiency is improved by 788.3× over the GPU baseline.

6. The Memory Processing Unit: A Generalized Interface for End-to-End In-Memory Execution

Venue: HPCA | Year: 2026 | Citations: 3 Authors: Minh S. Q. Truong, Yiqiu Sun 0002, Dawei Xiong, Amol Shah 0001, Alexander Glass et al.

The processing-using-memory (PUM; a.k.a. inmemory computing) paradigm aims to eliminate data movement energy and performance costs by using memory cell interactions to directly perform computation. Given PUM’s potential for large savings, prior works have proposed many different datapath microarchitectures to demonstrate how general-purpose PUM benefits a wide range of application kernels. Unfortunately, these efforts largely depend on microarchitecture-specific vector-like interfaces that (1) force many of an application’s operations to be offloaded to a CPU, (2) require significant programmer effort to scale up applications to an entire memory chip, and (3) make it impractical to develop badly-needed systems software and programming tools for PUM. To address these three issues, we propose the memory processing unit (MPU), a microarchitecture-agnostic interface layer for general-purpose PUM with three components. First, we develop an MPU instruction set architecture (ISA) with instructions to facilitate application scaling and task coordination. Second, we propose an ensemble execution model that coordinates execution across millions of PUM vector function units and maps to most general-purpose PUM microarchitectures. Third, we design a comprehensive MPU control path that efficiently executes MPU ISA binaries across multiple ensembles, and can enable CPU-free execution of complex end-to-end applications with PUM. We demonstrate how the MPU maps to multiple previously-proposed PUM datapaths, and how it achieves average performance/energy improvements of $\mathbf{1. 7 9} \times \boldsymbol{/} \mathbf{3. 2 3} \times$ for $\mathbf{2 1}$ data-intensive kernels over these prior works ($67 \times / 47 \times$ vs. a modern GPU), while also achieving performance and energy improvements for the complex end-to-end applications.

7. X-PIM: Fast Modeling and Validation Framework for Mixed-Signal Processing-in-Memory Using Compressed Equivalent Model in System Verilog

Venue: DATE | Year: 2024 | Citations: 2 Authors: Ingu Jeong, Jun-Eun Park

Mixed-signal processing-in-memory (PIM) has gained prominence as a promising approach for implementing deep neural networks in energy-constrained systems. However, the co-design and optimization of mixed-signal circuits in PIM demand substantial time and effort for simulation and validation. This work presents X-PIM, a fast modeling and validation framework for mixed-signal PIMs. X-PIM encompasses not only the precise modeling of transistor-level analog computation circuits in System Verilog but also the rapid validation of system-level neural networks implemented using the mixed-signal PIM model. For achieving both accuracy and speed in simulation, X-PIM introduces a technique called compressed equivalent model (CEM) for the mixed-signal PIM circuits. This technique transforms a two-dimensional PIM array into an equivalent single-cell model. Furthermore, X-PIM can account for the impact of non-ideal operations in mixed-signal circuits by incorporating effects such as ADC quantization noise, parasitic components, intrinsic noise, and finite bandwidth. Based on the proposed mixed-signal PIM modeling, X-PIM can perform the system-level neural network validation with a significantly reduced simulation time at least 200 times faster than that of SPICE-based validation. X-PIM demonstrates three mixed-signal PIMs: XNOR PIM, capacitive PIM, and ReRAM-based PIM. For multi-layer perceptron (MLP) network, X-PIM can complete accuracy evaluation for MNIST-IOOO dataset within only 30 minutes.

8. AttenPIM: Accelerating LLM Attention with Dual-mode GEMV in Processing-in-Memory

Venue: DAC | Year: 2025 | Citations: 2 Authors: Liyan Chen, Dongxu Lyu, Zhenyu Li, Jianfei Jiang 0001, Qin Wang 0009 et al.

Large Language Models (LLMs) have demonstrated unprecedented generative performance across a wide range of applications. While recent heterogeneous architectures attempt to address the memory-bound bottleneck from attention computations by processing-in-memory (PIM) offloading, they overlook two critical characteristics of attention GEMVs that distinguish them from traditional PIM scenarios: (1) dynamic matrix dimensions that scale with token length, and (2) distinct GEMV patterns between score computation ($Q \times K_{t}$) and context computation ($S \times V$). Existing PIM designs, employing either uniform or transposed computing modes, suffer from inefficiencies in newly generated element preparation or distinct GEMV execution. To address these limitations, we propose AttenPIM, a software-hardware co-design for efficient PIM-based attention acceleration. For bank-level execution, we propose dual-mode computing modes tailored for score and context computations with PIM-oriented data layouts and execution flows for KV storage, supported by a low-cost configurable per-bank PIM unit (PU). For system-level execution, we leverage token-level and head-level concurrency to ensure workload balance and maximize bank PU parallelism. Furthermore, dynamic allocation and kernel fusion methods are proposed to further minimize memory overhead. Experimental results demonstrate that AttenPIM achieves $1.13 \times-5.26 \times$ speedup and reduces energy consumption by 17 %-49 % compared to two state-of-the-art PIM baselines.

9. PICK: An SRAM-based Processing-in-Memory Accelerator for K-Nearest-Neighbor Search in Point Clouds

Venue: DAC | Year: 2025 | Citations: 2 Authors: Chen Nie, Chao Jiang, Liming Xiao, Weifeng Zhang 0003, Zhezhi He et al.

K-nearest neighbor (kNN) search is a fundamental operation in various point cloud applications, such as autonomous driving. However, the heavy computational intensity and memory demands of kNN search pose significant challenges for efficient implementation, especially in resource-constrained scenarios. To address these challenges, we propose PICK, a processing-in-memory (PIM) architecture designed to accelerate kNN search in point cloud applications. PICK leverages bit-serial-based PIM (BS-PIM) and customized circuits to efficiently handle key operations of kNN search: distance calculation and top-k selection. The run-time off-chip access is eliminated thanks to the large on-chip memory. For distance calculation, we introduce a bit-width clipping technique to reduce the latency of bit-serial execution with negligible accuracy degradation, providing flexible trade-offs between performance and precision. Besides, we propose a filtering-and-selection strategy that realizes approximately constant time complexity for arbitrary values of k. Furthermore, a two-stage pipeline is implemented to parallelize distance calculation and top-k search, effectively hiding latency and improving throughput. According to our experiments, PICK achieves $4.17 \times$ speedup and a $4.42 \times$ energy saving over the state-of-the-art design.

10. PIM-Malloc: A Fast and Scalable Dynamic Memory Allocator for Processing-In-Memory (PIM) Architectures

Venue: HPCA | Year: 2026 | Citations: 1 Authors: Dongjae Lee, Bongjoon Hyun, Youngjin Kwon, Minsoo Rhu

The ability to dynamically allocate memory is fundamental in modern programming languages. However, this feature is not adequately supported in current general-purpose PIM devices. To identify key design principles that PIM must consider, we conduct a design space exploration of PIM memory allocators, examining various strategies for metadata placement and management of the allocator. Based on this exploration, we introduce PIM-malloc, a fast and scalable memory allocator for general-purpose PIM that operates on real PIM hardware, achieving a $66 \times$ improvement in memory allocation performance. This design is further enhanced with a lightweight, per-PIM core hardware cache, specifically designed for dynamic memory allocation, achieving an additional 31% performance improvement. Finally, we demonstrate the applicability of PIM-malloc by developing several representative PIM workloads, demonstrating its effectiveness in enhancing programmability.

11. MIRACLE: Multimodal Information Retrieval via a Combined In-Memory Processing and Content Addressable Memory Approach

Venue: DAC | Year: 2025 | Citations: 1 Authors: Xuehui Liu, Xueyan Wang, Tianyang Yu, Chen Cheng, Shuo Ran et al.

The rapid advancement of information technology has brought multimodal information retrieval into the research spotlight. Neural networks, particularly Transformers, have emerged as the dominant solution for extracting multimodal feature vectors. While neural network acceleration has been extensively explored, the subsequent retrieval stage in multimodal scenarios remains under-optimized. Conventional retrieval approaches, such as cosine similarity sorting on von Neumann architectures, suffer from significant data migration and computational inefficiencies. Hashing methods enhance storage and computation efficiency but encounter challenges in energy-efficient implementation and mitigating accuracy losses due to modal heterogeneity. This paper presents a hybrid architecture that integrates in-memory processing (PIM) and content-addressable memory (CAM) to address these challenges. Transformer-extracted features are processed via in-memory random hashing leveraging device-intrinsic properties, with CAM facilitating parallel search space reduction. A final cosine similarity reranking stage refines the results while balancing accuracy with energy efficiency. Experimental evaluations validate that the proposed method, when compared to the baseline traditional CPU-based cosine similarity retrieval, 1) achieves almost identical level of accuracy, dramatically outperforming other pure CAMbased Hamming distance retrieval approaches; and 2) reduces latency by $9.45 \times$ and energy consumption by $30.20 \times$.

12. A PulseWidth-IN-PulseWidth-Out Universal Nonlinear Processing Element for Time-Domain In-Memory Computing Systems

Venue: DAC | Year: 2025 | Citations: 0 Authors: Yihao Chen, Pengcheng Feng, Zhigang Li, Gang Chen, Rongxuan Shen et al.

Time-Domain In-Memory Computing (TD-IMC) has emerged as a promising analog computing architecture for edge AI applications. However, the lack of developed hardware operators, especially general nonlinear operators, necessitates frequent cross-domain data transmission in practical TD-IMC systems, significantly reducing energy efficiency. In this work, we propose a PulseWidth-IN-PulseWidth-OUT Universal Nonlinear Processing Element (PIPO-UNPE) to address the challenges of nonlinear processing in analog computing. By implementing an RRAM-based two-layer ReLU network, the PIPO-UNPE performs universal nonlinear operations entirely in the time domain. Algorithmically, we introduce Dynamic Loss-Responsive Subset Enhancement (DLRSE) to boost the performance of this low-cost network in function approximation tasks. From a hardware perspective, we design an RRAM-based pulse-driven programmable current source and a low-latency dispersion comparator-based voltage-to-time converter (VTC) to enhance both the energy efficiency and precision of the PIPO-UNPE. Hybrid simulations reveal that the PIPO-UNPE consumes 912 uW of power while delivering a throughput of $\mathbf{1 0 M}$ NOPS (Nonlinear Operations Per Second). Incorporating the PIPO-UNPE into the TD-IMC accelerator can increase energy efficiency by a factor of 9.5 to 25, keeping the accuracy loss below 0.1%.

⚡ Latest Pre-Prints

No pre-prints selected.

🧠 Architectural & Methodological Insights

Trajectory: The subfield is converging on runtime‑adaptive, software‑driven orchestration of heterogeneous PIM fabrics. Architectural research is no longer limited to a single PIM datapath; instead, it emphasizes (i) flexible ISA/ABIs that can span many microarchitectures, (ii) intelligent data placement and NoC‑aware mapping, and (iii) hybrid analog‑digital or CAM‑augmented compute blocks that target algorithm‑specific bottlenecks. This evolution mirrors the broader trend in computer architecture toward co‑design across hardware, system software, and compiler/runtime layers.

🚀 Critical Research Gaps

1. Scalable, cross‑tier performance modeling for heterogeneous PIM systems

   • PAPI demonstrates online kernel characterization, while Minimizing Communication Conflicts uses a GNN‑based estimator for NoC conflicts. However, there is no unified analytical or learning‑based model that simultaneously captures (a) compute heterogeneity (GPU vs. multiple PIM microarchitectures), (b) memory‑tier latency/capacity trade‑offs (as explored in PIM‑STM), and (c) inter‑tile communication patterns. Existing evaluations are limited to per‑benchmark case studies, leaving system‑wide design‑space exploration under‑instrumented.

2. Programming abstractions that bridge transactional, graph, and ML workloads

   • PIM‑STM provides a TM library for UPMEM, and Optimal Data Allocation for Graph Processing proposes graph‑aware partitioning. Yet, there is no common runtime or language extension that can express both transactional semantics and graph‑centric data movement while also exposing the dual‑mode GEMV primitives of AttenPIM. The lack of a unified programming model hampers composability of heterogeneous kernels.

3. Robust evaluation of reliability and variability in mixed‑signal / analog PIM

   • X‑PIM models non‑idealities (ADC quantization, parasitics) but validates only on three exemplar PIM technologies. A PulseWidth‑IN‑PulseWidth‑OUT Universal Nonlinear Processing Element introduces a novel analog nonlinear unit without a systematic study of process variation, temperature drift, or endurance across large arrays. Consequently, the community lacks a standardized methodology for quantifying reliability impact on end‑to‑end application accuracy.

4. Memory management for fine‑grained dynamic workloads

   • PIM‑Malloc shows a 66× speedup for allocation, yet its evaluation focuses on static benchmark kernels. Real‑world workloads (e.g., LLM decoding in PAPI or k‑Clique counting in A Combined CAM and In‑Memory Processing Approach) exhibit highly dynamic allocation patterns (temporary buffers, per‑token structures). The interaction between dynamic allocators and runtime scheduling (e.g., token‑level concurrency in AttenPIM) remains unexplored.

💡 High‑Impact Open Problems

1. Unified Heterogeneous PIM Performance Model (U‑HPM)
   Problem Statement: Develop a hierarchical performance model that jointly predicts (i) compute latency/throughput across diverse PIM microarchitectures (digital, mixed‑signal, CAM‑augmented), (ii) memory‑tier access latency and bandwidth (DRAM‑based PIM, HMC, SRAM‑based PIM), and (iii) NoC communication contention. The model should be trainable on a modest set of micro‑benchmarks and generalize to unseen workloads (e.g., LLM decoding, graph analytics, k‑Clique).
   Research Tasks:

   • Formalize a parametric representation of kernel characteristics (compute intensity, memory‑access pattern, synchronization frequency).
   • Integrate a GNN‑based NoC contention estimator (extending the approach of Minimizing Communication Conflicts) with analytical models for per‑tile compute pipelines.
   • Validate against full‑system simulations of at least three heterogeneous platforms (GPU+PIM, NoC‑based PIM, CAM‑PIM hybrid).
     Impact: Enables systematic design‑space exploration, compiler auto‑tuning, and runtime scheduling decisions across the entire PIM ecosystem.

2. Transactional‑Graph‑ML Runtime (TG‑MLR) for PIM
   Problem Statement: Design a runtime that unifies transactional memory semantics, graph‑aware data placement, and ML‑specific tensor operations (including dual‑mode GEMV). The runtime should expose a high‑level API (e.g., extensions to C++/Rust) that automatically maps code regions to the most suitable PIM unit (digital, analog, CAM) while handling dynamic allocation via PIM‑Malloc.
   Research Tasks:

   • Define a language extension or library that annotates transactional regions, graph kernels, and tensor GEMV kernels.
   • Implement a scheduler that leverages online kernel characterization (PAPI) and conflict‑graph analysis (Minimizing Communication Conflicts) to decide placement.
   • Demonstrate end‑to‑end speedup on a mixed workload (e.g., a graph‑based recommendation model that uses transactional updates and attention layers).
     Impact: Bridges the current fragmentation of PIM software stacks, allowing developers to compose complex pipelines without hand‑tuned mapping.

3. Reliability‑Aware Mixed‑Signal PIM Design Space Exploration
   Problem Statement: Construct a methodology that quantifies the trade‑off between analog non‑idealities (quantization noise, device variability) and algorithmic accuracy for mixed‑signal PIM accelerators (as in X‑PIM and PIPO‑UNPE). The methodology should guide designers to select voltage, timing, and precision knobs that meet application‑level error budgets.
   Research Tasks:

   • Extend the compressed equivalent model (CEM) to incorporate statistical process variation models for ReRAM/CMOS devices.
   • Develop a Monte‑Carlo simulation framework that propagates analog errors through full DNN inference pipelines (including attention and k‑Clique counting).
   • Produce design guidelines (e.g., required ADC resolution vs. energy budget) for at least two target applications (MLP on MNIST, k‑Clique counting).
     Impact: Provides the first systematic, quantitative bridge between circuit‑level variability and system‑level performance/accuracy, essential for deploying mixed‑signal PIM in safety‑critical domains.