Inference Systems Meet New Memory Devices 推理系统遇上新型存储器件

With arXiv rate-limited today, the research queue leans on Semantic Scholar. The useful through-line is still strong: LLM serving is fighting KV-cache and scheduling overheads while device papers push FeFET, ReRAM, 2T0C DRAM and measured accelerator silicon. 今天arXiv被限流，研究队列主要来自Semantic Scholar。主线仍然清楚：LLM服务正在处理KV缓存和调度开销，而器件论文则推进FeFET、ReRAM、2T0C DRAM和实测加速器硅片。

Look-back window: 7 days · 10 paper(s) 回溯窗口: 7天 · 10篇

LLM Inference Systems 大模型推理系统

SparDA: Sparse Decoupled Attention for Efficient Long-Context LLM Inference SparDA：面向长上下文LLM推理的稀疏解耦注意力 LLM inferenceKV cachesparse attention

Yaosheng Fu, Guangxuan Xiao, Xin Dong, Song Han, Oreste Villa

arXiv:2606.04511 · 2026-06-03

SparDA adds a forecast projection to predict which KV blocks the next layer will need, allowing sparse-attention offload to overlap CPU-to-GPU prefetch with current-layer execution. The reported gains, including up to 1.7x decode speedup over a sparse-attention offload baseline, are directly relevant to the long-context memory wall. The caveat is that the method assumes model changes and training of the forecast projection, so deployment friction depends on how easily serving teams can adopt modified checkpoints. SparDA增加forecast投影来预测下一层需要哪些KV块，使稀疏注意力卸载能把CPU到GPU的预取与当前层执行重叠。论文报告相对稀疏注意力卸载基线最高1.7倍decode加速，直接对应长上下文内存墙。局限是该方法需要模型结构改动并训练forecast投影，因此部署摩擦取决于服务团队能否接受修改后的checkpoint。

FlexNPU: Transparent NPU Virtualization for Dynamic LLM Prefill-Decode Co-location FlexNPU：用于LLM prefill-decode动态共置的透明NPU虚拟化 NPUvirtualizationLLM serving

Jiongjiong Gu, Jianfeng Wang, Zidong Han, Yongqiao Wang, Pengfei Xia, Mingjie Zhang, Hong Liu, Yuanyi Xia, Jiajia Chu, Yifeng Tang, Hui Zang, Xin Yao

arXiv:2606.04415 · 2026-06-03

FlexNPU inserts a user-space virtualization layer around Ascend NPUs to schedule LLM prefill and decode phases without modifying model code, frameworks or drivers. That matters because prefill is compute-heavy while decode is memory/KV-cache-heavy, and static disaggregation can waste resources. The work is a useful reminder that accelerator utilization is now a runtime systems problem as much as a chip architecture problem. FlexNPU在Ascend NPU周围加入用户态虚拟化层，在不修改模型代码、框架或驱动的前提下调度LLM prefill和decode阶段。这很重要，因为prefill偏计算密集，而decode偏内存和KV缓存密集，静态拆分容易造成资源浪费。该工作提醒我们，加速器利用率现在既是芯片架构问题，也是运行时系统问题。

NetKV: Network-Aware Decode Instance Selection for Disaggregated LLM Inference NetKV：面向解耦LLM推理的网络感知decode实例选择 LLM servingnetworkingKV cache

M. Ojewale

arXiv:2606.03910 · 2026-06-02

NetKV treats network transfer of KV cache as part of the time-to-first-token budget in disaggregated LLM serving. The scheduler uses a network cost oracle and reports up to 21.2% lower mean TTFT in a 64-GPU fat-tree simulation. The paper is important because disaggregation only helps if the datacenter network does not become the new hidden memory bus. NetKV把解耦LLM服务中KV缓存跨网络传输纳入首token延迟预算。其调度器使用网络成本oracle，并在64-GPU fat-tree仿真中报告平均TTFT最高降低21.2%。这篇论文重要，因为解耦架构只有在数据中心网络不变成新的隐藏内存总线时才真正有收益。

DayPQ: Dynamic Layerwise Pruning and Quantization for LLM Inference Acceleration DayPQ：用于LLM推理加速的动态逐层剪枝与量化 quantizationVLSILLM acceleration

Yongchao Xu, Mingxuan Li, Ziying Zhuo, Qinzhe Zhi, Tianyu Jia, Meiqi Wang

IEEE Transactions on VLSI Systems · 2026-06-01

DayPQ co-designs adaptive pruning and FP8-style quantization with a multiplication-free shift-add PE array and switchable dataflow. The point is not just compression; it is dynamic compression with low hardware overhead and negligible extra latency. This is the kind of algorithm-hardware coupling likely to matter as LLM inference moves onto more constrained accelerators. DayPQ把自适应剪枝和类似FP8的量化，与无乘法shift-add PE阵列和可切换数据流协同设计。重点不只是压缩，而是在低硬件开销和几乎无额外延迟下做动态压缩。随着LLM推理进入更多受限加速器，这类算法-硬件耦合会变得重要。

Measured Accelerators & Compute-in-Memory 实测加速器与存算一体

A 40-nm Sub-mJ/Transfer HDC-SNN Hybrid Processor for On-Chip Few-Shot Transfer Learning 40nm亚毫焦HDC-SNN混合处理器，用于片上少样本迁移学习 measured siliconSNNIoT

Anqin Xiao, Jinqiao Yang, Wanlin Yang, Xin Zhang, Yuhan He, Ziyi Yang, Lirong Zheng, Zhuo Zou

IEEE Transactions on Circuits and Systems for Artificial Intelligence · 2026-06-01

This paper reports a fabricated 40-nm hybrid processor combining hyperdimensional computing and spiking neural networks for on-chip few-shot transfer learning. The measured-silicon angle makes it stronger than a pure architecture proposal, and the sub-mJ/transfer claim points at small IoT workloads where retraining in the cloud is unattractive. The question is how broadly the HDC-SNN representation transfers beyond the evaluated neuromorphic datasets. 这篇论文报道了一颗40nm实测混合处理器，结合超维计算和脉冲神经网络，用于片上少样本迁移学习。实测硅片让它比纯架构方案更有分量，亚毫焦级迁移能耗也指向不适合云端重训练的小型IoT工作负载。后续问题是HDC-SNN表示能否扩展到评估数据集之外的更多任务。

A 16-nm FinFET CNN Processor for Small-Object Detection on High-Resolution Video 16nm FinFET CNN处理器，用于高分辨率视频小目标检测 JSSCvision acceleratorFinFET

Yu-Chun Ding, Chia-Yu Chang, Hui-Yun Tsai, Chun-Yeh Lin, Hao-Jiun Tu, Yu-Ching Su, Kai-Feng Chang, Tsung-Han Hsieh, Wen-Ching Chen, Nian-Shyang Chang, Chun-Pin Lin, Chi-Shi Chen

IEEE Journal of Solid-State Circuits · 2026-06-01

This JSSC paper presents a 16-nm FinFET processor optimized for high-resolution small-object detection, with BiFPN-aware dataflow and memory scheduling. The relevance is practical: ADAS and robotics need high-resolution perception, but off-chip memory traffic can dominate energy and latency. The design's value is in tailoring the processor to the feature-pyramid workload instead of treating vision inference as generic convolution. 这篇JSSC论文提出一颗16nm FinFET处理器，针对高分辨率小目标检测优化，并包含面向BiFPN的数据流和存储调度。其实用性很强：ADAS和机器人需要高分辨率感知，但片外存储访问会主导能耗和延迟。该设计的价值在于针对特征金字塔工作负载定制处理器，而不是把视觉推理当作通用卷积。

NeRF-Learner: A 22-nm NeRF-SLAM Accelerator With Unified Inference and Training Compute-in-Memory NeRF-Learner：22nm NeRF-SLAM加速器，统一推理与训练存算一体 JSSCcompute-in-memoryrobotics

Yiqi Jing, Jiaqi Zhou, Yiyang Sun, Meng Wu, Yufei Ma, Tianyu Jia, Le Ye

IEEE Journal of Solid-State Circuits · 2026-06-01

NeRF-Learner reports a 22-nm accelerator for simultaneous neural rendering and SLAM, using unified inference/training compute-in-memory and memory-aware ray scheduling. The measured numbers, including 42.6 FPS rendering and 11.47 TFLOPS/W at BF16 precision, make it a serious edge-robotics silicon result. It also shows how training-in-the-loop workloads are pushing CIM beyond static inference. NeRF-Learner报道了一颗22nm加速器，用于同步神经渲染和SLAM，并采用统一推理/训练存算一体和存储感知ray调度。42.6 FPS渲染和BF16下11.47 TFLOPS/W等实测数字，让它成为严肃的边缘机器人硅片结果。它也说明，带在线训练的工作负载正在把CIM推过静态推理场景。

Memory Devices 存储器件

Multi-Level GAA FeFET by Multi-Ferroelectric/Dielectric Gate Stacks 通过多铁电/介电栅堆栈实现多级GAA FeFET FeFETGAAnonvolatile memory

Wei-Jen Chen, Tao Chou, Ching-Wang Yao, B. Huang, Min-Kuan Lin, Yun-Wen Chen, Yu-Rui Chen, Jhih-Yuan Liang, F. I. C. W. Liu

IEEE Electron Device Letters · 2026-06-01

This IEEE EDL paper demonstrates a gate-all-around FeFET with four threshold-voltage levels, a 3 V memory window, 20 ns write time and endurance above 1e8 cycles. The use of multi-ferroelectric/dielectric stacks within a 400 C thermal budget makes it relevant to advanced integration discussions. Multi-level FeFETs remain challenging, but the combination of speed, endurance and GAA geometry is a meaningful device result. 这篇IEEE EDL论文展示了一种GAA FeFET，实现四级阈值电压、3 V存储窗口、20 ns写入时间和超过1e8次循环耐久。多铁电/介电堆栈并保持400 C热预算，使其与先进集成讨论相关。多级FeFET仍很困难，但速度、耐久和GAA几何结构的组合是有意义的器件结果。

NanoCiM: Nano-ReRAM Computation-in-Memory With Weight-Aware X-Address Scrambling NanoCiM：采用权重感知X地址扰动的纳米ReRAM存算一体 ReRAMCIMIR drop

C. Matsui, Ayumu Yamada, Shota Suzuki, Naoko Misawa, Ken Takeuchi

IEEE Transactions on Circuits and Systems for Artificial Intelligence · 2026-06-01

NanoCiM attacks IR-drop-induced bit-line current variation in nanoscale ReRAM computation-in-memory arrays by scrambling row addresses based on neural-network weights. That is a very practical circuit problem: analog CIM accuracy often falls apart because interconnect and pattern-dependent current errors are not idealized away in silicon. The paper is useful because it treats mapping and circuit non-idealities as one problem. NanoCiM通过基于神经网络权重扰动行地址，抑制纳米ReRAM存算一体阵列中由IR drop导致的位线电流变化。这是非常实际的电路问题：模拟CIM精度常常因为互连和数据模式相关电流误差而在硅片上崩掉。该论文的价值在于把映射和电路非理想性作为一个问题处理。

Unlocking the Performance Limits of 2T0C DRAM With Omega-Gated Single-Crystal In2O3 FETs 用Omega栅单晶In2O3 FET突破2T0C DRAM性能限制 2T0C DRAMoxide semiconductormemory

Shiyue Zuo, Z. Xie, Xu Zhao, Lin Tang, Yidan Zhao, Yu Zeng, Wencheng Niu, Sen Liu, Qingjiang Li, Guanhua Yang, Ling Li, Zhipeng Wei

Science Advances · 2026-05-29

This Science Advances paper reports omega-gated single-crystal In2O3 FETs and uses them to build capacitorless 2T0C DRAM. The reported cell metrics include 5 ns access, 1e12-cycle endurance, 1e5 s retention and 3-bit storage, which makes it one of the stronger memory-device entries in today's queue. The caveat is manufacturability: impressive device metrics still need a credible path to array yield and process integration. 这篇Science Advances论文报道Omega栅单晶In2O3 FET，并用其构建无电容2T0C DRAM。论文给出的单元指标包括5 ns访问、1e12次耐久、1e5 s保持和3-bit存储，是今天研究队列中较强的存储器件结果之一。局限在可制造性：亮眼器件指标仍需要可信的阵列良率和工艺集成路径。