AI Accelerators & Compute-in-Memory
AI加速器与存算一体
J. Lee, I.-J. Jung, J.-Y. Kim
arXiv:2607.01617 · 2026-07-02T02:32:44Z
3DLS proposes a logic-on-logic 3D-stacked chiplet architecture for LLM serving that separates prefill-to-decode KV-cache transfers from decode-side tensor-parallel collectives. By routing KV traffic through vertical interconnects and keeping collectives on lateral die-to-die links, it reduces mixed-traffic contention. The reported gains reach up to 1.49x throughput and 60.2% lower end-to-end latency versus a shared-fabric planar baseline.
3DLS提出一种面向LLM服务的逻辑-逻辑3D堆叠chiplet架构，把prefill到decode的KV-cache传输与decode侧tensor parallel通信分离。它通过垂直互连承载KV流量，同时把collective通信保留在横向die-to-die链路上，从而降低混合流量竞争。论文报告相对共享平面互连基线最高可实现1.49倍吞吐，并降低60.2%的端到端延迟。
W. Zhou, C. Ding, M. Liu, et al.
arXiv:2607.01607 · 2026-07-02T02:14:32Z
MxGLUT targets LLM inference where FP8 activations meet low-bit compressed weights and conventional accelerators need separate datapaths. The design uses mixed-precision LUT-based processing elements and a reconfigurable broadcast dataflow to support FP8-INT4 and FP8-FP8 GEMMs under one execution mechanism. Synthesized in 28 nm CMOS at 200 MHz, it is a useful architecture study for reducing multiplier-heavy area in inference accelerators.
MxGLUT面向LLM推理中FP8 activation与低比特压缩权重混合计算的场景，解决传统加速器需要独立数据通路的问题。该设计使用混合精度LUT处理单元和可重构broadcast dataflow，在同一执行机制下支持FP8-INT4与FP8-FP8 GEMM。其在28 nm CMOS、200 MHz下综合验证，是减少推理加速器中乘法器面积开销的有价值架构研究。
J. Oh, C.-T. Lin, M. Seok
IEEE TCAS I DOI:10.1109/TCSI.2026.3665839 · 2026-07-01
D6CIM demonstrates an all-digital 6T-SRAM compute-in-memory macro supporting 1-to-8-bit fixed-point vector-matrix multiplication. The 28 nm CMOS prototype reaches about 360 MHz at 1.1 V, up to 60.4 TOPS/W, 1.46 TOPS/mm2 compute density, and 1005 kb/mm2 weight density. The key contribution is balancing energy efficiency, compute density, and weight density without relying on analog sensing margins.
D6CIM展示了一种全数字6T-SRAM存算一体宏，支持1到8 bit定点向量-矩阵乘法。其28 nm CMOS原型在1.1 V下约360 MHz运行，最高达到60.4 TOPS/W、1.46 TOPS/mm2计算密度和1005 kb/mm2权重密度。关键贡献在于不依赖模拟感测裕量，同时平衡能效、计算密度和权重密度。
M. Pasotti, R. Zurla, J. J. B. Agnoletto, et al.
IEEE JSSC DOI:10.1109/JSSC.2026.3693894 · 2026-07-01
This JSSC paper presents a phase-change-memory analog IMC macro fabricated in 28 nm FD-SOI CMOS with a 4M-cell array and eight 512-by-512-weight layers. The design emphasizes multilayer computation that is resilient to drift and temperature, reporting matrix-vector multiplication error below 2.14% from -40 C to +125 C. It is notable because dense non-volatile storage is directly tied to usable DNN compute, not just a standalone memory metric.
这篇JSSC论文展示了一个基于phase-change memory的模拟IMC宏，采用28 nm FD-SOI CMOS制造，集成4M cell阵列和8个512乘512权重层。设计重点是具备漂移和温度鲁棒性的多层计算，在-40 C到+125 C范围内报告矩阵-向量乘法误差低于2.14%。其意义在于把高密度非易失存储直接转化为可用的DNN计算能力，而不只是展示单独的存储指标。
LLM Inference & AI Systems
LLM推理与AI系统
W. Li, J. Dong, H. Zhang, et al.
arXiv:2606.31519 · 2026-06-30
RaBitQCache attacks the long-context KV-cache bottleneck with randomized rotated binary quantization and binary-INT4 arithmetic for estimating attention weights. Instead of a fixed Top-k budget, it uses an adaptive Top-p retrieval rule backed by an error-bounded proxy score. The system additions, including asynchronous pipelining and lazy updates, are important because KV compression only helps if its own overhead is hidden during decoding.
RaBitQCache用随机旋转二值量化和binary-INT4算术估计attention权重，以处理长上下文KV-cache瓶颈。它不使用固定Top-k预算，而是基于带误差界的代理分数执行自适应Top-p检索。系统层面的异步流水和lazy update同样关键，因为KV压缩只有在自身开销能被decode过程隐藏时才真正有效。
A. Abaskohi, G. Carenini, P. West, Y. He
arXiv:2606.31145 · 2026-06-30
SeKV keeps long-context information in a GPU-CPU hierarchy instead of simply evicting tokens from the KV cache. It groups context into entropy-guided semantic spans, stores lightweight summaries on GPU, and keeps low-rank SVD bases on CPU for query-triggered reconstruction. The approach is compelling because it preserves a path back to token-level detail when a compressed span becomes relevant later in generation.
SeKV不是简单从KV cache中驱逐token，而是把长上下文信息保存在GPU-CPU分层结构中。它将上下文组织为熵引导的语义span，在GPU上保留轻量摘要，并在CPU上保存低秩SVD基，用于查询触发的重构。该方法有吸引力，因为当压缩span在生成后续阶段重新变得相关时，系统仍能恢复到token级细节。