AI Accelerators & Compute-in-Memory
AI加速器与存算一体
Maximilian Thürmer, Nicolas Scotto di Perto, Eloise Stein, et al.
arXiv:2611.05104 · 2026-11-03T16:00:07Z
This optical circuit-switching design configures links once per model deployment and co-designs collective schedules with model parallelism. For a modeled 576-accelerator domain serving 5T-parameter MoE models with contexts up to 5M tokens, it reports up to 65% better interactivity and 119% better energy efficiency than a 4:1 oversubscribed packet-switched fabric. The gains come from GXNET workload emulation and analytical and flit-level network models grounded in hardware measurements, making oversubscription and communication schedules key evaluation assumptions.
这项光路交换设计在每次模型部署时配置一次链路,并将集合通信调度与模型并行方式协同优化。在模拟的576颗加速器域中,运行5万亿参数MoE模型、上下文最长500万token时,相较4:1超额订阅的分组交换网络,交互性能最高提升65%,能效最高提升119%。这些结果来自GXNET工作负载仿真及以硬件测量为依据的分析模型、flit级网络模型,超额订阅比例与通信调度是理解收益的关键评估条件。
Pietro Bonazzi, Luigi Capogrosso, Ahmet Celik, et al.
arXiv:2611.03392 · 2026-11-02T16:28:53Z
A benchmark evaluates four CNNs and two small transformers across ten commercial edge devices, including sensor-integrated processing and MCUs with and without NPUs. For the reported energy-above-idle measurements, Sony's IMX500 uses 0.13–0.35 mJ versus 0.31–1.05 mJ for STM32N6; different models also favor GAP9 or Alif E7. The workload-dependent rankings show why platform selection needs measured latency and energy alongside operation counts, with the quoted energy figures excluding idle consumption.
这项基准在十款商用边缘器件上评估了四种CNN与两种小型Transformer,覆盖传感器内处理器以及有、无NPU的MCU。按高于空闲基线的增量能耗计算,Sony IMX500为0.13–0.35 mJ,STM32N6为0.31–1.05 mJ;部分模型则以GAP9或Alif E7表现更优。排名随工作负载变化,说明平台选型需要结合实测时延与能耗评估运算量,上述能耗数值未计入空闲消耗。
Mohamed Allam, Thomas Spendlhofer, Hugo Sanz-González, Antonio J. Peña
arXiv:2611.03325 · 2026-11-02T15:49:14Z
An NTT/INTT kernel mapped across the AMD Alveo V80's three dies reports processing a 65,536-coefficient polynomial every 5.12 microseconds at 400 MHz. Its fully unrolled delay-commutator streaming structure addresses the routing overhead that can grow with parallelism on multi-die FPGAs. This targets a major cost in homomorphic-encryption inference, with application speedup also depending on transfers and the remaining FHE pipeline.
这项NTT/INTT内核映射到AMD Alveo V80的三个芯粒,在400 MHz下每5.12微秒可处理一个含65,536个系数的多项式。完全展开的延迟交换器流式结构针对多芯粒FPGA中随并行度上升的布线开销。该设计瞄准同态加密推理的一项主要计算成本,应用整体加速比还取决于数据传输及FHE流水线的其他部分。
AI Research & Inference Systems
AI研究与推理系统
Zimeng Wang, Jinrong Yang, Rong Chen
arXiv:2611.04253 · 2026-11-03T03:44:38Z
UMLoad reduces average on-demand model loading time by 36.6–71.9% against unmodified loaders and 19.9–62.7% against pipelined loaders in tests across Intel, AMD, and NVIDIA unified-memory platforms. It separates logical allocations from physical page backing, eliminating weight copies through on-demand reconstruction and accelerating allocation with GPU-assisted page zeroing. Mean time to first token in LLM-routing tests falls 51.3–59.3%, making loader design relevant to interactive latency; the evaluation covers six models and the tested software stacks.
在Intel、AMD与NVIDIA统一内存平台的测试中,UMLoad相较原始加载器将平均按需模型加载时间缩短36.6–71.9%,相较流水化加载器缩短19.9–62.7%。它将逻辑分配与物理页后备解耦,通过按需重建消除权重复制,并借助GPU辅助页清零加速内存分配。LLM路由测试的平均首token时延降低51.3–59.3%,显示加载器设计会显著影响交互响应;评估范围为六个模型及所测试的软件栈。
Yutong Lin, Yingdi Shan, Shaoyuan Chen, et al.
arXiv:2611.03173 · 2026-11-02T14:09:55Z
PagedMoE stores each SSD-resident MoE expert in a prefix-compatible representation, allowing cached low-precision data to be extended by fetching only missing bitplanes. It selects precision before I/O using routing weights and expert-specific error profiles, aiming to avoid repeated transfers when requested bit widths change. The design targets DRAM-limited local inference, with practical throughput and quality depending on cache capacity, SSD behavior, and the precision-selection policy.
PagedMoE将驻留SSD的每个MoE专家存成前缀兼容的表示,使已缓存的低精度数据能够通过补取缺失位平面扩展到更高精度。系统利用路由权重与各专家的误差特征,在I/O发生前选择精度,力求避免所需位宽变化时重复搬运相同数据。该设计面向DRAM容量受限的本地推理,实际吞吐量与模型质量取决于缓存容量、SSD特性及精度选择策略。