semi·news
Headlines要闻 / Research研究 / /
Research digest · Thursday, November 19, 2026 研究摘要 · 2026年11月19日 星期四

Inference Efficiency Depends on the Whole System 推理效率取决于整个系统

Eight papers connect quantization and accelerator design with interconnects, production workloads and evaluation methods. The reading queue combines conference publications with preprints and distinguishes measured results from architectural proposals. 本期8篇论文将量化和加速器设计,与互连、生产负载及评测方法联系起来。阅读清单同时纳入会议论文和预印本,并区分实测结果与架构方案。

Look-back window: 7 days · 8 paper(s) 回溯窗口: 7天 · 8篇

Accelerator Architecture & Systems 加速器架构与系统

Chimera Co-Designs FPGA Compute, HBM Access and Floorplan for LLM Decode Chimera协同设计FPGA计算、HBM访问与布局以支持大模型解码

Zhi-Heng Chen, Yi-Fan Zhang, Yuseon Chou, et al.

arXiv:2611.21677 · 2026-11-14

Chimera co-designs compute lanes, HBM access and placement across the dies of an FPGA, using HBM Proxy Ports to reach compute in every super logic region. On an AMD Alveo U280, the authors build ten designs for eight models, including an 80B-parameter MoE model using four memory tiers. The work addresses the routing and resource limits that prevent raw HBM bandwidth from becoming useful decode throughput; the supplied abstract does not include comparative performance figures. Chimera在多裸片FPGA上协同设计计算通路、HBM访问与布局,通过HBM Proxy Ports让各个超级逻辑区域中的计算资源都能访问HBM。作者在AMD Alveo U280上为8个模型构建了10种设计,其中包括使用4级存储的800亿参数MoE模型。这项工作针对布线和资源约束,解决HBM标称带宽难以充分转化为解码吞吐的问题;所提供的摘要尚未包含对比性能数字。

Mutable Inference Shares Accelerator Resources Between Prediction and Learning Mutable Inference在预测与学习之间共享加速器资源

Yi-Min Gao, Liang-Tao Dai, Melika Morsali, et al.

arXiv:2611.18523 · 2026-11-13

Learning While Serving shares accelerator compute between latency-critical prediction and more flexible inference-time learning. It exploits matrix capacity left idle by non-matrix attention operations and uses per-layer weight versioning so updates can coexist with in-flight computations. The architecture targets models that learn from context during inference, making it relevant to hardware beyond fixed-weight decoding; no silicon measurement is established by the supplied abstract. Learning While Serving让延迟敏感的预测与时间安排更灵活的推理时学习共享加速器计算资源。它利用注意力非矩阵运算期间闲置的矩阵计算能力,并通过逐层权重版本管理,使权重更新与尚未完成的计算并存。该架构面向推理过程中持续从上下文学习的模型,将硬件设计问题扩展到固定权重解码之外;所提供的摘要尚未给出芯片实测结果。

Two DGX Sparks Favor Different Parallelism at Different Loads 双DGX Spark在不同负载下适合不同并行方式

Kevin Bhimani, Christian Escobar, Daniel Mihovch, et al.

arXiv:2611.20289 · 2026-11-15

Across 17 checkpoints on one or two DGX Spark GB10 systems linked by 200 GbE, tensor parallelism delivered a median 1.59× peak-throughput gain over one system, versus 1.39× for pipeline parallelism. Tensor parallelism also cut unloaded inter-token latency to 0.52–0.73× of the single-system value. For models fitting on one system, replication generally offered higher throughput at high concurrency, so the best way to add a second machine depends on the serving load. 研究在1台或2台通过200 GbE互连的DGX Spark GB10上测试了17个模型检查点;相较单机,张量并行的峰值吞吐增益中位数为1.59倍,流水线并行为1.39倍。张量并行还将无负载条件下的词元间延迟降至单机的0.52至0.73倍。对于单机能够容纳的模型,高并发下复制部署通常提供更高吞吐,因此第二台机器的最佳使用方式取决于服务负载。

Quantization, KV Cache & Inference Systems 量化、KV缓存与推理系统

Beyond Bitwidth Tests Mixed Precision Against Real Blackwell Deployment Limits Beyond Bitwidth按Blackwell实际部署约束评测混合精度量化

Hyojung Han

arXiv:2611.21974 · 2026-11-16

For Qwen3-8B on an Nvidia B200 with vLLM, a configuration combining 98 NVFP4 weight-only modules, 46 INT8 modules and an INT8 output head passed the study's held-out quality and deployment gates. With 16-bit activations, it used 5.917 GiB for weight loading and achieved a time-per-output-token ratio of 0.939 relative to BF16. All seven tested INT2 grids degenerated during generation, showing why nominal bitwidth alone is a poor deployment criterion. The passing configuration remains a result for this model, hardware and evaluation set. 研究在英伟达B200与vLLM上测试Qwen3-8B,将98个仅权重采用NVFP4的模块、46个INT8模块及INT8输出头组合后,通过了独立保留测试集上的质量与部署门槛。在激活保持16位的条件下,该配置的权重加载内存为5.917 GiB,每输出词元耗时为BF16基线的0.939倍。测试的7组INT2配置均出现生成退化,说明名义位宽不足以作为部署标准。通过门槛的结果仍限于这一模型、硬件和评测集。

RoShift Turns Activation Sparsity into Measured Decode Speedups RoShift将激活稀疏性转化为实测解码加速

Geunsoo Song, Jiwon Seo, Hoeseok Yang, et al.

arXiv:2611.20549 · 2026-11-16

RoShift combines a deferred activation shift with weight-aware rotation to create sparsity without training, then fuses the required corrections to preserve decode efficiency. Across five models, it improves average accuracy over TEAL by 1.60 and 3.29 percentage points at 50% and 60% sparsity, respectively. On an RTX 5090, the authors report up to 1.72× decoding speed over the dense model at 50% sparsity. The measured speedup connects skipped weight reads to runtime, but its maximum is specific to the tested configuration. RoShift通过延迟执行补偿的激活平移与权重感知旋转,在无需训练的情况下提高稀疏性,并融合必要的补偿运算以保持解码效率。在5个模型上,50%和60%稀疏度下的平均准确率分别比TEAL高1.60和3.29个百分点。作者报告,在RTX 5090上以50%稀疏度运行时,解码速度最高为稠密模型的1.72倍。实测加速将减少权重读取与运行时间联系起来,但最高收益仅对应所测试的具体配置。

HACK Computes Attention Directly on INT2 KV Cache HACK直接在INT2 KV缓存上计算注意力

Ze-Yu Zhang, Hai-Ying Shen, S. Vargaftik, et al.

ACM SIGOPS Annual Technical Conference 2026 · 2026-11-14

HACK keeps KV data in compressed INT2 form for transfer and caching, and computes attention directly on that representation in disaggregated inference. This targets both the network cost of moving KV between prefill and decode and the repeated dequantization cost of conventional low-bit caching. The supplied abstract establishes the mechanism but does not expose quantitative accuracy or end-to-end latency results. HACK在分离式推理中,以压缩后的INT2格式传输和缓存KV数据,并直接在该表示上计算注意力。它同时针对预填充与解码阶段之间搬运KV的网络成本,以及传统低位宽缓存反复反量化的开销。所提供的摘要说明了技术机制,但尚未给出量化精度或端到端延迟的具体结果。

Production MaaS Trace Maps Resource Allocation Across More Than 100,000 GPUs 生产级MaaS轨迹分析覆盖逾10万颗GPU的资源分配

Yin Zheng, Ling-Yun Yang, Ying-Hao Yu, et al.

ACM SIGOPS Annual Technical Conference 2026 · 2026-11-14

A six-month trace from the xMaaS production platform covers more than 100,000 GPUs, 300,000 applications and 70,000 model variants. The study examines cluster-wide resource supply, changing application demand and deployment choices across heterogeneous GPUs, extending characterization beyond individual requests. This scale makes the workload evidence useful for judging serving-system assumptions, although the supplied excerpt does not report utilization rates or a demonstrated scheduling gain. 研究分析xMaaS生产平台长达6个月的运行轨迹,覆盖逾10万颗GPU、30万个应用和7万个模型变体。它从整个集群层面考察资源供给、应用需求变化和异构GPU上的部署选择,将负载刻画从单个请求扩展到资源分配。这一规模的生产数据有助于检验推理服务系统的设计假设,但所提供的摘要尚未报告利用率数字或已验证的调度收益。

LLM Inference Evaluation Study Exposes Baseline and Metric Pitfalls 大模型推理评测研究揭示基线与指标设计缺陷

Amey Agrawal, Nitin Kedia, Anmol Agarwal, et al.

ACM Symposium on Cloud Computing 2026, Volume 1 · 2026-11-17

This study identifies recurring inference-evaluation problems in baseline fairness, workload setup and metric design. It explains how implementation differences can obscure algorithmic contributions and how normalized metrics can hide variability in autoregressive token delivery. The analysis provides a useful checklist for assessing accelerator and serving claims, particularly when prefill, decode and user-visible latency are combined into a single headline number. 这项研究梳理了推理评测在基线公平性、负载设置和指标设计上的常见问题。它说明,实现细节的差异可能掩盖算法本身的贡献,而归一化指标也可能隐藏自回归词元输出的波动。这些分析有助于审视加速器与推理服务的性能主张,尤其适用于预填充、解码及用户感知延迟被合并成单一宣传数字的情况。

Newsletter 邮件订阅

Daily semiconductor briefing. 每日半导体简报。