semi·news
Headlines要闻 / Research研究 / /
Research digest · Monday, November 9, 2026 研究摘要 · 2026年11月9日 星期一

Locality and Scheduling Turn Hardware Capacity Into Throughput 数据局部性与调度将硬件能力转化为吞吐量

This week's selected preprints examine how die placement, precision delivery and communication scheduling affect realized performance. GPU measurements, a Tensix simulator and circuit-partitioning results connect software execution choices to hardware constraints. 本期预印本关注裸片间的数据放置、精度数据传输和通信调度如何影响实际性能。GPU实测、Tensix模拟器与电路划分结果,从不同层面揭示软件执行方式与硬件约束之间的联系。

Look-back window: 7 days · 9 paper(s) 回溯窗口: 7天 · 9篇

AI Accelerators & Compute-in-Memory AI加速器与存算一体

Characterizing CUDA Locality Domains on Blackwell GPUs Blackwell GPU上CUDA局部性域的性能特征分析

Rocky Wang, Joyce Zhou, Keith Jiang, et al.

arXiv:2611.07535 · 2026-11-05T06:44:09Z

Measurements on an eight-B300 node show that CUDA 13.4 Locality Domains can improve performance through die-aware placement and execution, with reported speedups up to 43.5% for NCCL AllGather and 37% for DeepGEMM EP. The study also identifies allocation overhead above 32 MB associated with barriers and TLB invalidation. These preprint results make cross-die locality a concrete tuning consideration, with gains dependent on access patterns and workload. 在配备8颗B300的节点上,研究通过感知裸片位置的数据放置与执行方式,评估CUDA 13.4局部性域,报告NCCL AllGather和DeepGEMM EP的最高性能提升分别为43.5%和37%。研究还发现,超过32 MB的内存分配存在与屏障和TLB失效相关的额外开销。这些预印本实测结果说明,跨裸片局部性已成为具体的调优因素,但收益取决于访问模式和工作负载。

Wolfpine: A RISC-V Simulator for Tenstorrent Tensix Wolfpine:面向Tenstorrent Tensix的RISC-V模拟器

Nick Brown, Daniyal Arshad

arXiv:2611.04740 · 2026-11-03T11:57:36Z

Wolfpine simulates Tenstorrent's Wormhole and Blackhole Tensix architectures in Python and integrates with TT-Metal. Built from public documentation, open software and hardware measurements, it provides component tracing, typed events and timeline export for investigating execution bottlenecks. The preprint lowers the barrier to architecture exploration, while timing fidelity is added incrementally through an optional cost model. Wolfpine以Python模拟Tenstorrent的Wormhole与Blackhole两代Tensix架构,并与TT-Metal集成。它基于公开文档、开源软件和硬件测量构建,提供组件追踪、类型化事件和时间线导出,便于定位执行瓶颈。该预印本工具降低了架构探索门槛,时序精度则通过可选的成本模型逐步补充。

AI Models & Inference Systems AI模型与推理系统

StrataFlux: Memory-Hierarchy-Native Low-Bit LLM Inference StrataFlux:利用存储层次结构的低比特大模型推理

Jake Hyun, Ziyu Zhao, Jae-sun Seo, et al.

arXiv:2611.08363 · 2026-11-05T17:42:06Z

StrataFlux streams selected precision refinements from host RAM and SSDs to improve low-bit weights and KV caches while overlapping transfers with computation. For 2-bit Llama-3.1-8B-Instruct weights on RTX 5090, it reduces WikiText-2 perplexity from 9.46 to 8.40 while retaining 95.4% of the quantized baseline's throughput. The preprint shows how the memory hierarchy can recover quantization quality, with the throughput result tied to the reported model and platform. StrataFlux从主机内存与SSD中选择性传输用于精度修正的数据,并将传输与计算重叠,以改善低比特权重和KV缓存的精度。在RTX 5090上,针对采用2比特权重的Llama-3.1-8B-Instruct,它将WikiText-2困惑度从9.46降至8.40,同时保留量化基线95.4%的吞吐量。该预印本展示了利用存储层次结构弥补量化精度损失的路径,上述吞吐表现对应其报告的特定模型与平台。

ZYFA: Speed and Accuracy of FP8 FlashAttention on AMD CDNA4 ZYFA:AMD CDNA4上FP8 FlashAttention的速度与精度优化

Quentin Anthony, Ganesh Bikshandi, Praneeth Medepalli, et al.

arXiv:2611.07496 · 2026-11-05T05:58:36Z

ZYFA develops MI355X prefill kernels that combine direct FP8 probability computation, ping-pong scheduling of softmax and matrix multiplication, and power-of-two probability scaling. The motivation is a measured utilization gap: at 32K tokens, the cited AITER kernels reach 39% of peak FP8 matrix throughput versus 49% in BF16. The preprint addresses vector-unit work and instruction ordering that limit attention performance, so the headline FP8 arithmetic advantage alone is insufficient to predict speed. ZYFA面向MI355X开发prefill内核,将直接计算FP8概率、交替执行softmax与矩阵乘法的乒乓调度,以及按2的幂缩放概率结合起来。其出发点是实测利用率差距:在32K token下,文中引用的AITER内核达到FP8矩阵峰值吞吐的39%,BF16则为49%。该预印本针对限制注意力性能的向量单元计算与指令排序开销展开优化,表明仅凭FP8算术峰值优势不足以预测实际速度。

KOSMOS: Self-Scheduling Persistent Kernels for Communication-Compute Overlap KOSMOS:以自调度常驻内核实现通信与计算重叠

Alexander Magro, Aristotle Martin, Ye Wang, et al.

arXiv:2611.07477 · 2026-11-05T05:36:08Z

KOSMOS lets GPU workgroups switch between communication and computation as data becomes ready, using consumer-demand-ordered queues inside persistent kernels. On MI350X and MI355X, the preprint reports 1.10–1.28× end-to-end BF16 TP+SP training speedups in Megatron-LM and 1.29–1.83× speedups over DeepEP for DeepSeek-V3 and Qwen3-235B proxy workloads. The contribution targets idle gaps from fixed schedules; the MoE results apply to the evaluated proxy training configurations. KOSMOS在常驻内核中按消费者需求组织队列,使GPU工作组随数据就绪情况,在通信与计算任务之间切换。该预印本在MI350X和MI355X上报告,Megatron-LM的BF16张量并行加序列并行训练端到端加速为1.10至1.28倍;针对DeepSeek-V3与Qwen3-235B的代理工作负载,相对DeepEP加速1.29至1.83倍。其贡献在于减少固定调度造成的空闲间隙,MoE性能数字对应所评估的代理训练配置。

FastAFD Measures Attention-FFN Disaggregation on Rack-Scale NVLink FastAFD评估机架级NVLink上的Attention与FFN解耦

Yichao Fu, Yuxuan Zhang, Minshen Zhang, et al.

arXiv:2611.07149 · 2026-11-04T21:21:57Z

FastAFD separates attention and MoE expert execution on GB200 NVL72 and overlaps the work of two microbatches. Against the best tested vLLM wide-expert-parallel configuration, it reaches up to 1.17× throughput on Qwen3 and 1.16× on Qwen3.5, while matching the baseline on MiniMax-M2.5. An analytical model predicts throughput within 3% in the reported evaluation, offering a pre-deployment check of whether disaggregation pays for a particular workload. FastAFD在GB200 NVL72上分离注意力与MoE专家执行,并将两个微批次的工作重叠。相对测试中最优的vLLM大规模专家并行配置,Qwen3和Qwen3.5的最高吞吐提升分别为1.17倍和1.16倍,MiniMax-M2.5则与基线持平。文中分析模型在所报告评估中的吞吐预测误差不超过3%,可用于在部署前判断特定工作负载是否适合解耦。

NUMA-Torch: Compiler-Driven NUMA-Aware LLM Prefill on Multi-Socket CPUs NUMA-Torch:以编译器优化多路CPU上的NUMA感知大模型prefill

Thisara Rathnayaka Weerakoon, Ishan Dewpura, Niman Ransindu, et al.

University of Moratuwa / Independent Researcher

arXiv:2611.07105 · 2026-11-04T20:17:40Z

NUMA-Torch makes TorchInductor partition weights and compute by CPU socket, keeping thread pools and memory allocations local through CPU affinity. The approach targets cross-socket weight traffic that reduces effective DRAM bandwidth during prefill. The preprint evaluates GPT-2 variants from 117M to 762M parameters and BERT Small and Large, so extrapolating its findings to much larger models requires further measurement. NUMA-Torch改造TorchInductor,按CPU插槽划分权重与计算,并通过CPU亲和性将线程池和内存分配限制在本地。该方法针对prefill阶段跨插槽读取权重、导致有效DRAM带宽下降的问题。预印本评估了参数量从1.17亿到7.62亿的GPT-2变体,以及BERT Small和Large,将结论推广到更大模型仍需要进一步测量。

FastKVComp Raises KV Compression Throughput for Disaggregated Serving FastKVComp提升解耦推理服务的KV缓存压缩吞吐量

Junhong Wu, Zedong Liu, Xinyang Ma, et al.

arXiv:2611.04382 · 2026-11-03T06:13:07Z

FastKVComp fuses distribution transformation and low-bit quantization in a TileLang kernel and replaces KVServe's lossless backend with a searchable compression pipeline. The preprint reports up to 5.9× speedup for the fused kernel and lossless-compression throughput rising from below 90 GB/s to over 500 GB/s. Faster compression can preserve the benefit of reduced KV transfers on high-bandwidth links, although these component metrics do not establish the same gain in end-to-end serving throughput. FastKVComp以TileLang内核融合分布变换与低比特量化,并为KVServe换用可搜索的无损压缩流水线。该预印本报告融合内核最高加速5.9倍,无损压缩吞吐量从不足90 GB/s提升至超过500 GB/s。更快的压缩有助于在高带宽互连下保留减少KV传输的收益,但这些组件指标并不代表端到端推理吞吐量也有同等提升。

EDA & Design Automation EDA与设计自动化

FRESHPart: Feedback-Reshaped Spectral Embedding for Hypergraph Partitioning FRESHPart:通过反馈重塑谱嵌入的超图划分方法

Rongjian Liang, Zhuo Feng, Brucek Khailany

arXiv:2611.05729 · 2026-11-03T23:00:57Z

FRESHPart uses geometry and search feedback to reshape spectral embeddings for circuit-hypergraph partitioning. On Titan23, the preprint reports average cut reductions relative to hMETIS of 13.09% at 2% imbalance tolerance and 25.21% at 20%, completing each benchmark in under 120 seconds on one GPU. The result offers a practical EDA partitioning improvement; the reported metric is partition cut quality rather than final placed-and-routed chip performance. FRESHPart利用几何反馈与搜索反馈重塑谱嵌入,改进电路超图划分。在Titan23基准上,该预印本报告:不平衡容限为2%和20%时,相对hMETIS的平均割代价分别降低13.09%和25.21%,每项基准均在单颗GPU上于120秒内完成。结果为EDA划分提供了实用改进,其指标衡量划分质量,尚不能直接推导最终布局布线后的芯片性能。

Newsletter 邮件订阅

Daily semiconductor briefing. 每日半导体简报。