Inference Systems, Models & Quantization
推理系统、模型与量化
Benjamin Novotny, Doug Szeto, Chang Wang, et al.
arXiv:2611.24891 · 2026-11-18T18:34:10Z
Cap-and-Track uses fast admission refusals to track available inference capacity, holding excess and retried work in durable priority queues outside execution. In a concurrent production A/B test at Meta, the traffic shaper served a classifier on 43% fewer H100s than its queued, fixed-rate predecessor, while lower-priority work waited longer during backfill peaks. Real-engine replay at steady capacity put goodput within 1% of a fixed rate tuned above measured capacity. The production result concerns asynchronous classifier serving rather than a general reduction in interactive LLM hardware needs.
Cap-and-Track利用快速准入拒绝信号追踪可用推理容量,将超额任务及待重试任务保存在执行路径之外的持久化优先队列中。在Meta的同期生产A/B测试中,该流量整形器运行一个分类器时,比采用队列与固定分发速率的前代方案少用43%的H100,但低优先级任务在回填高峰时等待更久。在稳定容量下的真实引擎回放中,其有效吞吐与调至高于实测容量的固定速率方案相差不到1%。生产结果针对异步分类服务,不能直接推广为交互式大模型所需硬件的普遍降幅。
Disen Lan, Xuanda Wang, Jianbin Zheng, et al.
arXiv:2611.24809 · 2026-11-18T17:45:51Z
FlashMorph2 converts pretrained Transformers into hybrid full- and linear-attention models by evaluating discrete architectures on their own hidden states and jointly updating layer-selection scores and weights. Across tested Qwen3 scales and linear-attention variants, average scores improve over FlashMorph by 2.42 points on RULER, 1.08 on LongBench and 0.76 on recall-intensive tasks. Layer selection takes 0.72–3.57 GPU-hours for 0.6B–8B models. These results address selection quality and cost; the supplied excerpt does not give an end-to-end serving speedup.
FlashMorph2通过使用离散架构自身产生的隐藏状态进行评估,并联合更新层选择分数与权重,将预训练Transformer转换为结合完整注意力和线性注意力的模型。在所测试的Qwen3规模与线性注意力变体上,相比FlashMorph,RULER、LongBench及高召回需求任务的平均分分别提高2.42、1.08和0.76分。对于6亿至80亿参数模型,层选择耗时为0.72至3.57 GPU小时。这些结果针对选择质量与成本,所提供的摘要尚未给出端到端推理服务加速比。
Robert Debbas, Éric Goubault, Sylvie Putot
arXiv:2611.24648 · 2026-11-18T15:52:07Z
This work verifies quantized ReLU networks with a zonotope-based abstract interpretation framework that handles rounding and clamping jointly. On ACAS Xu at the two smallest tested perturbation radii, constrained zonotopes reduce average error-bound widths by about 70–85% and runtimes by 65–80% relative to QEBVerif's symbolic DRA propagation. Plain zonotopes produce competitive sound bounds nearly 300 times faster on average. Gains vary with bitwidth, and symbolic DRA yields tighter bounds at larger radii, making the method's advantage regime-dependent.
研究提出基于zonotope(带状多面体)的抽象解释框架,联合处理舍入与截断,以验证量化ReLU网络。在ACAS Xu基准的两个最小测试扰动半径下,相比QEBVerif的符号DRA传播,约束zonotope将平均误差边界宽度缩小约70%至85%,运行时间减少65%至80%。普通zonotope方法也能给出有竞争力的可靠边界,平均速度接近快300倍。收益随位宽而变化,较大扰动半径下符号DRA的边界更紧,因此优势取决于具体参数区间。
DiffuSpace Technical Team
arXiv:2611.24636 · 2026-11-18T15:46:14Z
Clover-35B-A3B-Preview is a diffusion language model with 35B total parameters and 3B activated parameters, combining recurrent memory for prior context with bidirectional attention inside each generated block. The authors report scores of 73.6 on SWE-bench Verified and 51.7 on Terminal-Bench 2.1. On internal agent workloads, generation reaches a reported 680 tokens/s on one Nvidia GB300 at BF16 and batch size one. That throughput is tied to the stated internal workload and does not establish performance across serving conditions.
Clover-35B-A3B-Preview是一款总参数350亿、激活参数30亿的扩散语言模型,将历史上下文的循环记忆与当前生成块内的双向注意力结合。作者报告,其SWE-bench Verified和Terminal-Bench 2.1得分分别为73.6和51.7。在内部智能体负载上,单颗英伟达GB300以BF16精度、批量大小为1运行时,报告的生成速度达到每秒680个词元。该吞吐对应指定的内部负载,不能据此判断各种服务条件下的表现。
Firas Spanioli, Yaniv David
arXiv:2611.24619 · 2026-11-18T15:37:04Z
GateWatch expresses activation-monitor decisions as Boolean logic over a small set of binarized sparse-autoencoder features. GateWatchServe computes only selected features, streams binary values across layers and evaluates probes through compact lookup tables. Across 24 Gemma-2-9B tasks, the authors report 80.49% mean test accuracy with averages of 3.8 features and 3.5 logic gates per probe. The compact rule size is promising for monitoring cost, but the supplied excerpt does not quantify end-to-end inference overhead.
GateWatch将激活监测器的决策表示为少量二值化稀疏自编码器特征上的布尔逻辑。GateWatchServe只计算选中的特征,跨层流式传递二值结果,并通过紧凑查找表评估探针。在Gemma-2-9B的24项任务上,作者报告平均测试准确率为80.49%,每个探针平均仅使用3.8个特征与3.5个逻辑门。紧凑规则有助于降低监测成本,但所提供的摘要尚未量化端到端推理开销。