英文标题：PELM: Power Efficient On-Device LLM Inference with Speculative Decoding and Dynamic Voltage Frequency Scaling
作者：Weisi Yang, Stephen Xia
arXiv ID：2609.09662 | 分类：cs.LG | 发表：2026-09-09
许可：CC-BY
摘要。 将大型语言模型（LLM）直接部署在边缘侧的移动平台上，由于其在隐私性、个性化以及延迟等方面的诸多优势，正获得大量关注。然而，LLM具有沉重的计算需求，资源受限的移动与边缘平台难以满足。除了有限的计算资源外，移动与边缘系统通常具有紧凑的外形尺寸，缺乏物理机制来消散高处理器使用率所产生的热量（例如风扇），以防止降频和算力下降，而LLM很容易引发这些问题。为缓解这些影响，先前工作提出了各种功耗调控策略，例如动态电压频率调节（DVFS），以降低移动平台上重计算任务的功耗和发热。近来，针对移动LLM定制的DVFS也已被提出。然而，这些方法大多聚焦于优化硬件参数和处理器频率，在某些受热约束的场景下
Deploying Large Language Models (LLMs) directly on mobile platforms at the edge is gaining traction due to a myriad of benefits, such as increased privacy, personalization, and reduced latency. However, LLMs have heavy computational requirements, which are difficult for resource-constrained mobile and edge platforms to fulfill. In addition to limited compute resources, mobile and edge systems often have a compact form factor and lack physical mechanisms to dissipate heat generated from high processor usage rates (e.g., fans) to prevent throttling and reduced processing power, which LLMs can easily cause. To mitigate these effects, prior works have proposed various power governing strategies, such as dynamic voltage and frequency scaling (DVFS), for reducing power and heat generation for heavy computational tasks on mobile platforms. Recently, DVFS methods tailored for mobile LLMs have also been proposed. However, these methods mostly focus on optimizing hardware parameters and processor frequencies, and they fall short under some thermally constrained scenarios. Drawing from recent advances in machine learning, we identify and take advantage of the key insight that not all tokens require full-depth inference to maintain high-quality generation. Motivated by this, we present PELM, a solution that augments traditional DVFS processor frequency tuning with two additional workload-specific knobs: 1) speculative decoding and 2) variable verification depth to expand the optimization space to multiple dimensions for more power efficient on-device LLM inference. In extensive evaluations across hardware platforms and datasets, PELM demonstrates superior performance compared to state-of-the-art power governing methods, with up to 23.1% speedup and 52.4% reduction in energy consumption, while maintaining comparable task performance. The source code is available at https://github.com/imec-nu/PELM.
查看完整双语翻译 →
正在跳转到翻译阅读页… 如果没有自动跳转，请点击这里。