LACE: Layer-Wise Compression for Dynamic Frame Rate Codecs
This research improves the efficiency of neural audio codecs, which could lead to better performance in applications like text-to-speech systems.
A new method called LACE (Layer-Adaptive Codec Encoding) was proposed to improve neural audio codecs.
It applies independent compression steps at each quantization layer, allowing layer-specific segmentation boundaries.
This approach was tested on LibriTTS and showed better rate-quality tradeoff than prior methods.The code is released as part of the ESPnet3 codec recipe.
Neural audio codecs are a key component in speech language modeling. However, their high frame rates lead to long sequence lengths, increasing computational costs. Dynamic frame rate codecs mitigate this by reducing the effective frame rate using a compression step to merge multiple frames together. However, most prior methods either operate on single-codebook codecs or apply a single compression step before multi-layer quantization. This forces all quantization layers to share the same segmentation boundaries, despite the residual embeddings at different quantization layers exhibiting different rates of change over time. We propose LACE (Layer-Adaptive Codec Encoding), a dynamic frame rate codec that applies an independent compression step at each quantization layer, enabling layer-specific segmentation boundaries. To use LACE tokens in downstream text-to-speech (TTS), we further introduce union alignment and boundary anchor mechanisms to make durations consistent across layers while preserving compression benefits. Experiments on LibriTTS show that LACE offers a better rate-quality tradeoff than prior dynamic frame rate methods on the reconstruction task and improves TTS inference efficiency while maintaining competitive synthesis quality. Our code is released as part of the ESPnet3 codec recipe.
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include LACE, dynamic frame rate codecs, audio codecs, rate-quality tradeoff.
Source attribution: curated from arXiv cs.AI (@Thanapat Trachu, Samuele Cornell, William Chen, Shinji Watanabe).