# Attentive Temporal Pooling for Conformer-based Streaming Source: https://savrn.com/papers/attentive-temporal-pooling-for-conformer-based-streaming-language-identification-in-long-f Markdown alternate of the page above; the site index is https://savrn.com/llms.txt --- ## Abstract In this paper, we introduce a novel language identification system based on conformer layers. We propose an attentive temporal pooling mechanism to allow the model to carry information in long-form audio via a recurrent form, such that the inference can be performed in a streaming fashion. Additionally, we investigate two domain adaptation approaches to allow adapting an existing language identification model without retraining the model parameters for a new domain. We perform a comparative study of different model topologies under different constraints of model size, and find that conformer-based models significantly outperform LSTM and transformer based models. Our experiments also show that attentive temporal pooling and domain adaptation improve model accuracy. [Full paper on arXiv](https://arxiv.org/abs/2202.12163) ## Details arXiv identifier 2202.12163 Published 2022-02-24 Authors Quan Wang, Yang Yu, Jason Pelecanos, Yiling Huang, Ignacio Lopez Moreno ## Open Models Built on This Paper Every model in the SAVRN Model Hub whose card cites this paper, most downloaded first, with what it takes to run each one. | Model | Task | Size | License | Monthly downloads | Cheapest setup at 16-bit | | --- | --- | --- | --- | --- | --- | | [univ_conformer_small_ge2e](https://savrn.com/models/univ-conformer-small-ge2e) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 31 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [univ_lstm_small_ge2e](https://savrn.com/models/univ-lstm-small-ge2e) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 29 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [univ_ecapa_tdnn_small_ge2e](https://savrn.com/models/univ-ecapa-tdnn-small-ge2e) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 25 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [ls_lstm_small_ge2e](https://savrn.com/models/ls-lstm-small-ge2e) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 25 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [ls_conformer_small_ge2e](https://savrn.com/models/ls-conformer-small-ge2e) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 23 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [ls_ecapa_tdnn_small_ge2e](https://savrn.com/models/ls-ecapa-tdnn-small-ge2e) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 22 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [univ_conformer_tiny_ge2e](https://savrn.com/models/univ-conformer-tiny-ge2e) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 693,124 | apache-2.0 | 14 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [univ_conformer_small_dr_vectors](https://savrn.com/models/univ-conformer-small-dr-vectors) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 13 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [univ_conformer_small_arcface](https://savrn.com/models/univ-conformer-small-arcface) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 13 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [cn_conformer_small_pfas](https://savrn.com/models/cn-conformer-small-pfas) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 13 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [vox_ecapa_tdnn_small_ext_softmax](https://savrn.com/models/vox-ecapa-tdnn-small-ext-softmax) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 12 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [univ_transformer_small_ge2e](https://savrn.com/models/univ-transformer-small-ge2e) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 1M | apache-2.0 | 12 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [univ_mamba_small_ge2e](https://savrn.com/models/univ-mamba-small-ge2e) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 1M | apache-2.0 | 12 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [univ_ecapa_tdnn_tiny_ge2e](https://savrn.com/models/univ-ecapa-tdnn-tiny-ge2e) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 626,720 | apache-2.0 | 12 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [univ_ecapa_tdnn_base_ge2e](https://savrn.com/models/univ-ecapa-tdnn-base-ge2e) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 3M | apache-2.0 | 12 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [univ_conformer_small_pfas_ge2e](https://savrn.com/models/univ-conformer-small-pfas-ge2e) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 12 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [univ_conformer_small_ext_softmax](https://savrn.com/models/univ-conformer-small-ext-softmax) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 12 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [univ_conformer_small_pfas](https://savrn.com/models/univ-conformer-small-pfas) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 12 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [ls_resnet_small_ge2e](https://savrn.com/models/ls-resnet-small-ge2e) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 12 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [ls_transformer_small_ge2e](https://savrn.com/models/ls-transformer-small-ge2e) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 1M | apache-2.0 | 12 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [ls_conformer_small_pfas_ge2e](https://savrn.com/models/ls-conformer-small-pfas-ge2e) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 12 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [cn_conformer_small_ge2e](https://savrn.com/models/cn-conformer-small-ge2e) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 12 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [ls_conformer_small_arcface](https://savrn.com/models/ls-conformer-small-arcface) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 12 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [univ_resnet_small_ge2e](https://savrn.com/models/univ-resnet-small-ge2e) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 11 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [univ_conformer_small_triplet](https://savrn.com/models/univ-conformer-small-triplet) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 11 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [univ_conformer_small_sphereface](https://savrn.com/models/univ-conformer-small-sphereface) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 11 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [univ_conformer_small_cosface](https://savrn.com/models/univ-conformer-small-cosface) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 11 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [ls_ecapa_tdnn_tiny_ge2e](https://savrn.com/models/ls-ecapa-tdnn-tiny-ge2e) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 626,720 | apache-2.0 | 11 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [ls_mamba_small_ge2e](https://savrn.com/models/ls-mamba-small-ge2e) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 1M | apache-2.0 | 11 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [ls_conformer_small_sphereface](https://savrn.com/models/ls-conformer-small-sphereface) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 11 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [ls_conformer_tiny_ge2e](https://savrn.com/models/ls-conformer-tiny-ge2e) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 693,124 | apache-2.0 | 11 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [cn_conformer_small_dr_vectors](https://savrn.com/models/cn-conformer-small-dr-vectors) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 11 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [cn_ecapa_tdnn_small_arcface](https://savrn.com/models/cn-ecapa-tdnn-small-arcface) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 11 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [ls_conformer_small_cosface](https://savrn.com/models/ls-conformer-small-cosface) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 11 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [ls_conformer_small_dr_vectors](https://savrn.com/models/ls-conformer-small-dr-vectors) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 11 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [univ_conformer_base_ge2e](https://savrn.com/models/univ-conformer-base-ge2e) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 5M | apache-2.0 | 10 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [ls_ecapa_tdnn_base_ge2e](https://savrn.com/models/ls-ecapa-tdnn-base-ge2e) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 3M | apache-2.0 | 10 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [ls_conformer_small_pfas](https://savrn.com/models/ls-conformer-small-pfas) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 10 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [ls_conformer_small_triplet](https://savrn.com/models/ls-conformer-small-triplet) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 10 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | | [vox_conformer_small_ge2e](https://savrn.com/models/vox-conformer-small-ge2e) Quan | [Audio classification](https://savrn.com/models/tasks/audio-classification) | 2M | apache-2.0 | 9 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr | ## Explore More - [All research papers](https://savrn.com/papers) - [The model directory](https://savrn.com/models) - [Datasets](https://savrn.com/datasets) ## Source - arXiv metadata, read 2026-10-18. - [How the hub is built](https://savrn.com/model-hub/methodology)