Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Huggingface-Aishell get wrong prediction #2549

Open
Ht-zhang-xianyu opened this issue May 16, 2024 · 2 comments
Open

Huggingface-Aishell get wrong prediction #2549

Ht-zhang-xianyu opened this issue May 16, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@Ht-zhang-xianyu
Copy link

Ht-zhang-xianyu commented May 16, 2024

Describe the bug

Use the pretrained from huggingface case in Aishell dataset, but the prediction seems strange
The script I use is as follows:

`
from speechbrain.inference.ASR import EncoderDecoderASR

asr_model = EncoderDecoderASR.from_hparams(source="speechbrain/asr-transformer-aishell", savedir="pretrained_models/asr-transformer-aishell")
asr_model.transcribe_file("speechbrain/asr-transformer-aishell/example_mandarin2.flac")`

but return the predict token is :
"一 日 一一 一一 一一 六 克一 件 第 一"

Expected behaviour

expect the correct asr script of wav

To Reproduce

No response

Environment Details

No response

Relevant Log Output

No response

Additional Context

No response

@Ht-zhang-xianyu Ht-zhang-xianyu added the bug Something isn't working label May 16, 2024
@LudwigFancy
Copy link

LudwigFancy commented May 19, 2024

i meet the same question, then i use this pretrained model to test another examples which is created by myself, but more errors happened. i don't understand whether the data need a preprocess when we test?(I'm a novice at this)

@asumagic
Copy link
Collaborator

asumagic commented Jun 5, 2024

Seems like develop is even more broken due to the mutihead_attn rename, resulting in some layer weights not being loaded correctly:

loading from pretrained_models/asr-transformer-aishell/asr.ckpt, the object could not use the parameters loaded with the key: 1.decoder.layers.5.mutihead_attn.att.out_proj.bias

#2529 solves that.

That being said, the current issue above is separate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants