Basic Information

Email Address

• Email Address

  li.sheng.7xkyoto-u.ac.jp

Professional Memberships

• From Dec. 2023, To Present

  APNNS (Asia Pacific Neural Network Society)
• APSIPA (Asia Pacific Signal and Information Processing Association)
• SIG-CSLP (Chinese Spoken Language Processing)
• ASJ (日本音響学会)
• ISCA (International Speech Communication Association)
• IEEE/IEEE-SPS/IEEE-RAS
• ACM (Association for Computing Machinery)

Academic Degree

• 01 Jul. 2009

  南京大学修士（工程学）
• 23 Mar. 2016

  京都大学博士（情報学）

Academic Resume (Graduate Schools)

• 京都大学, 大学院情報学研究科博士後期課程知能情報学専攻, 修了
• 南京大学, 大学院ソフトウェア工程学専攻修士課程, 修了

Academic Resume (Undergraduate School/Majors)

• 南京大学, 計算機科学技術専攻, 卒業

Research History

• From Mar. 2025, To Present

  Institute of Science Tokyo, Assistant Professor
• From Apr. 2026

  Kyoto University, Project-Specific Assistant Professor
• From Oct. 2025

  RIKEN Visiting Scientist
• From 2020, To Feb. 2025

  National Institute of Information and Communications Technology (NICT), Advanced Speech Technology Laboratory (ASTL), Tenure-track Researcher
• From Nov. 2024

  Eindhoven University of Technology (TU/e), visiting assistant professor
• From Feb. 2024, To Mar. 2024

  Nanyang Technological University, visiting researcher
• From Dec. 2021, To Mar. 2023

  Kyoto University, master course advisor
• From Apr. 2019, To May 2019

  Oxford University, Computer science department, visiting researcher
• From 2017, To 2019

  National Institute of Information and Communications Technology (NICT), Advanced Speech Technology Laboratory (ASTL), researcher (hired by Tokyo Olympic2020 project)
• From Apr. 2016, To Dec. 2016

  Kyoto University, Speech and Audio Processing Lab., researcher (hired by Erica Humanoid robot project)
• From Apr. 2012, To Sep. 2012

  Sogou/Sohu Pinyin IME [Beijing, China], researcher (working on speech input)
• From Jul. 2009, To Apr. 2012

  Shenzhen Institute of Advanced Technology [Shenzhen, Guangdong China], researcher (computer-assisted language learning)

Profile

• Profile

  Sheng LI received his BS and ME degrees in 2006 and 2009, respectively, from Nanjing University, Nanjing, China, and his Ph.D. in 2016 from Kyoto University, Kyoto, Japan. From 2009 to 2012, he worked at the joint lab of the Chinese University of Hong Kong and Shenzhen City, researching speech technology-assisted language learning. From 2016 to 2017, he worked as a researcher at Kyoto University, studying speech recognition systems for humanoid robots. From 2017 to Feb. 2025, he was a researcher at the National Institute of Information and Communications Technology in Kyoto, Japan, working on speech-to-speech translation. In March 2025, he was an assistant professor at the Institute of Science, Tokyo. From April 2026, he was hired by both the Institute of Science and Kyoto University as an assistant professor working on speech recognition. He is also a visiting scientist at RIKEN.

  
  

  He served as a workshop/special session co-organizer and session chair in Interspeech2020, COLING2022, Odyssey2022, ACM Multimedia Asia2023/2024, RO-MAN2025, IROS2025, and ICASSP2024/2026. He is a member of the Acoustical Society of Japan (ASJ) and the International Speech Communication Association (ISCA), and a senior member of IEEE. He is now a member of the Speech, Language, and Audio (SLA) Technical Committee for APSIPA. He is also a member of the Applied Signal Processing Systems Technical Committee (ASPS TC) of the IEEE Signal Processing Society (SPS).
  
  Focusing on 

  
  

  next-generation multilingual speech recognition,  translation, and synthesis technology

  
  

  security-aware speech processing technology

  
  

  robot audition/embodied voices

  
  

  https://search.star.titech.ac.jp/titech-ss/pursuer.act?event=outside&key_t2r2Rid=CTT100930321&lang=jp

  
  

  https://educ.titech.ac.jp/ict/eng/faculty/

  
  

  https://youtu.be/pP6YtlSVqlM

ID,URL

• ORCID ID

  0000-0001-7636-3797

• J-Global ID

  201801014603986279

• Researcher ID

  AGF-2329-2022

• Researcher number

  70840940

• External link

  https://halspeech.github.io/index.html

Website(s) (URL(s))

• https://researchmap.jp/listen
• https://halspeech.github.io

researchmap URL

• https://researchmap.jp/listen

Research

Research Interests

• large language model (speech, text)
• security-aware speech processing
• multimodal speech process
• computer assisted language learning
• Speech Recognition/translation

Research Areas

• Informatics, Perceptual information processing

Papers

• Matrix-product-state Compression of Quantum-encoded Vocal-tract Transfer Functions

  Sheng Li; Jiyi Li

  JASA Express Letter, Sep. 2026, Peer-reviewed

  
• Retain-Free Machine Unlearning for Speech Emotion Recognition

  Zhao Ren; Rathi Adarshi Rammohan; Kevin Scheck; Thanh Tam Nguyen; Sheng Li; Tanja Schultz

  IEEE MLSP, Dec. 2026, Peer-reviewed

  
• Neural TTS-based Dynamic Data Augmentation for Improved Speech Separation.

  K. Wang; L. Yin; S. Li; M. Mansurova; H. Huang

  European Signal Processing Conference (EUSIPCO), Sep. 2026

  
• TSA-LLM: A Survey of Time-Step Alignment on Speech Large Language Models

  Sheng Li; Jiyi Li; Takahiro Shinozaki

  Communications in Computer and Information Science, 12 Aug. 2026, Peer-reviewed

  
• Combining CLAP Embeddings for Improving Text-Audio Alignment Correspondence Scoring

  Sheng Li; Jiyi Li; Takahiro Shinozaki

  ICML 2026 Workshop on Machine Learning for Audio, Jul. 2026, Peer-reviewed

  
• SpeechBench4Edge: Seamlessly Upgrading On-Device Speech Interface with More Recent Foundation Models

  Sheng Li; Takahiro Shinozaki

  The 23rd International Conference on Ubiquitous Robots, Jul. 2026

  
• Spolacq-GDS: A generative dialogue simulator for spoken interaction learning

  Taisei Awashima; Renon Toyosaki; Koki Mikuriya; Kota Kawakita; Sheng Li; Takahiro Shinozaki

  The Journal of the Acoustical Society of America, 01 Oct. 2025, Peer-reviewed

  
• Casting Everything to Online API Services? A Survey of Integrating Localized Speech Recognition Models in Robotic Systems

  Sheng Li; Jing Li; Felix Schijve; Jun Hu; Emilia Barakova

  International Conference on Social Robotics (ICSR), Jul. 2026, Peer-reviewed, Lead author, Corresponding author

  
• Evaluating ASR-LLM Setups for Japanese Speech Recognition with Multi-Pass Augmented Generative Error Correction

  Yuka Ko; Sheng Li; Chao-Han Huck Yang; Tatsuya Kawahara

  ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 03 May 2026, Peer-reviewed, Corresponding author

  
• Expressive Voice Conversion with Controllable Emotional Intensity

  Nannan Teng; Ying Hu; Zhijian Ou; Sheng Li

  ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 03 May 2026, Peer-reviewed, Last author

  
• What Should Automated Vehicles Communicate to Human Drivers? Prioritizing External Human-Machine Interface Information Based on the Four-Sides Model

  Di Zhou; Guanghui Zhang; Tianqi Peng; Sheng Li

  International Journal of Human–Computer Interaction, 07 Apr. 2026, Last author, Corresponding author

  
• Unified multi-prototype network with pretrained swin transformer for visual and audio open set recognition

  Haiyan Yang; Sheng Li; Juncheng Li; Jun Shi; Jun Wang

  Signal, Image and Video Processing, 16 Jan. 2026, Peer-reviewed

  
• Speech Foundation Bench for Robotic and EdgeAI systems

  Sheng Li; Takahiro Shinozaki

  Proc. IEEE-ICASSP demo 2026., 2026, Peer-reviewed

  
• Emotion-aware Speech Translation Correction with Large Language Models

  Zhengdong Yang; Sheng Li; Chenhui Chu

  Journal of Natural Language Processing, 2026, Peer-reviewed

  
• Designing an LLM-Powered Social Robot for Supporting Emotion Regulation in Parent-Child Dyads

  Jing Li; Felix Schijve; Sheng Li; Emilia Barakova; Jun Hu

  Lecture Notes in Computer Science, 02 Jul. 2026

  
• Emotional Text-To-Speech Based on Mutual-Information-Guided Emotion-Timbre Disentanglement

  Jianing Yang; Sheng Li; Takahiro Shinozaki; Yuki Saito; Hiroshi Saruwatari

  APSIPA ASC 2025, Dec. 2025

  
• Designing an LLM-powered Social Robot for Supporting Emotion Regulation In Parent-Child Dyads

  Jing Li; Sheng Li; Emilia I. Barakova; Felix Schijve; Jun Hu

  Proc. RO-MAN (late breaking), Dec. 2025

  
• Towards Emotion Co-regulation with LLM-powered Socially Assistive Robots: Integrating LLM Prompts and Robotic Behaviors to Support Parent-Neurodivergent Child Dyads

  Jing Li; Felix Schijve; Sheng Li; Yuye Yang; Jun Hu; Emilia Barakova

  2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 19 Oct. 2025, Peer-reviewed

  
• LatentSpeech: Latent Diffusion for Text-To-Speech Generation

  Haowei Lou; Hye young Paik; Pari Delir Haghighi; Sheng Li; Wen Hu; Lina Yao

  Proc. RO-MAN, Dec. 2025, Peer-reviewed, Invited

  
• Language-Aware Prompt Tuning for Parameter-Efficient Seamless Language Expansion in Multilingual ASR,

  Hongli Yang; S. Li; Hao Huang; Ayiduosi Tuohan; Yizhou Peng

  Proc. Interspeech, Dec. 2025, Peer-reviewed

  
• Adapting Whisper for Parameter-efficient Code-Switching Speech Recognition via Soft Prompt Tuning

  Hongli Yang; Yizhou Peng; Hao Huang; S. Li

  Proc. Interspeech, Dec. 2025, Peer-reviewed

  
• Simple and Effective Content Encoder for Singing Voice Conversion via Dimension Reduction,

  Wangjin Zhou; Tianjiao Du; Chenglin Xu; S. Li; Yi Zhao; Tatsuya Kawahara

  Proc. Interspeech, Dec. 2025, Peer-reviewed

  
• SIQ: Exterminating Speech Intelligence Quotient Cross Cognitive Levels in Voice Understanding Large Language Models

  Zhen Wan; Chao-Han Huck Yang; Yahan Yu; Jinchuan Tian; Sheng Li; Ke Hu; Zhehuai Chen; Shinji Watanabe; Fei Cheng; Chenhui Chu; Sadao Kurohashi

  Proc. ACL (long main), Dec. 2025, Peer-reviewed

  
• Generative Error Correction for Emotion-aware Speech-to-text Translation

  Zhengdong Yang; Sheng Li; Chenhui Chu

  Proc. ACL (findings), Dec. 2025, Peer-reviewed

  
• Bandwidth Extension System for Throat Microphone Speech Reconstruction

  Yu Xu; Xiaokai Qin; Tianyu Fan; Eng Siong Chng; Sheng Li; Nobuaki Minematsu; Daisuke Saito

  Proc. IEEE-ICME, Dec. 2025, Peer-reviewed

  
• End-to-end Acoustic-linguistic Emotion and Intent Recognition Enhanced by Semi-supervised Learning

  Zhao Ren; Rathi Adarshi Rammohan; Kevin Scheck; Sheng Li; Tanja Schultz

  International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Dec. 2025, Peer-reviewed

  
• Empowering Māori Automatic Speech Recognition through EMD-Based Augmentation

  Chengxi Lei; Sheng Li; Satwinder Singh; Feng Hou; Huia Jahnke; Ruili Wang

  22nd Pacific Rim International Conference on Artificial Intelligence (PRICAI 2025), Nov. 2025

  
• Joint Automatic Speech Recognition And Structure Learning For Better Speech Understanding

  Jiliang Hu; Zuchao Li; Mengjia Shen; Haojun Ai; Sheng Li; Jun Zhang

  IEEE-ICASSP, Mar. 2025, Peer-reviewed

  
• Similarity-based accent recognition with continuous and discrete self-supervised speech representations

  Jun-You Wang; Sheng Li; Li-An Lu; Sydney Chia-Chun Kao; Jyh-Shing Roger Jang

  IEEE-ICASSP, Mar. 2025, Peer-reviewed

  
• Extending Whisper for Emotion Prediction Using Word-level Pseudo Labels

  Chin Yuen Kwok; Sheng Li; Jia Qi Yip; Chenhui Chu; Tatsuya Kawahara; Eng Siong Chng

  IEEE-ICASSP, Mar. 2025, Peer-reviewed

  
• SpeechIQ: Speech-Agentic Intelligence Quotient Across Cognitive Levels in Voice Understanding by Large Language Models.

  Zhen Wan; Chao-Han Huck Yang; Yahan Yu; Jinchuan Tian; Sheng Li 0010; Ke Hu; Zhehuai Chen; Shinji Watanabe 0001; Fei Cheng 0002; Chenhui Chu; Sadao Kurohashi

  ACL (1), 2025

  
• Collaborative Transformer Prototype Network With Pretrained Contrastive Language-Audio Encoder for Open Set Audio Recognition

  Haiyan Yang; Jun Wang; Sheng Li; Di Zhou; Xingwei Chen; Juncheng Li; Yufeng Hua; Jun Shi

  IEEE Transactions on Signal Processing, 2025, Peer-reviewed

  
• Cross-Lingual Embedding Clustering for Hierarchical Softmax in Low-Resource Multilingual Speech Recognition

  Zhengdong Yang; Qianying Liu; Sheng Li; Fei Cheng; Chenhui Chu

  IEEE Transactions on Audio, Speech and Language Processing, 2025, Peer-reviewed

  
• CoVoGER: A Multilingual Multitask Benchmark for Speech-to-text Generative Error Correction with Large Language Models,

  Zhengdong Yang; Zhen Wan; Sheng Li; Chao-Han Huck Yang; Chenhui Chu

  Proc. EMNLP (long main), 2025, Peer-reviewed

  
• RAG-Boost: Retrieval-Augmented Generation Enhanced LLM-based Speech Recognition,

  Pengcheng Wang; Sheng Li; Takahiro Shinozaki

  Interspeech2025 MLC-SLM Challenge workshop, 2025, Peer-reviewed

  
• Multi-Domain Dialogue State Tracking with Large Language Model Rationale and Disentangled Domain-Slot Attention

  Longfei Yang; Jiyi Li; Sheng Li; Takahiro Shinozaki

  IEEE Transactions on Audio, Speech and Language Processing, 2025, Peer-reviewed

  
• Neural TTS-Based Dynamic Data Augmentation for Improved Speech Separation

  Kai Wang; Cuicui Zhu; Lili Yin; Sheng Li; Madina Mansurova; Hao Huang

  IEEE Transactions on Audio, Speech and Language Processing, 2025, Peer-reviewed

  
• A Two-Stage LoRA Strategy for Expanding Language Capabilities in Multilingual ASR Models

  Chin Yuen Kwok; Hexin Liu; Jia Qi Yip; Sheng Li; Eng Siong Chng

  IEEE Transactions on Audio, Speech and Language Processing, 2025, Peer-reviewed

  
• Parallel and Limited Data Voice Conversions on Myanmar Language Speech for Spoofed Detection

  Hay Mar Soe Naing; Win Pa Pa; Sheng Li

  Proceedings of the 6th ACM International Conference on Multimedia in Asia Workshops, 03 Dec. 2024, Peer-reviewed

  
• LaMuCo: Large-Scale Multilingual Conversation Speech Recognition Challenge

  Qingqing Zhang; Lei Luo; Simin Xu; Yongjing Chen; Chuang Li; Sheng Li; Ruili Wang

  Proceedings of the 6th ACM International Conference on Multimedia in Asia Workshops, 03 Dec. 2024, Peer-reviewed

  
• Low-resource Language Adaptation with Ensemble of PEFT Approaches

  Chin Yuen Kwok; Sheng Li; Jia Qi Yip; Eng Siong Chng

  2024 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 03 Dec. 2024, Peer-reviewed

  
• LLM as decoder: Investigating Lattice-based Speech Recognition Hypotheses Rescoring Using LLM

  Sheng Li; Yuka Ko; Akinori Ito

  2024 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 03 Dec. 2024, Peer-reviewed

  
• Data Selection using Spoken Language Identification for Low-Resource and Zero-Resource Speech Recognition

  Jianan Chen; Chenhui Chu; Sheng Li; Tatsuya Kawahara

  2024 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 03 Dec. 2024, Peer-reviewed

  Access this paper :

  
• Investigating Effective Speaker Property Privacy Protection in Federated Learning for Speech Emotion Recognition

  Chao Tan; Sheng Li; Yang Cao; Zhao Ren; Tanja Schultz

  ACM Multimedia Asia 2024, Dec. 2024, Peer-reviewed

  
• Enhancing Privacy of Spatiotemporal Federated Learning Against Gradient Inversion Attacks

  Lele Zheng; Yang Cao; Renhe Jiang; Kenjiro Taura; Yulong Shen; Sheng Li; Masatoshi Yoshikawa

  Lecture Notes in Computer Science, 01 Oct. 2024, Peer-reviewed

  
• Investigating ASR Error Correction with Large Language Model and Multilingual 1-best Hypotheses

  Sheng Li; Chen Chen; Chin Yuen Kwok; Chenhui Chu; Eng Siong Chng; Hisashi Kawai

  Interspeech 2024, 01 Sep. 2024, Peer-reviewed, Lead author, Corresponding author

  
• Automatic Post-Editing of Speech Recognition System Output Using Large Language Models

  Sheng Li; Jiyi Li; Yang Cao

  The DASFAA 2024 Workshop, Jul. 2024, Peer-reviewed

  
• Revisiting Generative Adversarial Network for Downstream Task of Speech Recognition

  Sheng Li; Bei Liu; Jianlong Fu

  Proc. IEEE GEM, Jun. 2024, Peer-reviewed

  
• Voices of the Himalayas: Benchmarking Speech Recognition Systems for the Tibetan Language

  Sheng Li; Jiyi Li; Chenhui Chu

  International Journal of Asian Language Processing, 31 May 2024, Peer-reviewed

  
• Reproducibility Companion Paper: Stable Diffusion for Content-Style Disentanglement in Art Analysis

  Yankun Wu; Yuta Nakashima; Noa Garcia; Sheng Li; Zhaoyang Zeng

  Proceedings of the 2024 International Conference on Multimedia Retrieval, 30 May 2024, Peer-reviewed

  
• Enhancing Realism in 3D Facial Animation Using Conformer-Based Generation and Automated Post-Processing

  Yi Zhao; Chunyu Qiang; Hao Li; Yulan Hu; Wangjin Zhou; Sheng Li

  ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 14 Apr. 2024, Peer-reviewed

  
• MOS-FAD: Improving Fake Audio Detection Via Automatic Mean Opinion Score Prediction

  Wangjin Zhou; Zhengdong Yang; Chenhui Chu; Sheng Li; Raj Dabre; Yi Zhao; Kawahara Tatsuya

  ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 14 Apr. 2024, Peer-reviewed

  
• Phantom in the opera: adversarial music attack for robot dialogue system

  Sheng Li; Jiyi Li; Yang Cao

  Frontiers in Computer Science, 15 Feb. 2024, Peer-reviewed, Invited

  
• End-to-end Japanese-English Speech-to-text Translation with Spoken-to-Written Style Conversion

  Zhengdong Yang; Shuichiro Shimizu; Chenhui Chu; Sheng Li; Sadao Kurohashi

  Journal of Natural Language Processing, 2024, Peer-reviewed

  
• LE-SSL-MOS: Self-Supervised Learning MOS Prediction with Listener Enhancement

  Zili Qi; Xinhui Hu; Wangjin Zhou; Sheng Li; Hao Wu; Jian Lu; Xinkang Xu

  2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 16 Dec. 2023, Peer-reviewed

  
• FedCPC: An Effective Federated Contrastive Learning Method for Privacy Preserving Early-Stage Alzheimers Speech Detection

  Wenqing Wei; Zhengdong Yang; Yuan Gao; Jiyi Li; Chenhui Chu; Shogo Okada; Sheng Li

  2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 16 Dec. 2023, Peer-reviewed, Last author, Corresponding author

  
• KyotoMOS: An Automatic MOS Scoring System for Speech Synthesis

  Wangjin Zhou; Zhengdong Yang; Sheng Li; Chenhui Chu

  ACM Multimedia Asia Workshops, 06 Dec. 2023, Peer-reviewed, Invited, Corresponding author

  Access this paper :

  
• GhostVec: A New Threat to Speaker Privacy of End-to-End Speech Recognition System

  Xiaojiao Chen; Sheng Li; Jiyi Li; Yang Cao; Hao Huang; Liang He

  ACM Multimedia Asia 2023, 06 Dec. 2023, Peer-reviewed

  
• Reprogramming Self-supervised Learning-based Speech Representations for Speaker Anonymization

  Xiaojiao Chen; Sheng Li; Jiyi Li; Hao Huang; Yang Cao; Liang He

  ACM Multimedia Asia 2023, 06 Dec. 2023, Peer-reviewed

  
• Robust voice activity detection using an auditory-inspired masked modulation encoder based convolutional attention network

  Nan Li; Longbiao Wang; Meng Ge; Masashi Unoki; Sheng Li; Jianwu Dang

  Speech Communication, Dec. 2023, Peer-reviewed

  
• Finetuning Pretrained Model with Embedding of Domain and Language Information for ASR of Very Low-Resource Settings

  Soky Kak; Sheng Li; Chenhui Chu; Tatsuya Kawahara

  International Journal of Asian Language Processing (IJALP), Nov. 2023, Peer-reviewed

  Access this paper :

  
• Disordered speech recognition considering low resources and abnormal articulation

  Yuqin Lin; Longbiao Wang; Jianwu Dang; Sheng Li; Chenchen Ding

  Speech Communication, Nov. 2023

  
• Correction while Recognition: Combining Pretrained Language Model for Taiwan-Accented Speech Recognition

  Sheng Li; Jiyi Li

  Artificial Neural Networks and Machine Learning – ICANN 2023, 22 Sep. 2023, Peer-reviewed, Lead author

  
• The Kyoto Speech-to-Speech Translation System for IWSLT 2023

  Zhengdong Yang; Shuichiro Shimizu; Wangjin Zhou; Sheng Li; Chenhui Chu

  International Conference on Spoken Language Translation (IWSLT), Jul. 2023, Peer-reviewed

  
• Dialogue State Tracking with Sparse Local Slot Attention

  Longfei Yang; Jiyi Li; Sheng Li; Takahiro Shinozaki

  ACL 2023 Workshop on NLP for Conversational AI, Jul. 2023, Peer-reviewed

  
• Multi-Domain Dialogue State Tracking with Disentangled Domain-Slot Attention

  Longfei Yang; Jiyi Li; Sheng Li; Takahiro Shinozaki

  In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023): Findings Volume, Jul. 2023, Peer-reviewed

  
• Towards Speech Dialogue Translation Mediating Speakers of Different Languages

  Shuichiro Shimizu; Chenhui Chu; Sheng Li; Sadao Kurohashi

  In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023): Findings Volume, Jul. 2023, Peer-reviewed

  
• Tendency-and-attention-informed deep learning for ENSO forecasts

  Shen Qiao; Cuicui Zhang; Xuefeng Zhang; Kai Zhang; Hao Shi; Sheng Li; Hao Wei

  Climate Dynamics, 17 Jun. 2023

  
• Development of a Pain Signaling System Using Machine Learning

  Helen Korving; Sheng Li; Di Zhou; Paula Sterkenburg; Panos Markopoulos; Emilia Barakova

  2023 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW), 04 Jun. 2023, Peer-reviewed

  
• General or Specific? Investigating Effective Privacy Protection in Federated Learning for Speech Emotion Recognition

  Chao Tan; Yang Cao; Sheng Li; Masatoshi Yoshikawa

  ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 04 Jun. 2023, Peer-reviewed

  
• Speech-Text Based Multi-Modal Training with Bidirectional Attention for Improved Speech Recognition

  Yuhang Yang; Haihua Xu; Hao Huang; Eng Siong Chng; Sheng Li

  ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 04 Jun. 2023, Peer-reviewed

  
• Speakeraugment: Data Augmentation for Generalizable Source Separation via Speaker Parameter Manipulation

  Kai Wang; Yuhang Yang; Hao Huang; Ying Hu; Sheng Li

  ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 04 Jun. 2023, Peer-reviewed

  
• Domain and Language Adaptation Using Heterogeneous Datasets for Wav2vec2.0-Based Speech Recognition of Low-Resource Language

  Kak Soky; Sheng Li; Chenhui Chu; Tatsuya Kawahara

  ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 04 Jun. 2023, Peer-reviewed

  
• Hierarchical Softmax for End-To-End Low-Resource Multilingual Speech Recognition

  Qianying Liu; Zhuo Gong; Zhengdong Yang; Yuhang Yang; Sheng Li; Chenchen Ding; Nobuaki Minematsu; Hao Huang; Fei Cheng; Chenhui Chu; Sadao Kurohashi

  ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 04 Jun. 2023, Peer-reviewed, Corresponding author

  
• An End-to-End Chinese and Japanese Bilingual Speech Recognition Systems with Shared Character Decomposition

  Sheng Li; Jiyi Li; Qianying Liu; Zhuo Gong

  Communications in Computer and Information Science, 14 Apr. 2023, Peer-reviewed, Lead author, Corresponding author

  
• GhostVec: Directly Extracting Speaker Embedding from End-to-End Speech Recognition Model Using Adversarial Examples

  Xiaojiao Chen; Sheng Li; Hao Huang

  Communications in Computer and Information Science, 14 Apr. 2023, Peer-reviewed, Lead author

  
• Investigating Effective Domain Adaptation Method for Speaker Verification Task

  Guangxing Li; Wangjin Zhou; Sheng Li; Yi Zhao; Jichen Yang; Hao Huang

  Communications in Computer and Information Science, 14 Apr. 2023, Peer-reviewed

  
• Can We Train a Language Model Inside an End-to-End ASR Model? - Investigating Effective Implicit Language Modeling

  Zhuo Gong; Saito Daisuke; Sheng Li; Hisashi Kawai; Minematsu Nobuaki

  Proceedings of the Second Workshop on When Creative AI Meets Conversational AI, Dec. 2022, Peer-reviewed

  
• SpecMNet: Spectrum Mend Network for Monaural Speech Enhancement

  Cunhang Fan; Hongmei Zhang; Jiangyan Yi; Zhao Lv; Jianhua Tao; Taihao Li; Guanxiong Pei; Xiaopei Wu; Sheng Li

  Applied Acoustics, Dec. 2022, Peer-reviewed

  
• Improving low-resource Tibetan end-to-end ASR by multilingual and multilevel unit modeling

  Siqing Qin; Longbiao Wang; Sheng Li; Jianwu Dang; Lixin Pan

  EURASIP Journal on Audio, Speech, and Music Processing, Dec. 2022, Peer-reviewed

  
• Subband-based Spectrogram Fusion for Speech Enhancement by Combining Mapping and Masking Approaches

  Hao Shi; Longbiao Wang; Sheng Li; Jianwu Dang; Tatsuya Kawahara

  2022 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 07 Nov. 2022, Peer-reviewed

  
• Nict-Tib1: A Public Speech Corpus Of Lhasa Dialect For Benchmarking Tibetan Language Speech Recognition Systems

  Kak Soky; Zhuo Gong; Sheng Li

  2022 25th Conference of the Oriental COCOSDA International Committee for the Co-ordination and Standardisation of Speech Databases and Assessment Techniques (O-COCOSDA), Nov. 2022, Peer-reviewed, Corresponding author

  
• Monaural Speech Enhancement Based on Spectrogram Decomposition for Convolutional Neural Network-sensitive Feature Extraction

  Hao Shi; Longbiao Wang; Sheng Li; Jianwu Dang; Tatsuya Kawahara

  Interspeech 2022, 18 Sep. 2022, Peer-reviewed

  
• Multi-Domain Dialogue State Tracking with Top-k Slot Self Attention

  Longfei Yang; Jiyi Li; Sheng Li; Takahiro Shinozaki

  In Proc. SIGdial Meeting Discourse \& Dialogue, Sep. 2022, Peer-reviewed, Lead author

  
• Global Signal-to-noise Ratio Estimation Based on Multi-subband Processing Using Convolutional Neural Network

  Nan LI; Meng Ge; Longbiao Wang; Masashi Unoki; Sheng Li; Jianwu Dang

  in Proc. INTERSPEECH, Sep. 2022, Peer-reviewed, Corresponding author

  
• Finer-grained Modeling units-based Meta-Learning for Low-resource Tibetan Speech Recognition

  Siqing Qin; Longbiao Wang; Sheng Li; Yuqin Lin; Jianwu Dang

  in Proc. INTERSPEECH, Sep. 2022, Peer-reviewed, Corresponding author

  
• Fusion of Self-supervised Learned Models for MOS Prediction

  Zhengdong Yang; Wangjin Zhou; Chenhui Chu; Sheng Li; Raj Dabre; Raphael Rubino; Yi Zhao

  in Proc. INTERSPEECH, Sep. 2022, Peer-reviewed, Corresponding author

  
• Data Augmentation Using McAdams-Coefficient-Based Speaker Anonymization for Fake Audio Detection

  Kai Li; Sheng Li; Xugang Lu; Masato Akagi; Meng Liu; Lin Zhang; Chang Zeng; Longbiao Wang; Jianwu Dang; Masashi Unoki

  in Proc. INTERSPEECH, Sep. 2022, Peer-reviewed, Corresponding author

  
• Augmented Adversarial Self-Supervised Learning for Early-Stage Alzheimer's Speech Detection

  Longfei Yang; Wenqing Wei; Sheng Li; Jiyi Li; Takahiro Shinozaki

  in Proc. INTERSPEECH, Sep. 2022, Peer-reviewed, Corresponding author

  
• Leveraging Simultaneous Translation for Enhancing Transcription of Low-resource Language via Cross Attention Mechanism

  Kak Soky; Sheng Li; Masato Mimura; Chenhui Chu; Tatsuya Kawahara

  in Proc. INTERSPEECH, Sep. 2022, Peer-reviewed, Corresponding author

  
• Relationship Between Speakers' Physiological Structure and Acoustic Speech Signals: Data-Driven Study Based on Frequency-Wise Attentional Neural Network

  Kai Li; Xugang Lu; Masato Akagi; Jianwu Dang; Sheng Li; Masashi Unoki

  2022 30th European Signal Processing Conference (EUSIPCO), 29 Aug. 2022, Peer-reviewed, Corresponding author

  
• TriECCC: Trilingual Corpus of the Extraordinary Chambers in the Courts of Cambodia for Speech Recognition and Translation Studies

  Kak Soky; Masato Mimura; Tatsuya Kawahara; Chenhui Chu; Sheng Li; Chenchen Ding; Sethserey Sam

  International Journal of Asian Language Processing, 21 Jul. 2022, Peer-reviewed, Invited, Lead author

  
• Self-Adaptive Multilingual ASR Rescoring with Language Identification and Unified Language Model

  Zhuo Gong; Daisuke Saito; Longfei Yang; Takahiro Shinozaki; Sheng Li; Hisashi Kawai; Nobuaki Minematsu

  The Speaker and Language Recognition Workshop (Odyssey 2022), 28 Jun. 2022, Peer-reviewed, Lead author

  
• Adversarial Speech Generation and Natural Speech Recovery for Speech Content Protection.

  Sheng Li; Jiyi Li; Qianying Liu; Zhuo Gong

  In Proc. LREC (Language Resources and Evaluation Conference), Jun. 2022, Peer-reviewed, Lead author, Corresponding author

  
• Mining Hard Samples Locally And Globally For Improved Speech Separation

  Kai Wang; Yizhou Peng; Hao Huang; Ying Hu; Sheng Li

  ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 23 May 2022, Peer-reviewed, Lead author

  
• Compressing Transformer-Based ASR Model by Task-Driven Loss and Attention-Based Multi-Level Feature Distillation

  Yongjie Lv; Longbiao Wang; Meng Ge; Sheng Li; Chenchen Ding; Lixin Pan; Yuguang Wang; Jianwu Dang; Kiyoshi Honda

  ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 23 May 2022, Peer-reviewed, Lead author, Corresponding author

  
• Cross-Lingual Transfer Learningfor End-to-End Speech Translation

  Shuichiro Shimizu; Chenhui Chu; Sheng Li; Sadao Kurohashi

  Journal of Natural Language Processing, 2022

  
• Adversarial Attack and Defense on Deep Neural Network-based Voice Processing Systems: An Overview

  Xiaojiao Chen; Sheng Li; Hao Huang

  Applied Sciences, Special Issues of Machine Speech Communication, 2021., Dec. 2021, Peer-reviewed, Corresponding author

  
• Khmer Speech Translation Corpus of the Extraordinary Chambers in the Courts of Cambodia (ECCC)

  Kak Soky; Masato Mimura; Tatsuya Kawahara; Sheng Li; Chenchen Ding; Chenhui Chu; Sethserey Sam

  in Proc. O-COCOSDA, Dec. 2021, Peer-reviewed

  
• On the Use of Speaker Information for Automatic Speech Recognition in Speaker-imbalanced Corpora

  Kak Soky; Sheng Li; Masato Mimura; Chenhui Chu; Tatsuya Kawahara

  In Proc. APSIPA ASC, Dec. 2021, Peer-reviewed

  
• Multilingual Approach to Joint Speech and Accent Recognition with DNN-HMM Framework

  Yizhou Peng; Jicheng Zhang; Haobo Zhang; Haihua Xu; Hao Huang; Sheng Li; Eng Siong Chng

  In Proc. APSIPA ASC, Dec. 2021, Peer-reviewed

  
• Spectrograms Fusion-based End-to-End Robust Automatic Speech Recognition

  Hao Shi; Longbiao Wang; Sheng Li; Cunhang Fan; Jianwu Dang; Tatsuya Kawahara

  In Proc. APSIPA ASC, Dec. 2021, Peer-reviewed

  
• End-to-End Speech Separation Using Orthogonal Representation in Complex and Real Time-Frequency Domain

  Kai Wang; Hao Huang; Ying Hu; Zhihua Huang; Sheng Li

  Interspeech 2021, 30 Aug. 2021, Peer-reviewed

  
• An End-to-End Dialect Identification System with Transfer Learning from a Multilingual Automatic Speech Recognition Model

  Ding Wang; Shuaishuai Ye; Xinhui Hu; Sheng Li; Xinkang Xu

  Interspeech 2021, 30 Aug. 2021, Peer-reviewed

  
• Robust Voice Activity Detection Using a Masked Auditory Encoder Based Convolutional Neural Network

  Nan Li; Longbiao Wang; Masashi Unoki; Sheng Li; Rui Wang; Meng Ge; Jianwu Dang

  ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 06 Jun. 2021

  
• Encoder-Decoder Based Pitch Tracking and Joint Model Training for Mandarin Tone Classification

  Hao Huang; Kai Wang; Ying Hu; Sheng Li

  ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 06 Jun. 2021, Peer-reviewed

  
• An Investigation of Using Hybrid Modeling Units for Improving End-to-End Speech Recognition System

  Shunfei Chen; Xinhui Hu; Sheng Li; Xinkang Xu

  ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 06 Jun. 2021

  
• Simultaneous Progressive Filtering-Based Monaural Speech Enhancement

  Haoran Yin; Hao Shi; Longbiao Wang; Luya Qiang; Sheng Li; Meng Ge; Gaoyan Zhang; Jianwu Dang

  Communications in Computer and Information Science, 2021, Peer-reviewed

  
• Speech Dereverberation Based on Scale-Aware Mean Square Error Loss

  Luya Qiang; Hao Shi; Meng Ge; Haoran Yin; Nan Li; Longbiao Wang; Sheng Li; Jianwu Dang

  Communications in Computer and Information Science, 2021, Peer-reviewed

  
• Exploring Effective Speech Representation via ASR for High-Quality End-to-End Multispeaker TTS

  Dawei Liu; Longbiao Wang; Sheng Li; Haoyu Li; Chenchen Ding; Ju Zhang; Jianwu Dang

  Communications in Computer and Information Science, 2021, Peer-reviewed, Corresponding author

  
• Phantom in the Opera: Effective Adversarial Music Attack on Keyword Spotting Systems.

  Heran Zhang; Sheng Li; Xingjun Ma; Yi Zhao; Yang Cao; Tatsuya Kawahara

  IEEE-SLT2021, 2021, Corresponding author

  
• VOIS: The First Speech Therapy App Specifically Designed for Myanmar Hearing-Impaired Children

  Aye Thida; Nway Nway Han; Sheinn Thawtar Oo; Sheng Li; Chenchen Ding

  2020 23rd Conference of the Oriental COCOSDA International Committee for the Co-ordination and Standardisation of Speech Databases and Assessment Techniques (O-COCOSDA), 05 Nov. 2020

  
• Compensation on x-vector for Short Utterance Spoken Language Identification

  Peng Shen; Xugang Lu; Komei Sugiura; Sheng Li; Hisashi Kawai

  Odyssey 2020 The Speaker and Language Recognition Workshop, 01 Nov. 2020

  
• Joint Training End-to-End Speech Recognition Systems with Speaker Attributes

  Sheng Li; Xugang Lu; Raj Dabre; Peng Shen; Hisashi Kawai

  Odyssey 2020 The Speaker and Language Recognition Workshop, 01 Nov. 2020

  
• Voice-Indistinguishability -- Protecting Voiceprint with Differential Privacy under an Untrusted Server

  Yaowei Han; Yang Cao; Sheng Li; Qiang Ma; Masatoshi Yoshikawa

  Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, 30 Oct. 2020

  
• Singing Voice Extraction with Attention-Based Spectrograms Fusion

  Hao Shi; Longbiao Wang; Sheng Li; Chenchen Ding; Meng Ge; Nan Li; Jianwu Dang; Hiroshi Seki

  Interspeech 2020, 25 Oct. 2020

  
• Staged Knowledge Distillation for End-to-End Dysarthric Speech Recognition and Speech Attribute Transcription

  Yuqin Lin; Longbiao Wang; Sheng Li; Jianwu Dang; Chenchen Ding

  Interspeech 2020, 25 Oct. 2020

  
• Voice-Indistinguishability: Protecting Voiceprint In Privacy-Preserving Speech Data Release

  Yaowei Han; Sheng Li; Yang Cao; Qiang Ma; Masatoshi Yoshikawa

  2020 IEEE International Conference on Multimedia and Expo (ICME), Jul. 2020

  
• Spectrograms Fusion with Minimum Difference Masks Estimation for Monaural Speech Dereverberation

  Hao Shi; Longbiao Wang; Meng Ge; Sheng Li; Jianwu Dang

  ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May 2020

  
• End-to-End Articulatory Modeling for Dysarthric Articulatory Attribute Detection

  Yuqin Lin; Longbiao Wang; Jianwu Dang; Sheng Li; Chenchen Ding

  ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May 2020

  
• Automatic speech recognition

  Xugang Lu; Sheng Li; Masakiyo Fujimoto

  SpringerBriefs in Computer Science, 2020

  
• Knowledge Distillation-Based Representation Learning for Short-Utterance Spoken Language Identification

  Peng Shen; Xugang Lu; Sheng Li; Hisashi Kawai

  IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2020

  
• Investigation of Effectively Synthesizing Code-Switched Speech Using Highly Imbalanced Mix-Lingual Data

  Shaotong Guo; Longbiao Wang; Sheng Li; Ju Zhang; Cheng Gong; Yuguang Wang; Jianwu Dang; Kiyoshi Honda

  Neural Information Processing, 2020

  
• Multi-lingual transformer training for khmer automatic speech recognition

  Kak Soky; Sheng Li; Tatsuya Kawahara; Sopheap Seng

  2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2019, Nov. 2019, Peer-reviewed

  
• Effective Training End-to-End ASR systems for Low-resource Lhasa Dialect of Tibetan Language

  Lixin Pan; Sheng Li; Longbiao Wang; Jianwu Dang

  2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), Nov. 2019

  
• Class-Wise Centroid Distance Metric Learning for Acoustic Event Detection

  Xugang Lu; Peng Shen; Sheng Li; Yu Tsao; Hisashi Kawai

  Interspeech 2019, 15 Sep. 2019

  
• Investigating Radical-Based End-to-End Speech Recognition Systems for Chinese Dialects and Japanese

  Sheng Li; Xugang Lu; Chenchen Ding; Peng Shen; Tatsuya Kawahara; Hisashi Kawai

  Interspeech 2019, 15 Sep. 2019

  
• End-to-End Articulatory Attribute Modeling for Low-Resource Multilingual Speech Recognition

  Sheng Li; Chenchen Ding; Xugang Lu; Peng Shen; Tatsuya Kawahara; Hisashi Kawai

  Interspeech 2019, 15 Sep. 2019

  
• Improving Transformer-Based Speech Recognition Systems with Compressed Structure and Speech Attributes Augmentation

  Sheng Li; Dabre Raj; Xugang Lu; Peng Shen; Tatsuya Kawahara; Hisashi Kawai

  Interspeech 2019, 15 Sep. 2019

  
• Interactive learning of teacher-student model for short utterance spoken language identification.

  P.Shen; X.Lu; S. Li; H.Kawai

  Proc. IEEE-ICASSP, 2019, Peer-reviewed

  
• INVESTIGATION OF SEQUENCE-LEVEL KNOWLEDGE DISTILLATION METHODS FOR CTC ACOUSTIC MODELS

  Ryoichi Takashima; Li Sheng; Hisashi Kawai

  2019 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2019, Peer-reviewed

  
• Feature Representation of Short Utterances based on Knowledge Distillation for Spoken Language Identification.

  P.Shen; X.Lu; S. Li; H.Kawai

  Proc. INTERSPEECH, 2018, Peer-reviewed

  
• Temporal Attentive Pooling for Acoustic Event Detection.

  X.Lu; P.Shen; S. Li; Y.Tsao; H.Kawai

  Proc. INTERSPEECH, 2018, Peer-reviewed

  
• AN INVESTIGATION OF A KNOWLEDGE DISTILLATION METHOD FOR CTC ACOUSTIC MODELS

  Ryoichi Takashima; Sheng Li; Hisashi Kawai

  2018 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2018, Peer-reviewed

  
• CTC LOSS FUNCTION WITH A UNIT-LEVEL AMBIGUITY PENALTY

  Ryoichi Takashima; Sheng Li; Hisashi Kawai

  2018 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2018, Peer-reviewed

  
• Improving CTC-based Acoustic Model with Very Deep Residual Time-delay Neural Networks

  Sheng Li; Xugang Lu; Ryoichi Takashima; Peng Shen; Tatsuya Kawahara; Hisashi Kawai

  19TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION (INTERSPEECH 2018), VOLS 1-6, 2018, Peer-reviewed

  
• IMPROVING VERY DEEP TIME-DELAY NEURAL NETWORK WITH VERTICAL-ATTENTION FOR EFFECTIVELY TRAINING CTC-BASED ASR SYSTEMS

  Sheng Li; Xugang Lu; Ryoichi Takashima; Peng Shen; Tatsuya Kawahara; Hisashi Kawai

  2018 IEEE WORKSHOP ON SPOKEN LANGUAGE TECHNOLOGY (SLT 2018), 2018, Peer-reviewed

  
• INCREMENTAL TRAINING AND CONSTRUCTING THE VERY DEEP CONVOLUTIONAL RESIDUAL NETWORK ACOUSTIC MODELS

  Sheng Li; Xugang Lu; Peng Shen; Ryoichi Takashima; Tatsuya Kawahara; Hisashi Kawai

  2017 IEEE AUTOMATIC SPEECH RECOGNITION AND UNDERSTANDING WORKSHOP (ASRU), 2017, Peer-reviewed

  
• Conditional generative adversarial nets classifier for spoken language identification

  Peng Shen; Xugang Lu; Sheng Li; Hisashi Kawai

  Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, 2017, Peer-reviewed

  
• SEMI-SUPERVISED ENSEMBLE DNN ACOUSTIC MODEL TRAINING

  Sheng Li; Xugang Lu; Shinsuke Sakai; Masato Mimura; Tatsuya Kawahara

  2017 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2017, Peer-reviewed

  
• Semi-Supervised Acoustic Model Training by Discriminative Data Selection From Multiple ASR Systems' Hypotheses

  Sheng Li; Yuya Akita; Tatsuya Kawahara

  IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING, Sep. 2016, Peer-reviewed

  Access this paper :

  
• Confidence Estimation for Speech Recognition Systems using Conditional Random Fields Trained with Partially Annotated Data

  Sheng Li; Xugang Lu; Shinsuke Mori; Yuya Akita; Tatsuya Kawahara

  2016 10TH INTERNATIONAL SYMPOSIUM ON CHINESE SPOKEN LANGUAGE PROCESSING (ISCSLP), 2016, Peer-reviewed

  
• Data Selection from Multiple ASR Systems' Hypotheses for Unsupervised Acoustic Model Training

  Sheng Li; Yuya Akita; Tatsuya Kawahara

  2016 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING PROCEEDINGS, 2016, Peer-reviewed

  
• Automatic Lecture Transcription Based on Discriminative Data Selection for Lightly Supervised Acoustic Model Training

  Sheng Li; Yuya Akita; Tatsuya Kawahara

  IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, Aug. 2015, Peer-reviewed

  Access this paper :

  
• Discriminative Data Selection for Lightly Supervised Training of Acoustic Model using Closed Caption Texts

  Sheng Li; Yuya Akita; Tatsuya Kawahara

  16TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION (INTERSPEECH 2015), VOLS 1-5, 2015, Peer-reviewed

  
• Ensemble Speaker Modeling using Speaker Adaptive Training Deep Neural Network for Speaker Adaptation

  Sheng Li; Xugang Lu; Yuya Akita; Tatsuya Kawahara

  16TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION (INTERSPEECH 2015), VOLS 1-5, 2015, Peer-reviewed

  
• Corpus and Transcription System of Chinese Lecture Room

  Sheng Li; Yuya Akita; Tatsuya Kawahara

  2014 9TH INTERNATIONAL SYMPOSIUM ON CHINESE SPOKEN LANGUAGE PROCESSING (ISCSLP), 2014, Peer-reviewed

  
• Phoneme-level articulatory animation in pronunciation training

  Lan Wang; Hui Chen; Sheng Li; Helen M. Meng

  SPEECH COMMUNICATION, Sep. 2012, Peer-reviewed

  
• Cross Linguistic Comparison of Mandarin and English EMA Articulatory Data

  Sheng Li; Lan Wang

  13TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION 2012 (INTERSPEECH 2012), VOLS 1-3, 2012, Peer-reviewed

  
• The Phoneme-level Articulator Dynamics for Pronunciation Animation

  Sheng Li; Lan Wang; En Qi

  Proc. IALP, 2011, Peer-reviewed

  
• IELS: A Computer-aided Pronunciation Training System for Undergraduate Students

  Jinyu Chen; Lan Wang; Chongguo Li; Jin Hu; Sheng Li

  ICETC, 2010, Peer-reviewed

  

Misc.

• Casting Everything to Online API Services? A Survey of Integrating Localized Speech Recognition Models in Robotic Systems

  Sheng Li; Jing Li; Felix Schijve; Jun Hu; Emilia Barakova

  arXiv:2607.11792v1, May 2026, Lead author

  
• Beyond Speech Captions: Speech-Rewarded Style Planning for Conversational Text-to-Speech

  Shiao Zhu; Lianbo Liu; Sizhen Lyu; Yuzhe Wang; Sheng Li; Takahiro Shinozaki

  arxiv, Oct. 2026

  
• Synchronized Three-Dimensional Vocal-Tract Motion for Speech Synthesis via Joint-Embedding Predictive Architecture Alignment

  Sheng Li, Takahiro Shinozaki

  Jul. 2026

  
• Cached LLM Probability Retrieval for Speech Recognition

  Sheng Li; Takahiro Shinozaki; Tatsuya Kawahara

  arXiv preprint arXiv:2608.16023, Aug. 2026

  
• DuplexGen: Decoupling Content, Timing, and Acoustics for Synthetic Dialogue Speech

  Pengcheng Wang; Sheng Li; Jiyi Li; Takahiro Shinozaki

  arXiv preprint arXiv:2608.16053, Aug. 2026

  
• Evaluating Tibetan ASR with Segmented Word Error Rate: Beyond Character-Level Metrics

  Jacob Moore; Sheng Li; Paula Lauren

  TechRxiv, 13 Feb. 2026

  
• End-to-end Acoustic-linguistic Emotion and Intent Recognition Enhanced by Semi-supervised Learning.

  Zhao Ren; Rathi Adarshi Rammohan; Kevin Scheck; Sheng Li; Tanja Schultz

  Dec. 2025

  
• Emotional Text-To-Speech Based on Mutual-Information-Guided Emotion-Timbre Disentanglement

  Jianing Yang; Sheng Li; Takahiro Shinozaki; Yuki Saito; Hiroshi Saruwatari

  arXiv, Oct. 2025

  
• Adapting Whisper for Parameter-efficient Code-Switching Speech Recognition via Soft Prompt Tuning

  Hongli Yang; Yizhou Peng; Hao Huang; Sheng Li

  Jul. 2025

  
• Language-Aware Prompt Tuning for Parameter-Efficient Seamless Language Expansion in Multilingual ASR

  Hongli Yang; Sheng Li; Hao Huang; Ayiduosi Tuohan; Yizhou Peng

  arxiv, Jul. 2025

  
• Generalized Multilingual Text-to-Speech Generation with Language-Aware Style Adaptation

  Haowei Lou; Hye-young Paik; Sheng Li; Wen Hu; Lina Yao

  arXiv preprint arXiv:2504.08274, Apr. 2025

  
• Multi-Prototype  Network with Swin Transformer for Open Set Recognition

  Jun Wang; Haiyan Yang; Sheng Li; Di Zhou; Xingwei Chen; Juncheng Li; Yufeng Hua; Jun Shi

  SSRN, 2025

  
• Multi-Prototype  Network with Swin Transformer for Open Set Recognition

  Jun Wang; Haiyan Yang; Sheng Li; Di Zhou; Xingwei Chen; Juncheng Li; Yufeng Hua; Jun Shi

  SSRN, 2025

  
• A Unified Speech LLM for Diarization and Speech Recognition in Multilingual Conversations

  Phurich Saengthong; Boonnithi Jiaramaneepini; Sheng Li; Manabu Okumura; Takahiro Shinozaki

  arXiv, 2025

  
• Towards Emotion Co-regulation with LLM-powered Socially Assistive Robots: Integrating LLM Prompts and Robotic Behaviors to Support Parent-Neurodivergent Child Dyads

  Jing Li; Felix Schijve; Sheng Li; Yuye Yang; Jun Hu; Emilia Barakova

  arXiv, 2025

  
• Cross-lingual Embedding Clustering for Hierarchical Softmax in Low-Resource Multilingual Speech Recognition

  Zhengdong Yang; Qianying Liu; Sheng Li; Fei Cheng; Chenhui Chu

  arXiv, Jan. 2025

  
• Joint Automatic Speech Recognition And Structure Learning For Better Speech Understanding

  Jiliang Hu; Zuchao Li; Mengjia Shen; Haojun Ai; Sheng Li; Jun Zhang

  arXiv, Jan. 2025

  
• Benchmarking Japanese Speech Recognition on ASR-LLM Setups with Multi-Pass Augmented Generative Error Correction

  Yuka Ko; Sheng Li; Chao-Han Huck Yang; Tatsuya Kawahara

  arXiv, Dec. 2024

  
• Extracting Spatiotemporal Data from Gradients with Large Language Models

  Lele Zheng; Yang Cao; Renhe Jiang; Kenjiro Taura; Yulong Shen; Sheng Li; Masatoshi Yoshikawa

  arXiv, Oct. 2024

  
• Investigating Effective Speaker Property Privacy Protection in Federated Learning for Speech Emotion Recognition

  Chao Tan; Sheng Li; Yang Cao; Zhao Ren; Tanja Schultz

  arXiv, Oct. 2024

  
• Enhancing Privacy of Spatiotemporal Federated Learning against Gradient Inversion Attacks

  Lele Zheng; Yang Cao; Renhe Jiang; Kenjiro Taura; Yulong Shen; Sheng Li; Masatoshi Yoshikawa

  arXiv, Jul. 2024

  
• MOS-FAD: Improving Fake Audio Detection Via Automatic Mean Opinion Score Prediction

  Wangjin Zhou; Zhengdong Yang; Chenhui Chu; Sheng Li; Raj Dabre; Yi Zhao; Tatsuya Kawahara

  Jan. 2024

  
• End-to-End Speech-to-Speech Translation toolkit

  Wangjin Zhou; Zhengdong Yang; Chenhui Chu; Sheng Li

  ACM Multimedia Asia 2023 workshop released tookit, Dec. 2023

  
• FedCPC: An Effective Federated Contrastive Learning Method for Privacy Preserving Early-Stage Alzheimer's Speech Detection

  Wenqing Wei; Zhengdong Yang; Yuan Gao; Jiyi Li; Chenhui Chu; Shogo Okada; Sheng Li

  Nov. 2023, Last author, Corresponding author

  
• GhostVec: A New Threat to Speaker Privacy of End-to-End Speech Recognition System

  Xiaojiao Chen; Sheng Li; Jiyi Li; Hao Huang; Yang Cao; Liang He

  Nov. 2023

  
• Reprogramming Self-supervised Learning-based Speech Representations for Speaker Anonymization

  Xiaojiao Chen; Sheng Li; Jiyi Li; Hao Huang; Yang Cao; Liang He

  Nov. 2023

  
• LE-SSL-MOS: Self-Supervised Learning MOS Prediction with Listener Enhancement

  Zili Qi; Xinhui Hu; Wangjin Zhou; Sheng Li; Hao Wu; Jian Lu; Xinkang Xu

  Nov. 2023

  
• Towards Speech Dialogue Translation Mediating Speakers of Different Languages

  Shuichiro Shimizu; Chenhui Chu; Sheng Li; Sadao Kurohashi

  arXiv:2305.09210, May 2023

  
• Robust Voice Activity Detection Using an Auditory-Inspired Masked Modulation Encoder Based Convolutional Attention Network

  Nan LI; Longbiao Wang; Meng Ge; Masashi Unoki; Sheng Li; Jianwu Dang

  2023

  
• Speech-text based multi-modal training with bidirectional attention for improved speech recognition

  Yuhang Yang; Haihua Xu; Hao Huang; Eng Siong Chng; Sheng Li

  arXiv:2211.00325, Oct. 2022

  
• Tendency-and-Attention-Informed Deep Learning for ENSO Forecasts

  Shen Qiao; Cuicui Zhang; Xuefeng Zhang; Kai Zhang; Hao Shi; Sheng Li; Hao Wei

  13 Jun. 2022

  
• Fusion of Self-supervised Learned Models for MOS Prediction

  Zhengdong Yang; Wangjin Zhou; Chenhui Chu; Sheng Li; Raj Dabre; Raphael Rubino; Yi Zhao

  CoRR abs/2204.04855, Apr. 2022, Corresponding author

  
• Hierarchical Softmax for End-to-End Low-resource Multilingual Speech Recognition.

  Qianying Liu; Yuhang Yang; Zhuo Gong; Sheng Li; Chenchen Ding; Nobuaki Minematsu; Hao Huang; Fei Cheng; Sadao Kurohashi

  abs/2204.03855, Apr. 2022, Corresponding author

  
• Voice-Indistinguishability: Protecting Voiceprint in Privacy-Preserving Speech Data Release

  Yaowei Han; Sheng Li; Yang Cao; Qiang Ma; Masatoshi Yoshikawa

  CoRR abs/2004.07442, Jun. 2020, Lead author

  
• Deep progressive multi-scale attention for acoustic event classification

  Xugang Lu; Peng Shen; Sheng Li; Yu Tsao; Hisashi Kawai

  CoRR abs/1912.12011, Apr. 2019, Lead author

  

Presentations

• See Speech in Motion: Synchronization Three-Dimensional Vocal-Tract Motion and Synthesized Speech via Joint-Embedding Predictive Architecture Alignment

  Sheng Li; Pengcheng Wang; Takahiro Shinozaki

  accv 2026 demo track, 15 Dec. 2026

  
• Neural Speech Synthesis With Synchronized 3-D Articulatory Motion

  Sheng Li ,Pengcheng Wang; Takahiro Shinozaki

  ASJ2026 autumn, 10 Sep. 2026

  
• Self-maintaining Knowledge Graph RAG for Real-Time Full-Duplex Speech Model

  Shuenn Yuen Han; Pengcheng Wang; Sheng Li; Takahiro Shinozaki

  ASJ2026 autumn, 10 Sep. 2026

  
• Neural TTS-based Dynamic Data Augmentation for Improved Speech Separation.

  K. Wang; L. Yin; S. Li; M. Mansurova; H. Huang

  EUSIPCO, Sep. 2026

  
• RAG-Boost: RAG-Enhanced Speech Recognition in an LLM-Based Speech Dialogue System

  Pengcheng Wang; Sheng Li; Takahiro Shinozaki

  2026年度 人工知能学会全国大会（第40回）, 09 Jun. 2026

  
• Seamlessly Upgrading On-Device Speech Recognition System with More Recent Foundation Models

  Sheng Li

  The 40th Annual Conference of the Japanese Society for Artificial Intelligence, 2026, 09 Jun. 2026

  
• Multilingual Retrieval-Augmented Generation Enhanced LLM-based Speech Recognition

  王 鵬程; 李 勝; 篠崎 隆宏

  日本音響学会第155回(2026年春季)研究発表会, 19 Mar. 2026

  
• 指示再構成手法に基づく言語モデルベース音声合成のスタイル制御

  Zhu Shiao; Li Sheng; 篠崎 隆宏

  日本音響学会第155回(2026年春季)研究発表会, 18 Mar. 2026

  
• 音声認識および音声翻訳における生成的誤り訂正のための多言語ベンチマーク

  Zhengdong Yang; Zhen Wan; Sheng Li; Chao-Han Huck Yang; Chenhui Chu

  言語処理学会 第32回年次大会, Mar. 2026

  
• RAG-Boost: Retrieval-Augmented Generation Enhanced Speech Recognition in LLM-based Spoken Dialogue Systems

  王鵬程; 李勝; 篠崎隆宏

  日本音響学会研究発表会講演論文集, 秋季, 10 Oct. 2025

  
• 相互情報量最小化による感情・音色の分離に基づく感情的音声合成,

  楊家寧; 李勝; 篠崎 隆宏; 齋藤佑樹; 猿渡洋

  日本音響学会研究発表会講演論文集, 秋季, 10 Oct. 2025

  
• CEFR-J Level Estimation of English Learners' Speech Using Large Language Models

  Takahiro Shinozai; Syutaro Sato; Sheng Li

  CEFR-J 2025 International Symposium, 25 Mar. 2025

  
• Enhancing Multi-Step Reasoning in Language Models with Synthetic Math Data Augmentation (HP_Fighters team)

  Jieqing Mei; Jiyi Li; Qianying Liu; Sheng Li

  The first workshop of fine-tuning and evaluation large language model (FT-LLM2025)@NLP2025, 14 Mar. 2025

  
• 大規模言語モデルの統合による音声認識システムの改善

  李 勝

  NICT Open House 2024, 28 Jun. 2024, Invited

  
• Combining Large Language Model with Speech Recognition System in Low-resource Settings

  Sheng Li; Zhengdong Yang; Wangjin Zhou; Chenhui Chu; Chen Chen; Eng Siong Chng; Hisashi Kawai

  NLP2024, 11 Mar. 2024

  
• Investigating effective methods for combining large language model with speech recognition system

  Sheng Li; Zhengdong Yang; Wangjin Zhou; Chenhui Chu; Hisashi Kawai

  Acoustical Society of Japan 151st (Spring 2024) meeting, 06 Mar. 2024

  
• Hierarchical Softmax for End-to-End Low-resource Multilingual Speech Recognition

  Zhengdong Yang

  ICT-innovation 2023 (Kyoto Univ.), 21 Feb. 2024, Invited

  
• Self-Supervised Learning MOS Prediction with Listener Enhancement

  Sheng Li

  VoiceMOS mini workshop, 20 Nov. 2023, Invited

  
• Correction while Recognition: Combining Pretrained Language Model for Taiwan-accented Speech Recognition

  Sheng Li

  Joint Seminar with NECTEC Language Understand Group, 18 Nov. 2023, Invited

  
• Cross-lingual Mapping for Hierarchical Softmax in Low-Resource Multilingual Speech Recognition

  Zhengdong Yang; Qianying Liu; Sheng Li; Chenhui Chu; Fei Cheng; Sadao Kurohashi

  ASJ 2023 autumn, Sep. 2023

  
• Towards Security-aware Speech Recognition System,

  Sheng Li

  NECTEC-NICT joint seminar, Aug. 2023, Invited

  
• 異言語話者の対話を仲介する音声対話翻訳

  清水 周一郎; 褚 晨翚; 李 勝; 黒橋 禎夫

  言語処理学会第 29 回年次大会（NLP2023）, Mar. 2023

  
• Domain and Language Adaptation of Large-scale Pretrained Model for Speech Recognition of Low-resource Language

  Kak Soky; Sheng Li; Chenhui Chu; Tatsuya Kawahara

  IEICE Tech. Rep. (信学技報), Dec. 2022

  
• VoicePrivacy Challenge: System description

  X. Chen; G. Li; H. Huang; W. Zhou; Y. Cao; S. Li; Y. Zhao

  VoicePrivacy 2022 Challenge Workshop (Interspeech2022), 18 Sep. 2022

  
• System Description for the Voiceprivacy Challenge 2022

  Xiaojiao Chen; Guangxing Li; Wangjin Zhou; Sheng Li; Yang Cao; Hao Huang; Yi Zhao

  Voiceprivacy Challenge 2022, Sep. 2022

  
• Study on Relationship Between Speakers' Physiological Structure and Acoustic Speech Signals: Data-Driven Study Based on Frequency-Wise Attentional Neural Network

  Li Kai; Xugang Lu; Masato Akagi; Jianwu Dang; Sheng Li; Unoki Masashi

  IEICE Tech. Rep., Aug. 2022

  
• System Description for the CN-Celeb Speaker Recognition Challenge 2022

  Guangxing Li; Wangjin Zhou; Sheng Li; Yi Zhao; Hao Huang; Jichen Yang

  CNSRC (the CN-Celeb Speaker Recognition Challenge), Speaker Odyssey 2022, Jun. 2022

  
• The System Description for VoiceMOS Challenge 2022 (main/ood tasks)

  2022

  
• The RoyalFlush(NICT) System Description for AP21-OLR Challenge

  Ding Wang; Shuaishuai Ye; Xinhui Hu; Sheng Li

  AP21-OLR Challenge, Jan. 2022, Invited

  
• System description of joint speech and accent recognition (published in APSIPA ASC, 2021)

  Y. Peng; J. Zhang; H. Zhang; H. Xu; H. Huang; S. Li; E.S. Chng

  in Challenge of Interspeech2020 Accented English Speech Recognition, AESR, 2020., Dec. 2021

  
• System description of Alzheimer's disease early detection (Silk-road team, short speech track)

  Wenqing Wei; Rui Wong; Sheng Li; Yachao Guo; Hao Huang

  Alzheimer's disease detection challenge (NCMMSC2021), 22 Oct. 2021

  
• Adversarial Attack and Defense on Deep Neural Network-based Voice Processing Systems: An Overview

  Xiaojiao Chen; Sheng Li; Hao Huang

  NCMMSC2021, 17 Oct. 2021

  
• Description of End-to-End Dialect Identification System (accepted in INTERSPEECH2021)

  Ding Wang; Shuaishuai Ye; Xinhui Hu; Sheng Li; Xinkang Xu

  In special session of INTERSPEECH2021 (OLR2020 challenge), Sep. 2021

  
• Comparison of End-to-End Models for Joint Speaker and Speech Recognition

  K Soky; S Li; M Mimura; C Chu; T Kawahara

  IEICE-SP, 2021

  
• End-to-End Speech Translation with Cross-lingual Transfer Learning

  S Shimizu; C Chu, S Li; S Kurohashi

  NLP2021, 2021

  
• Voice-Indistinguishability: Protecting Voiceprint in Privacy-Preserving Speech Data Release

  Y. Han; S. Li; Y. Cao; Q. Ma; M. Yoshikawa

  INTERSPEECH 2020 Satellite Workshop (SLIMTS2020) (invited report), Oct. 2020, Invited

  
• System Description for Voice Privacy Challenge (Kyoto Team).

  Y. Han; S. Li; Y. Cao; M. Yoshikawa

  In special session of INTERSPEECH2020 (VoicePrivacy challenge 2020), Sep. 2020

  
• Multi-lingual transformer training for Khmer automatic speech recognition,

  K. Soky; S. Li; T. Kawahara; S. Seng

  INTERSPEECH 2020 Satellite Workshop (SLIMTS2020)., Sep. 2020, Invited

  
• Investigation of Effectively Synthesizing Code-switched Speech Using Highly Imbalanced Mix-lingual Data and mask embedding,

  S. Guo; L. Wang; S. Li; J. Zhang; C. Gong; Y. Wang; J. Dang; K. Honda

  INTERSPEECH 2020 Satellite Workshop (SLIMTS2020)., Sep. 2020

  
• A Mixture of Character and Word End-to-End System for Keyword Spotting

  H. Zhang; S. Ueno; M. Mimura; S. Li; W. Zhang; T. Kawahara

  INTERSPEECH 2020 Satellite Workshop (SLIMTS2020)(full paper), Sep. 2020, Invited

  
• Improvement of x-vector for short utterance spoken language identification,

  P. Shen, X. Lu, K. Sugiura, S. Li, H. Kawai,

  Acoustical Society of Japan, spring, 2020

  
• Joint Training End-to-End Systems for Speech and Speaker Recognition with Speaker Attributes,

  S. Li, X. Lu, R. Dabre, P. Shen and H. Kawai,

  Acoustical Society of Japan, spring, 2020

  
• End-to-End Articulatory Attribute Modeling for Low-resource Multilingual Speech Recognition

  S. Li, C. Ding, X. Lu, P. Shen and H. Kawai,

  Acoustical Society of Japan, spring, 2020

  
• End-to-end音声認識技術の研究

  李 勝

  情報通信フェア2019, Sep. 2019

  
• Investigation of multi-domain training for speech recognition,

  P. Shen; X. Lu; S. Li; H. Kawai

  Acoustical Society of Japan, spring, Mar. 2019

  
• Short utterance-based spoken language identification

  P. Shen; X. Lu; S. Li; H. Kawai

  Acoustical Society of Japan, autumn, 2018

  
• CTC 音響モデルのための knowledge distillation 方式の検討

  R.Takashima; S. Li; H. Kawai

  Acoustical Society of Japan, spring, 2018

  
• Improving CTC-based acoustic model with very deep residual neural network

  S. Li; X. Lu; R.Takashima; P. Shen; H. Kawai

  Acoustical Society of Japan, spring, 2018

  
• An Empirical Comparison of Sequence Training Methods for the Very Deep Time-delay Neural Network

  S. Li; X. Lu; R.Takashima; P. Shen; H. Kawai

  Acoustical Society of Japan, autumn, 2018

  
• CTC音響モデルのためのシーケンスレベル知識蒸留法の検討

  高島 遼一; 李 勝; 河井 恒

  IPSJ SIG-SLP, 2018

  
• Training CTC and LFMMI-based TDNN with CNTK

  Sheng LI

  NICT internal report, 2018

  
• cGAN-classifier: Conditional Generative Adversarial Nets for Classification

  P. Shen; X. Lu; S. Li; H. Kawai

  Acoustical Society of Japan, autumn, 2017

  
• Very deep convolutional residual network acoustic models for Japanese lecture transcription

  S. Li; X. Lu; P. Shen; H. Kawai

  Acoustical Society of Japan, autumn, 2017

  
• Diversity-driven Semi-supervised Ensemble DNN Acoustic Model Training (音声)

  LI Sheng; LU Xugang; SAKAI Shinsuke; KAWAHARA Tatsuya

  電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 24 Aug. 2016, 電子情報通信学会

  
• Diversity-driven Semi-supervised Ensemble DNN Acoustic Model Training

  S. Li; X. Lu; S. Sakai; T. Kawahara

  Acoustical Society of Japan, autumn, 2016

  
• Discriminative Data Selection from Multiple ASR Systems' Hypotheses for Unsupervised Acoustic Model Training (音声) -- (第17回音声言語シンポジウム)

  LI SHENG; AKITA YUYA; KAWAHARA TATSUYA

  電子情報通信学会技術研究報告 = IEICE technical report : 信学技報, 02 Dec. 2015, 電子情報通信学会

  
• Discriminative data selection from multiple ASR systems' hypotheses for unsupervised acoustic model training

  S. Li; Y. Akita; T. Kawahara

  IPSJ SIG-SLP-109-8, 2015

  
• Effective combination of multiple ASR hypotheses with CRF-based classifiers

  S. Li; Y. Akita; T. Kawahara

  Acoustical Society of Japan, autumn, 2015

  
• Incorporating divergences from hypotheses of multiple ASR systems to improve unsupervised acoustic model training

  S. Li; Y. Akita; T. Kawahara

  Acoustical Society of Japan, 2015

  
• Unsupervised Training of Deep Neural Network Acoustic Models for Lecture Transcriptions

  S. Li; Y. Akita; T. Kawahara

  Acoustical Society of Japan, autumn, 2014

  
• Classifier-based data selection for lightly-supervised training of acoustic model for lecture transcription

  S. Li; Y. Akita; T. Kawahara

  IPSJ SIG-SLP-102-4, 2014

  
• Data Selection Assisted by Caption to Improve Acoustic Modeling for Lecture Transcription

  S. Li; Y. Akita; T. Kawahara

  Acoustical Society of Japan, spring, 2014

  
• Lightly-supervised training and confidence estimation by using CRF classifiers,

  Sheng LI

  Speech and Cognition Lab., Tianjin University, host: Prof. Jianwu Dang and Prof. Kiyoshi Honda., 2014

  
• DNN-based Acoustic Modeling and Decoding for Chinese Spontaneous Speech Recogntion with HTK

  Sheng LI

  Technical-report.（Kyoto university）, 2014

  
• Automatic Transcription of Chinese Spoken Lectures

  S. Li; M. Mimura; T. Kawahara

  Acoustical Society of Japan, autumn, 2013

  
• Vocal Tract Length Normalization for Chinese Spontaneous Speech Recogntion

  Sheng LI

  Technical-report.（Kyoto university）, 2013

  
• Phoneme-level articulatory animation in pronunciation training using EMA data

  Sheng LI

  Speech Synthesis Lab., Tsinghua University, host: Prof. Zhiyong Wu., 2012

  
• The Phoneme-level Articulator Dynamics for 3D Pronunciation Animation for Chinese

  S. Li; K. Luo; L. Wang

  Bulletin of Advanced Technology Research, 2011

  
• Application of the RFID based audio service in regional navigation system

  S. Li; C. Li

  Bulletin of Advanced Technology Research, 2009

  
• Voice-Indistinguishability: Protecting Voiceprint in Privacy-Preserving Speech Data Release

  Y. Han; S. Li; Y. Cao; Q. Ma; M. Yoshikawa

  Interspeech 2020 Satellite Workshop (SLIMTS2020). (abstract paper)(invited report)

  
• Multi-lingual transformer training for Khmer automatic speech recognition

  K. Soky; S. Li; T. Kawahara; S. Seng

  Interspeech 2020 Satellite Workshop (SLIMTS2020). (abstract paper)

  

Books and Other Publications

• Voices of the Himalayas: Investigation of Speech Recognition Technology for the Tibetan Language

  Sheng Li, Single work

  Feb. 2023

  Find Kyoto University's collection of books :

  
• Bridging Eurasia: Multilingual Speech Recognition for Silkroad

  Sheng Li, Single work

  Jan. 2023

  Find Kyoto University's collection of books :

  
• Phantom in the Opera: The Vulnerabilities of Speech-based Artificial Intelligence Systems

  Sheng Li, Single work

  Nov. 2022

  Find Kyoto University's collection of books :

  
• Automatic speech recognition: Speech-to-Speech Translation

  X. Lu; S. Li; M. Fujimoto, Joint work, Chapter 3.3.2: From Shallow to Deep and Very Deep. Chapter 3.3.3: End-to-End and CTC models.

  Springer Singapore, 2020, Not refereed

  

Industrial Property Rights

• 特許7423056, 特願2020-059962, 推論器および推論器の学習方法

  李勝; ルーシュガン; 河井恒

  
• 特許7385900, 特願2019-163555, 推論器、推論プログラムおよび学習方法

  李 勝; ルー・シュガン; 丁 塵辰; 河原 達也; 河井 恒

  
• 特許7282363, 特願2019-086005, 言語識別モデルの訓練方法及び装置、並びにそのためのコンピュータプログラム

  沈鵬; ルーシュガン; 李勝; 河井恒

  
• 特許7209330, 特願2018-142418, 識別器、学習済モデル、学習方法

  李勝; ルーシュガン; 高島遼一; 沈鵬; 河井恒

  
• 特許7109771, 特願2018-044491, 音声認識システム、音声認識方法、学習済モデル

  李勝; シュガンルー・; 高島遼一; 沈鵬, 河井恒

  
• 特許7070894, 特願2018-044134, 時系列情報の学習システム、方法および ニューラルネットワークモデル

  高島 遼一; 李 勝; 河井 恒

  
• 特許6979203, 特開2019-105899, 特願2017-236626, training method

  Sheng Li; Xugang Lu; Ryoichi takashima; Peng Shen; Hisashi Kawai

  

Works

• HSoftmax: Hierachical Softmax (https://github.com/Derek-Gong/hsoftmax/)

  Zhuo Gong; Qianying Liu; Sheng Li; Zhengdong Yang; Yuhang Yang

  From 2020

  
• https://openslr.org/158/

  
• foundation models for Tibetan language

  
• Julius for speech foundation models

  https; github.com/halspeech/julius-speech-foundation-model

  
• very deep residual time-delay neural network (TDNN) with LFMMI objective implemented with MS-CNTK

  
• online speech recognition module for Erica the human robot

  
• Julius decoder with EESEN CTC acoustic model

  
• VTLN for Julius/HTK acoustic model

  
• Julius decoder with Kaldi feature extractor

  
• Julius decoder with Kaldi acoustic model

  

Awards

• 01 Dec. 2018

  第34回電気通信普及財団テレコムシステム技術学生賞
• 01 Dec. 2021

  令和３年度情報通信研究機構表彰 成績優秀表彰優秀賞（団体）
• 01 Oct. 2025

  Next Generation Star．IEEE IROS2025(IEEE IROS conference)

External funds: Kakenhi

• 意図を的確に伝える音声対話翻訳の基盤技術の創出

  Grant-in-Aid for Scientific Research (B)

  Basic Section 61030:Intelligent informatics-related

  Kyoto University

  チョ シンキ

  From 01 Apr. 2024, To 31 Mar. 2027, Granted

  感情音声認識;音声翻訳;感情を捉えた音声翻訳;音声対話翻訳;意図理解;多言語対話;メタ介入

  https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-23K28144/
• M3OLR: Towards Effective Multilingual, Multimodal and Multitask Oriental Low-resourced Language Speech Recognition

  Grant-in-Aid for Scientific Research (C)

  Basic Section 61030:Intelligent informatics-related

  Institute of Science Tokyo;National Institute of Information and Communications Technology

  李 勝

  From 01 Apr. 2023, To 31 Mar. 2026, Granted

  speech recognition;large language model;multilingual;multimodal;multitask;low-resource;Multitask;Multimodal;Multilingual;Low-resource;quality estimation;federated learning

  https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-23K11227/
• Phantom in the Opera: the Vulnerabilities of Speech Interface for Robotic Dialogue System

  Grant-in-Aid for Early-Career Scientists

  Basic Section 61050:Intelligent robotics-related

  National Institute of Information and Communications Technology

  Sheng Li

  From 01 Apr. 2021, To 31 Mar. 2023, Project Closed

  speech recognition;adversarial attack;privacy perserving;deepfake detection;spoken dialogue;federated learning;security;privacy preserving;quality estimation;spoken dialogue system;adversarial attacks;speech enhancement;Speech recognition;Dialogue robotic system;Adversarial attack;Deep neural network

  https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-21K17837/
• Next generation multilingual End-to-End speech recognition (from G30 to G200)

  Grant-in-Aid for Research Activity Start-up

  1002:Human informatics, applied informatics and related fields

  National Institute of Information and Communications Technology

  Sheng Li

  From 30 Aug. 2019, To 31 Mar. 2021, Project Closed

  speech recognition;multilingual;articulation;End-to-End;multilingual modeling;low-resourced modeling;speech translation;multi-unit modeling;language identification;disordered speech;code-switched;end-to-end;speaker diarization

  https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-19K24376/

Academic, Social Contribution

Committee Memberships

• From Aug. 2026

  Associate Editor, Human-Media Interaction Frontiers in Computer Science
• From Jun. 2026

  area chair, IEEE SLT2026
• From Oct. 2026

  co-organizer, From Robust Training to Safe Inference: New Security Paradigms in the Era of Large Vision Models（PRCV 2026 special session）
• From Apr. 2025, To Present

  senior member, IEEE
• From Jan. 2025, To Jan. 2027

  Applied Signal Processing Systems Technical Committee (ASPS TC), IEEE Signal Processing Society (SPS)
• From Jun. 2026, To Jun. 2026

  Co-organizer of OS, JSAI
• From Jan. 2026, To Jan. 2026

  meta reviewer, IEEE ICASSP2026
• To 2026

  Speech, Language, and Audio (SLA) Technical Committee (till 2026), APSIPA
• From Oct. 2025, To Oct. 2025

  session chair, IEEE IROS2025
• From Sep. 2025, To Sep. 2025

  Co-organizer of special session, IEEE RO-MAN2025
• From Jun. 2024, To Dec. 2024

  Publicity Chair of ACM Multimedia Asia 2024
• From Dec. 2024

  Co-organizer, Co-organizing ACM Multimedia Asia 2024 workshop: Multimodal, Multilingual and Multitask Modeling Technologies for Oriental Languages (M3Oriental)
• From Jul. 2024

  Session Chair of DASFAA2024
• From Apr. 2024, To Apr. 2024

  Session Chair of IEEE-ICASSP2024
• From Dec. 2023

  Co-organizing ACM Multimedia Asia 2023 workshop: Multimodal, Multilingual and Multitask Modeling Technologies for Oriental Languages (M3Oriental)
• From Sep. 2023

  Session Chair of ICANN 2023
• From Jul. 2023

  Area Chair of EMNLP 2023
• From Jul. 2023

  Area Chair of APSIPA ASC 2023
• From Oct. 2022

  Co-organizing Coling2022 workshop: when creative ai meets conversational ai (cai + cai = cai^2)
• From Jun. 2022, To Jun. 2022

  Session Chair for Speaker Odyssey2022 (Evaluation and Benchmarking Session)
• From Oct. 2020, To Oct. 2020

  Session Chair for INTERSPEECH2020 (Topics of ASR I)
• From Oct. 2020, To Oct. 2020

  Co-organizing INTERSPEECH2020 SLIMTS (Spoken Language Interaction for Mobile Transportation System) workshop