ML Research Engineer Speech

Blue Machines AI; Bengaluru; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Blue Machines (ApnaTime Tech Pvt. Ltd.) is seeking an ML Research Engineer, Speech, to build and improve speech models for Indian languages and BFSI vocabulary.

You will pre-train from scratch, fine-tune, and ship models with tight latency constraints in a production setting. You will contribute to data pipelines, evaluation metrics, and streaming architectures while collaborating with senior researchers and the CTO organization.

Qualifications

• 1–3 years of ML experience with at least one project in speech/audio
• B.Tech/M.Tech/MS in CS, EE or related field; PhD not required
• Strong Python and PyTorch; implement model architecture and training loop from scratch
• Knowledge of speech basics: spectrograms, MFCC/mel features, CTC, attention-based seq2seq, vocoders
• Hands-on with NeMo, ESPnet, Hugging Face Transformers/Audio, fairseq, Coqui, k2/icefall
• Trained at least one model from random initialisation, including multi-GPU, mixed precision, and experiment tracking (W&B, MLflow)
• Careful with evaluation: know why a WER number can mislead and check data

Responsibilities

• Pre-train STT models from scratch and fine-tune for Indian languages, Hinglish, BFSI vocabulary
• Train TTS models and extend to new voices, languages and speaking styles
• Build speech-to-speech components: audio tokenizers and codecs, streaming decoders
• Turn papers and ideas into PyTorch code, train from random initialisation
• Build data pipelines: cleaning, segmentation, augmentation, pseudo-labelling
• Maintain evaluation suites: WER/CER, MOS, latency, and entity accuracy
• Analyse production errors and close the loop with the platform team

Skills

Python

PyTorch

Speech/Audio

NeMo

ESPnet

Transformers

k2/icefall

Random init

WER evaluation

Education

B.Tech/M.Tech/MS in CS/EE

Tools

W&B

MLflow

Job description

Job Description

About Blue Machines and the Speech Research Team

Blue Machines (ApnaTime Tech Pvt. Ltd.) builds enterprise voice AI agents for banks, insurers, healthcare and telecom companies. Our platform has handled tens of millions of production voice minutes, is ISO 27001/27701 and SOC 2 Type II certified, and runs both on cloud and on-prem.

We already run our own models in production: Aurora (STT tuned for BFSI), Floe (language-switch detection), end-of-utterance models, and noise cancellation, all on BM Zap, our low-latency voice framework. The Speech Research team takes this further: better accuracy on Indian languages and code-mixed speech, natural expressive TTS, and full-duplex speech-to-speech models that cut latency across the whole conversation.

What makes this work different

• Real traffic: your model ships to live enterprise calls, not a leaderboard
• Hard problems: 8 kHz telephony audio, Hinglish and code-switching, accents, noisy environments, domain entities (loan numbers, policy IDs, names)
• Tight latency budgets: streaming inference where every 50 ms is noticed by callers
• A full loop: data, training, evaluation, serving and production feedback in one team

Location: Bengaluru (on-site / hybrid) · Team: Speech Research, reporting into the CTO org

Role 1: ML Research Engineer, Speech (1–3 years)

You will build speech models from the ground up: implement architectures from scratch, pre-train them on large audio corpora, then adapt and ship them. Youll own well-scoped experiments from data to deployed checkpoint, with guidance from senior researchers.

What you will do

• Pre-train STT models from scratch (Conformer/Zipformer encoders, CTC/RNN-T/TDT decoders, self-supervised pre-training like wav2vec 2.0/HuBERT/BEST-RQ), then fine-tune them for Indian languages, Hinglish and BFSI vocabulary
• Train TTS models from scratch (flow-matching, diffusion, neural codec LMs, vocoders) and extend them to new voices, languages and speaking styles
• Build speech-to-speech components from scratch: audio tokenizers and codecs, speech encoders connected to LLMs, streaming decoders
• Turn papers and new ideas into working PyTorch code: write the model, tokenizer and training loop yourself, and train from random initialisation
• Build data pipelines: cleaning, segmentation, forced alignment, pseudo-labelling, augmentation (noise, codecs, 8 kHz telephony simulation)
• Maintain evaluation suites: WER/CER, entity error rate, code-switch accuracy, MOS and speaker similarity, latency (time to first token / byte)
• Optimise models for serving: quantisation, streaming chunking, ONNX/TensorRT export, batching on GPUs
• Analyse production errors, turn them into test sets, and close the loop with the platform team
• Write clear experiment reports and share findings in weekly research reviews

What you bring

• 1–3 years in ML, with at least one real project in speech or audio (industry, research lab, or a strong thesis)
• B.Tech/M.Tech/MS in CS, EE or a related field; a PhD is not required
• Strong Python and PyTorch; able to implement a model architecture and training loop from scratch, not just call APIs or fine-tune checkpoints
• Working knowledge of speech basics: spectrograms, MFCC/mel features, CTC, attention-based seq2seq, vocoders
• Hands-on with at least one of: NeMo, ESPnet, Hugging Face Transformers/Audio, fairseq, Coqui, k2/icefall
• Has trained at least one model from random initialisation (speech, audio or language), including multi-GPU jobs, mixed precision and experiment tracking (W&B, MLflow)
• Careful with evaluation: you know why a WER number can mislead, and you check your data

Nice to have

• Speak or understand Hindi or another Indian language
• Publications, Kaggle/benchmark results, or open-source contributions in speech
• Exposure to real-time audio (WebRTC, SIP, streaming inference) or telephony audio
• Familiarity with LLM fine-tuning (LoRA, SFT) or audio-language models

What success looks like in 6 months

• Contributed to a pre-trained-from-scratch model that shipped to production with a measured gain on a customer-relevant test set
• Owns a piece of the evaluation or data pipeline that others rely on
• Runs experiments independently and reports results the team can trust

Requirements