ML Engineer: Speech & LLMs

Socket.dev

San Francisco (CA)

On-site

USD 140,000 - 210,000

Full time

6 days ago

Be an early applicant

Get more replies from employers

Send a job-specific resume in minutes.

Benefits offered by this job

Unlimited PTO

Premium health coverage

401(k) plan

Equity compensation

Hybrid in-person work

Job summary

Knowtex is building next-gen AI for clinical documentation, hiring ML Engineers/Researchers to develop medical speech-to-text or clinical LLMs. You will design experiments, train models on large proprietary clinical data, and collaborate with clinicians and engineers to deploy scalable solutions.

The role emphasizes research depth, production readiness, and improving clinical workflows across healthcare settings, with a hybrid work model and strong emphasis on responsible AI in HIPAA-regulated

Qualifications

2+ years of experience in machine learning research or ML engineering
Strong expertise in Python and PyTorch
Deep understanding of modern transformer architectures and model training techniques
Experience training, fine-tuning, or post-training large neural models
Strong experimental methodology and ability to independently design and execute research projects
Experience working with large-scale datasets and distributed training environments
Ability to translate research results into production systems
Strong understanding of model evaluation and benchmarking

Responsibilities

Research-focused role with design of experiments, datasets, evaluation systems
Train and fine-tune models for clinical documentation and speech tasks
Collaborate with engineers and clinicians to deploy models at production scale
Develop and optimize speech and/or LLM models depending on track
Move from idea to dataset to evaluation to production
Balance model quality with latency and cost in real-world settings

Skills

Python

PyTorch

Transformer architectures

Experiment design

Large-scale datasets

Production deployment

Research-to-production

Education

Bachelor's degree in Computer Science, Machine Learning, or related field

Master's/PhD in Computer Science, ML, or related

Tools

AWS

Job description

About KnowtexKnowtex is building the future of voice AI operating systems for clinicians, transforming how healthcare documentation happens at the point of care. We are experiencing rapid growth across both commercial health systems and federal healthcare, with our ambient documentation platform scaling to thousands of clinicians across hundreds of specialties.We are at an inflection point where advances in speech, language models, and clinical AI can fundamentally change how clinicians interact with technology, giving them more time to focus on what matters most: their patients.

Position Overview

We are hiring two ML Engineers / Researchers to help build the next generation of Knowtex's AI stack. We are looking for researchers with deep expertise in one of two areas: Speech & Audio: Build state-of-the-art medical speech-to-text systems using our large proprietary dataset of real-world clinical audio, with the goal of bringing more of our speech stack in-house. Large Language Models: Develop and optimize models for clinical documentation and structured clinical reasoning, improving quality, cost, latency, and control. You do not need to be an expert in both areas. We are looking for exceptional depth in either speech/audio modeling or LLMs. These are research-heavy roles with a direct path to production. You will design experiments, build datasets and evaluation systems, train and fine-tune models, and work closely with engineering and clinical teams to deploy successful approaches at scale. This role plays a central part in defining Knowtex's long-term ML strategy.

Key Responsibilities

Speech & Audio
- Develop and train speech recognition models optimized for medical conversations across hundreds of specialties
- Leverage Knowtex's large proprietary clinical audio dataset to train and fine-tune domain-specific speech models
- Research approaches for improving medical terminology recognition, speaker attribution, punctuation, timestamps, and robustness across accents and clinical environments
- Build rigorous speech evaluation frameworks beyond traditional WER, including medical terminology and clinically significant error measurement
- Explore modern speech architectures, self-supervised learning, speech foundation models, and audio-language models
- Optimize models for low-latency, real-time inference at production scale
Large Language Models
- Develop and optimize models for generating high-quality clinical documentation, including SOAP notes and specialty-specific note formats
- Build models for downstream clinical tasks such as medication extraction, orders, ICD-10 coding, E&M coding, patient visit summaries, and other structured clinical artifacts
- Evaluate open-weight and proprietary model architectures and determine where fine-tuning, distillation, structured generation, or task-specific models can outperform general-purpose API-based approaches
- Fine-tune and post-train models using Knowtex's proprietary clinical datasets
- Develop rigorous evaluation frameworks for clinical accuracy, hallucinations, completeness, formatting, and clinician preferences
- Research approaches for reducing inference cost and latency while maintaining or improving clinical quality
Across Both Tracks
- Move quickly from idea → dataset → experiment → evaluation → production
- Design experiments that clearly measure whether an approach improves real-world clinical outcomes
- Build datasets, benchmarks, and evaluation infrastructure that make model improvements measurable and reproducible
- Collaborate closely with clinicians, applied ML engineers, and platform engineers
- Take successful research beyond prototypes and help deploy models into production
- Balance model quality with latency, inference cost, reliability, and scalability

Required Qualifications

2+ years of experience in machine learning research or ML engineering, with deep expertise in speech/audio modeling or large language models
Strong expertise in Python and PyTorch
Deep understanding of modern transformer architectures and model training techniques
Experience training, fine-tuning, or post-training large neural models
Strong experimental methodology and ability to independently design and execute research projects
Experience working with large-scale datasets and distributed training environments
Ability to translate research results into production systems
Strong understanding of model evaluation and benchmarking
Bachelor’s, Master’s, or PhD in Computer Science, Machine Learning, or a related technical field, or equivalent research experience

Preferred Qualifications

For Speech Researchers

Deep experience with automatic speech recognition (ASR)
Experience training or fine-tuning Whisper, Conformer, wav2vec, or similar speech architectures
Experience with large-scale audio datasets and speech data pipelines
Familiarity with speaker diarization, voice activity detection, streaming ASR, or audio-language models
Experience optimizing speech models for real-time inference

For LLM Researchers

Experience fine-tuning or post-training open-weight LLMs
Experience with supervised fine-tuning, distillation, preference optimization, or reinforcement learning
Experience building LLM evaluation systems and model benchmarks
Experience serving and optimizing open-weight models at scale
Experience with structured generation, tool use, or agentic systems

For Either Track

Experience in healthcare AI, clinical NLP, or medical speech
Familiarity with clinical documentation workflows and medical terminology
Knowledge of coding systems such as ICD-10, CPT, E&M, or SNOMED
Publications at leading ML, NLP, or speech conferences
Experience deploying ML systems in HIPAA-compliant or regulated environments
Experience working in fast-moving startup environments where researchers own projects from experimentation through production

Technical Environment