Core Speech AI Evaluation Engineer

Solventum

Pittsburgh (Allegheny County)

Hybrid

USD 106,000 - 146,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Solventum is seeking a Core Speech AI Evaluation Engineer to lead quality benchmarks for AI-produced medical notes and to build data analysis pipelines supporting NLP/NLU products. You will collaborate with global teams to cover languages like English, German, Spanish, and Italian.

The role requires at least 4 years of data analytics experience and proficiency in AI-model quality analysis and evaluation design. Remote/hybrid work with possible on-site onboarding is available.

Qualifications

  • Bachelor's Degree or higher AND 4 years of data analytics experience.
  • OR
  • High School Diploma/GED AND 8 years of data analytics experience.
  • Experience in AI-model quality analysis.
  • Additional qualifications that could help you succeed even further in this role include:
  • Experience in linguistics and/or human computer interaction.
  • Experience with natural language processing/understanding.
  • Experience with medical/clinical software development.
  • Experience in evaluation design: rater training, blinding, inter-rater agreement, sample sizing, and identifying noise.
  • Specific experience in AI-model quality and safety analysis.
  • Skill in working across team boundaries toward a shared goal.

Responsibilities

  • Leading efforts to establish quality benchmarks and processes for AI-produced medical notes, including automated LLM-as-judge evaluation
  • Implementing data analysis pipelines to produce statistical quality reports for research and development and support teams
  • Supporting overall team efforts to produce meaningful and actionable analytics on company NLP and NLU products
  • Working with teams from around the world on products supporting many languages, including English, German, Spanish, and Italian
  • Participating in a business-hours restricted on-call rotation to resolve team tool errors

Skills

Data analytics
AI-model quality analysis
Linguistics / HCI
NLP / NLU
Medical/clinical software development
Evaluation design
Inter-team collaboration

Education

Bachelor's Degree or higher
High School Diploma/GED

Job description

Job Description

By applying to this position, you acknowledge that Solventum recently publicly announced its intention to divest its Health Information Systems Business. No decision has been made regarding the ultimate structure or timing of any potential transaction, and there can be no assurance that a separation will occur. Any transaction would be subject to customary approvals, including approval by Solventum's Board of Directors, regulatory requirements and other customary conditions. This position supports the Health Information Systems Business. If a future transaction involving that business were to occur, the impact on this role would be determined as part of that process and would remain subject to applicable legal requirements, approvals and business decisions. Link: https://news.solventum.com/press-releases/detail/144/solventum-announces-intent-to-separate-its-health-information-systems-business

At Solventum, we improve lives by helping healthcare professionals do their best work. Join a team using innovation and insight to make a real impact on the future of healthcare. All roles listed are with Solventum, and we're committed to protecting your personal information-learn more in our Privacy Policy: https://www.solventum.com/en-us/home/legal/website-privacy-statement/applicant-privacy/

Core Speech AI Evaluation Engineer (Solventum)

3M Health Care is now Solventum

At Solventum, we enable better, smarter, safer healthcare to improve lives. As a new company with a long legacy of creating breakthrough solutions for our customers' toughest challenges, we pioneer game-changing innovations at the intersection of health, material and data science that change patients' lives for the better while enabling healthcare professionals to perform at their best. Because people, and their wellbeing, are at the heart of every scientific advancement we pursue.

We partner closely with the brightest minds in healthcare to ensure that every solution we create melds the latest technology with compassion and empathy. Because at Solventum, we never stop solving for you.

The Impact You'll Make in this Role
  • Leading efforts to establish quality benchmarks and processes for AI-produced medical notes, including automated LLM-as-judge evaluation
  • Implementing data analysis pipelines to produce statistical quality reports for research and development and support teams
  • Supporting overall team efforts to produce meaningful and actionable analytics on company NLP and NLU products
  • Working with teams from around the world on products supporting many languages, including English, German, Spanish, and Italian
  • Participating in a business-hours restricted on-call rotation to resolve team tool errors
Your Skills and Expertise

To set you up for success in this role from day one, Solventum requires (at a minimum) the following qualifications:

  • Bachelor's Degree or higher AND 4 years of data analytics experience

OR

  • High School Diploma/GED AND 8 years of data analytics experience
  • Experience in AI-model quality analysis

Additional qualifications that could help you succeed even further in this role include:

  • Experience in linguistics and/or human computer interaction
  • Experience with natural language processing/understanding
  • Experience with medical/clinical software development
  • Experience in evaluation design: rater training, blinding, inter-rater agreement, sample sizing, and identifying noise
  • Specific experience in AI-model quality and safety analysis
  • Skill in working across team boundaries toward a shared goal
Work Location
  • Remote/Hybrid
Travel

May include up to 5% domestic/international

Relocation Assistance

May be authorized

Employment Eligibility

Must be legally authorized to work in country of employment without sponsorship for employment visa status (e.g., H1B status).

Supporting Your Well-being

Solventum offers many programs to help you live your best life - both physically and financially. To ensure competitive pay and benefits, Solventum regularly benchmarks with other companies that are comparable in size and scope.

Onboarding Requirement: To improve the onboarding experience, you will have an opportunity to meet with your manager and other new employees as part of the Solventum new employee orientation. As a result, new employees hired for this position will be required to travel to a designated company location for on-site onboarding during their initial days of employment. Travel arrangements and related expenses will be coordinated and paid for by the company in accordance with its travel policy. Applies to new hires with a start date of October 1st 2025 or later.

Applicable to US Applicants Only: The expected compensation range for this position is $106,000 - $145,750, which includes base pay plus variable incentive pay, if eligible. This range represents a good faith estimate for this position. The specific compensation offered to a candidate may vary based on factors including, but not limited to, the candidate's relevant knowledge, training, skills, work location, and/or experience. In addition, this position may be eligible for a range of benefits (e.g., Medical, Dental & Vision, Health Savings Accounts, Health Care & Dependent Care Flexible Spending Accounts, Disability Benefits, Life Insurance, Voluntary Benefits, Paid Absences and Retirement Benefits, etc.). Additional information is available at: https://www.solventum.com/en-us/home/our-company/careers/#Total-Rewards

Responsibilities and Compliance

Responsibilities of this position include that corporate policies, procedures and security standards are complied with while performing assigned duties.

Equal Opportunity

Solventum is an equal opportunity employer. Solventum will not discriminate against any applicant for employment on the basis of race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, or veteran status.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Core Speech AI Evaluation Engineer
Core Speech AI Evaluation Engineer

Life in Pittsburgh • Pittsburgh

Hybrid
USD 106,000 - 146,000
Medical, Dental & Vision
Health Savings Accounts
Health Care & Dependent Care Flexible
+4
ENGR ADV SOFTWARE QUALITY
ENGR ADV SOFTWARE QUALITY

Solventum • East Hartford (CT)

On-site
USD 85,000 - 117,000
Software Quality Engineer/Analyst
Software Quality Engineer/Analyst

Solventum • Maplewood (NJ)

Remote
USD 106,000 - 146,000
Senior Data Engineer
Senior Data Engineer

Solventum • Maplewood (NJ)

Remote
USD 143,000 - 197,000
Senior Software Engineer (New Health Care Company)
Senior Software Engineer (New Health Care Company)

Solventum • Maplewood (NJ)

Remote
USD 106,000 - 146,000
Sr. Software Engineer
Sr. Software Engineer

Solventum Corporation • Murray (UT)

Remote
USD 124,000 - 171,000
ML Engineer
ML Engineer

Solventum • Austin (TX)

Remote
USD 90,000 - 120,000
Competitive pay
Health benefits
Flexible work hours
+1
Product Analyst
Product Analyst

Remote Jobs • Murray (UT)

Remote
USD 108,000 - 148,000
Medical, Dental & Vision
Health Savings Accounts
Retirement Benefits
Implementation Technical Engineer
Implementation Technical Engineer

Solventum • Linden (PA)

Remote
USD 96,000 - 132,000
Senior Software Business Analyst
Senior Software Business Analyst

Solventum • Maplewood (NJ)

Remote
USD 108,000 - 148,000