Principal Software Engineer — Real-Time ML & LLM Systems

Hispanic Alliance for Career Enhancement; Concord (NH); On-site; USD 144,200 - 288,400; Full time

Job summary

CVS Health is seeking a Principal Software Engineer to lead hands-on development of a real-time, multi-service clinical platform on Google Cloud. You will design, build, and ship critical components for LLM-driven decision support, working across UI, backend, and ML teams.

You will own architecture, drive performance and reliability with strict SLOs, mentor engineers, and collaborate with Data Science, Product, and Security to ensure compliant, scalable solutions.

Qualifications

• 10+ years of software engineering delivering production systems at scale.
• 5+ years as hands-on Staff or Principal‑level engineer.
• Expert proficiency in Python including performance tuning and concurrency.
• 5+ years building real‑time distributed systems with latency and reliability requirements.
• 5+ years working in a major public cloud (GCP preferred).
• Deep experience with microservices architecture and REST APIs.
• 3+ years building and operating production ML systems (MLOps).
• 3+ years developing LLM‑powered applications in production (RAG/vector search).
• Experience building modern UI components with React or Angular.
• Strong CS fundamentals in distributed systems, algorithms, and concurrency.

Responsibilities

• Own architecture and execution of a real-time multi-service platform on GCP.
• Build production-grade LLM systems with prompts, evals, and retrieval.
• Deliver high-performance, sub-second latency systems with strict SLOs.
• Operationalize ML: deployment, drift detection, CI/CD quality controls.
• Design scalable APIs, microservices, and data contracts.
• Drive high-quality code reviews and engineering standards.
• Translate clinical and business needs into delivered solutions.
• Collaborate with Data Science, Product, Clinical SMEs, AI Governance, and Security.

Skills

Python

Real-time distributed systems

GCP

Microservices

REST APIs

ML/AI systems

LLM-powered apps

CI/CD

React/Angular

Distributed systems

Education

Bachelor's degree in Computer Science or related

Master's or PhD in CS/ML (plus)

Tools

Vertex AI

GKE

Cloud Run

Pub/Sub

BigQuery

IAM

Job description

CVS Health is seeking a Principal Software Engineer to lead hands-on development of a real-time, multi-service clinical platform on Google Cloud. You will design, build, and ship critical components for LLM-driven decision support, working across UI, backend, and ML teams.

You will own architecture, drive performance and reliability with strict SLOs, mentor engineers, and collaborate with Data Science, Product, and Security to ensure compliant, scalable solutions.