Director of Data Science & AI/ML Operations

CareSet; Houston (TX); Remote; USD 180,000 - 280,000; Full time

Benefits offered by this job

Unlimited PTO

Health, dental, vision, and 401(k)

Job summary

CareSet is seeking a director-level AI/ML leader to scale Medicare/Medicaid analytics. You will own end-to-end ML lifecycles on claims data, build a high-impact team, and partner with the CDAO on the roadmap. The role blends hands-on work with strategic leadership, client engagement, and governance across the organization.

Drive MLOps on Databricks, deploy LLM-based applications, and ensure production readiness, compliance, and outcomes for pharma/life sciences clients.

Qualifications

• Advanced degree in a quantitative or health data field.
• 10+ years in ML/data science in healthcare or life sciences.
• 3+ years of people management of ML/data teams.
• Hands-on ML modelling and real-world evidence experience.

Responsibilities

• Own end-to-end ML lifecycle on Medicare/Medicaid data.
• Lead a team of ML engineers and biostatisticians.
• Define AI governance and data privacy standards.
• Collaborate with clients and senior stakeholders.

Education

Advanced degree in quantitative/clinical/health data science

Tools

Databricks

Delta Lake

Unity Catalog

MLflow

Spark

SynapseML

XGBoost-on-Spark

Job description

CareSet is a thriving young healthcare data analytics company on a mission to unlock Medicare claims data for the pharmaceutical industry. We've built the analytical engine; now we need a leader to scale the machine learning and AI capabilities that power it.

About the Role

This is a player-coach position for a senior technical leader who wants to do real work and build a team around it. You will own the end-to-end machine learning lifecycle on Medicare and Medicaid claims data: from raw line-level claims through feature engineering, model development, MLOps deployment on Databricks, generative AI and LLM-based applications, and integration into analytical delivery pipelines that go directly to pharmaceutical and life sciences clients. You will also build and lead a team of machine learning engineers and biostatisticians, setting the technical standard for everything the team produces, and set the AI/ML technical roadmap in partnership with the CDAO. The work demands both deep methodological expertise in real-world evidence and the engineering discipline to move models reliably from development into production. We expect this role to be roughly 60% hands-on technical work and 40% team leadership, client engagement, and strategy, shifting toward leadership as the team grows.

Responsibilities

MLOps and Model Lifecycle Management

• Own the full model lifecycle on Databricks: data ingestion and validation, feature engineering, model training, evaluation, registration, deployment, monitoring, rollback, and retirement, with reproducibility and governance built into every stage.
• Build and maintain a versioned feature store to ensure reusable, consistently defined feature sets are available across models and use cases, reducing duplication and improving cross-model consistency.
• Establish model approval gates: define evaluation criteria and review standards a model must satisfy before advancing from experiment to registry to production; maintain documented evidence at each gate.
• Define and monitor model health indicators across accuracy, calibration, feature drift, data quality, and operational performance; set alert thresholds and own the response when models degrade in production.
• Own ML incident response: when a production model degrades or fails, lead the investigation, determine whether the response is rollback or retrain, execute the fix, and document the lesson.
• Establish release readiness standards: automated pipeline tests, reproducible builds, peer code review, and compliance documentation before any model ships to a client deliverable.
• Ensure all model development and deployment complies with Data use agreements obligations, CareSet's AI governance framework, and applicable data privacy and security standards.

Generative AI and LLM Applications

• Lead the design, deployment, and monitoring of LLM-based applications on Databricks, including retrieval-augmented generation (RAG) and agentic workflows, for internal and client use cases such as intelligent querying of claims and clinical knowledge, AI-assisted evidence synthesis, and automated literature review.
• Establish evaluation standards for generative AI systems, including accuracy and groundedness testing, hallucination controls, human review, and production monitoring, consistent with the approval gates used for traditional models.
• Define which data may be used with which models and vendors, so that all generative AI use complies with data use agreements, HIPAA, and CareSet’s AI governance framework.

Machine Learning and Analytics Delivery

• Apply advanced machine learning and statistical methods to large-scale Medicare and Medicaid claims data to generate insights that inform pharmaceutical and life sciences decisions, including treatment effectiveness, patient journey mapping, disease progression, and real-world evidence generation.
• Design and build analytical frameworks grounded in sound epidemiological principles: appropriate cohort construction, longitudinal follow-up, bias control, and confounding adjustment on observational healthcare data.
• Perform large-scale feature engineering and data preparation using Python, PySpark, and SQL within Databricks, working across billion-row administrative health datasets with the query discipline and performance optimization that scale demands.
• Translate complex model outputs into transparent, clinically meaningful insights for pharmaceutical and life sciences stakeholders, using model interpretability and explainability methods appropriate to each use case and audience.
• Integrate diverse data sources including biomarker labs, biometric data, and CMS claims; reconcile privacy-linked data via tokenization into high-quality, AI-ready analytical assets; apply re-identification risk assessment and HIPAA safe harbor protocols across use cases including line of therapy, disease staging, persistency, and HCP targeting.
• Stay current with advances in machine learning, health informatics, and real-world evidence methodology, and bring relevant techniques into the team's practice as the work evolves.
• Build, mentor, and grow a team of machine learning engineers and biostatisticians: hiring, onboarding, technical goal-setting, peer code review culture, and performance management.
• Set and enforce team standards for code quality, peer review, model validation, and documentation; drive adoption of shared engineering practices and continuous quality improvement.
• Coordinate with delivery and engineering leads to align modeling work with client timelines and resource capacity.
• Partner with the CDAO on the AI/ML roadmap, workforce planning, team structure, and technical governance as the data science function scales.

Client and Cross-Functional Collaboration

• Engage directly with pharmaceutical and life sciences clients to frame analytical problems, design research plans, and translate model outputs into clear, defensible narratives for non-technical and executive audiences.
• Present complex findings and study design decisions to senior stakeholders; model the communication standard for the team.
• Support business development through technical scoping, methodology review, and feasibility analysis for new engagements.
• Build and maintain deep expertise in CMS data environments including the CCW and the VRDC, and ensure all work is conducted in full compliance with CMS DUA obligations.

Qualifications

Required

• Advanced degree (such as Master's, PhD, MD, MPH, or equivalent) in a quantitative, clinical, public health, health data science, or closely related field.
• 10 or more years of professional experience in machine learning, data science, or applied biostatistics in healthcare or life sciences, with a track record of delivering production-grade models and evidence-generation work on large observational datasets.
• 3 or more years of direct people management experience leading ML or data science teams, including hiring and performance management of full-time employees (not only contractors or project-based teams).
• Hands-on experience building machine learning models and conducting real-world evidence or observational studies using healthcare data, including cohort construction, predictive modeling, confounding adjustment, and bias control.
• Expert-level Python and PySpark proficiency, with specific depth in query optimization, handling data skew, and leveraging window functions on massive datasets.
• Deep, hands-on Databricks experience including Delta Lake, Unity Catalog, MLflow, and large-scale Spark-based data processing.
• Proven MLOps delivery at scale on Databricks: end-to-end model lifecycle management using PySpark MLlib or scalable packages such as SynapseML and XGBoost-on-Spark, model registry, deployment pipelines, drift monitoring, model approval workflows, and incident response.
• Hands-on experience taking LLM-based applications (e.g., RAG, AI agents) from prototype to production, including evaluation, monitoring, and governance.
• Demonstrated mastery of machine learning explainability frameworks to validate model outputs and communicate feature importance clearly to healthcare and life sciences stakeholders.
• Experience establishing responsible AI standards in a regulated environment, including for generative AI systems: model governance, human oversight, auditability, and release readiness controls.
• Deep familiarity with healthcare data and claims data structures, longitudinal characteristics, and appropriate analytical methodologies for observational research.
• Working knowledge of CMS DUA obligations and the governance constraints they impose on data use, model deployment, and client deliverables.

Preferred

• Documented, hands-on experience working with CMS Chronic Condition Warehouse (CCW) data.
• Operational experience working within the VRDC secure research environment, including file handling, output review, and data governance compliance.
• Experience working directly with pharmaceutical or life sciences clients on real-world evidence engagements.
• Databricks Machine Learning Professional or Generative AI Engineer certification.
• PhD in a quantitative field and/or a record of peer-reviewed publications in health services research, epidemiology, or machine learning.
• Seniority Level: Director
• Employment Type: Full-time
• Location: Remote (United States)
• Benefits: Unlimited PTO; health, dental, vision, and 401(k).

CareSet is an equal opportunity employer. We consider all qualified applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, veteran status, or any other characteristic protected by applicable law.