Senior Data Engineer

ZipRecruiter; San Jose (CA); Remote; USD 90,000 - 150,000; Full time

Job summary

An innovative company is seeking a Big Data & ML Infrastructure Engineer to shape the architecture of a groundbreaking AI training platform for healthcare. This role offers the chance to lead the development of cloud-based ML infrastructure, directly impacting patient outcomes and clinical research. Collaborate with industry experts in a flexible, mission-driven environment that fosters growth and technical excellence. If you are passionate about building scalable, efficient infrastructure and thrive in fast-paced settings, this opportunity is perfect for you.

Benefits

Comprehensive benefits package

Equity opportunities

Flexible work environment

Qualifications

• 3+ years of experience in Python development across the full software lifecycle.
• Deep experience with OLAPs and SQL, especially in healthcare data.

Responsibilities

• Architect and optimize ETL pipelines for petabytes of healthcare data.
• Develop scalable solutions for data processing and cloud-based ML models.

Skills

Python Development

Data Processing

ETL Pipelines

Problem-Solving

Healthcare Data

Tools

AWS Redshift

BigQuery

Snowflake

Terraform

Docker

AWS

GCP

Azure

Job description

Job DescriptionJob Description

Our client is looking for a Big Data & ML Infrastructure Engineer to help them on their mission to building the world’s largest AI training and validation platform for healthcare.

The Opportunity

As a Software Engineer (Big Data & ML Infrastructure), you will be at the heart of this company’s mission, shaping the architecture that powers its AI ecosystem. This role is ideal for a data engineering expert with a deep passion for building scalable, efficient, and secure infrastructure that can handle the complexities of real-world healthcare data. You will work closely with data scientists, product managers, and healthcare partners to design data pipelines that make AI development faster, safer, and more impactful.

Why This Role?

• Cutting-Edge Work: Lead the development of cloud-based ML infrastructure at scale, handling structured and unstructured data from legacy healthcare systems.

• High Impact: Your work will directly contribute to building AI models that improve patient outcomes and advance clinical research.

• Elite Team: Collaborate with industry-leading experts in AI, healthcare, and technology.

• Growth Potential: Join a well-funded, rapidly growing company with a culture of learning, innovation, and technical excellence.

• Flexible Location: This role is based in New York but offers remote flexibility for the right candidate.

Key Responsibilities

• Architect and optimize ETL pipelines to handle petabytes of healthcare data.

• Develop scalable solutions for data processing, storage, and cloud-based machine learning models.

• Ensure compliance with healthcare regulations while maintaining best-in-class data security.

• Partner with health system stakeholders to facilitate seamless data movement.

• Create and maintain clear documentation, ensuring transparency and auditability.

Ideal Candidate Profile

• 3+ years of Python development across the full software lifecycle.

• Deep experience with OLAPs (AWS Redshift, BigQuery, Snowflake) and SQL.

• Hands-on expertise with Terraform, Docker, and cloud-based infrastructure (AWS, GCP, Azure).

• Strong problem-solving skills and ability to work in fast-paced, ambiguous environments.

• Prior experience in healthcare data, NLP, OCR, or AI tools is a plus.

• A team player with a practical, solutions-driven mindset.

Compensation & Benefits

• Comprehensive benefits package

• Equity opportunities

• Flexible, mission-driven work environment