Staff Software Engineer, Data Platform Engineering

Paradigm Health, Inc.; New York, Northern (NY, KY); Hybrid; USD 180,000 - 240,000; Full time

Job summary

Paradigm Health, Inc. is building a scalable data platform to ingest, unify, and analyze clinical data from multiple sources.

The Staff Engineer will lead cross-functional teams to design end-to-end data pipelines, ensure governance, and drive architectural decisions that support analytics and AI use cases. You will collaborate with health systems, AI, and BI teams to set a technical strategy, reduce data friction, and mentor engineers across the organization.

Qualifications

• 10+ years of experience in data/engineering roles.
• Deep expertise in SQL, data modeling, and analytics.
• Experience with Spark or Flink for batch and streaming analytics.
• Proficiency with cloud platforms and infrastructure as code (AWS, Terraform).
• Strong leadership and cross-team collaboration.
• Proficiency in Python or similar languages and AI-assisted tooling.

Responsibilities

• Lead cross-team engineering initiatives from start to production.
• Shape data platform architecture and governance.
• Build and maintain batch and streaming pipelines on the cloud.
• Mentor engineers and drive best practices across teams.
• Ensure data privacy and governance with cross-functional teams.
• Align with product roadmap to deliver business value.

Skills

Data engineering

SQL

Distributed systems

Cloud platforms

Python

Data modeling

Data mesh

Spark/Flink

Leadership

Job description

Paradigm Health is rebuilding the clinical research ecosystem by enabling equitable access to trials for all patients. Our platform enhances trial efficiency and reduces the barriers to participation for healthcare providers. Incubated by ARCH Venture Partners and backed by leading healthcare and life sciences investors, Paradigm’s seamless infrastructure implemented at healthcare provider organizations, will bring potentially life-saving therapies to patients faster.

Our team hails from a broad range of disciplines and is committed to the company’s mission to create equitable access to clinical trials for any patient, anywhere. Join us, and bring your expertise, passion, creativity, and drive as we work together to realize this mission.

The Data Engineering team is building Paradigm’s platform for clinical care information, ingesting data from multiple sources, including Electronic Health Records, lab providers, Health Information Exchanges, and more. As Paradigm’s data needs evolve with an expanding population, research, and health system footprint, Data Engineering will scale its architecture to support downstream application, analytics and AI use cases.

As a Staff Engineer on the Data team, you’ll operate at the intersection of software, data and AI, partnering closely with product, AI and infra teams to improve the breadth, quality, and completeness of our data. We're seeking someone with experience in database technologies, distributed systems, and cloud computing, who can focus on the end-to-end technical architecture, drive operational excellence and design extensible platform abstractions to support the future growth of the company.

This is a high visibility and high impact technical leadership role. You will be working with our health systems integration, AI and business intelligence teams to maintain a high technical bar, make the right business and technology tradeoffs, manage technical debt and mentor engineers cross functionally. You will set the technical strategy and incremental plan to redesign the current legacy and messy data pipelines into a coherent modular and reusable data platform.

This role also requires someone willing to learn and dive deep into the clinical data domain and embodies customer obsession as a core leadership principle.

What you’ll do:

• Drive large cross team initiatives within engineering from start to production

• Owns cross team effects of their work, breaks down work and drives with the appropriate stakeholders

• Communicates effectively across the organization

• Architectural alignment at the company level

• Defines and champions your teams best practices to the organization

• Influences product roadmap and shapes what we should be building to be impactful for customer

• Works following the lean execution model through iterations and feedback loops to deliver business value.

• Shape the engineering vision while owning the data models and toolchain for our data platform.

• Build and maintain robust batch and streaming pipelines, leveraging cloud technologies and distributed systems.

• Develop tools and processes to deploy configurational data and self-service tools.

• Enhance the maturity of our monitoring systems to improve visibility and failure detection within our infrastructure.

• Become a subject matter expert on our diverse datasets and associated tooling.

• Ensure privacy and security in the governance of our data in collaboration with cross-functional teams.

• Advocate for a scalable, maintainable data platform that supports our evolving business needs.

• Understands business priorities / deadlines and executes on initiatives that are most impactful, while communicating to the various engineering teams your goals and set expectations correctly.

Who you are:

• 10+ Years of experience

• Curious - you look at your work beyond the surface, and care about why and how things work

• Get Stuff Done - Say what you will do, and do that. Deliver work timely and reliably.

• Growth Mindset - Believe in potential, embrace challenges, learn from mistakes.

• Teachable - Willing to learn from anyone and want to teach others

• Respectful - All viewpoints are valuable - we listen and provide appropriate feedback

• Collaborative - Prioritizes unblocking teammates, users, stakeholders

• Extensive experience driving technical projects, prioritizing work, identifying dependencies, and facilitating technical decisions across teams.

• Deep expertise in SQL, data analysis, and data modeling, with a strong understanding of distributed systems and cloud environments.

• Experience with data processing technologies like Spark or Flink - batch and microbatch analytics

• Able to evangelize and use data mesh design principles to build data products.

• Proven track record in designing systems that are scalable, available, maintainable, and performant.

• Proficiency in coding (Python or other languages) and knows when to apply AI assisted development tools

• Proficiency building infrastructure using major cloud providers e.g. AWS and infrastructure as code solutions e.g. Terraform

At Paradigm Health, we are committed to providing equal employment opportunities to all qualified individuals. We encourage and welcome candidates from all backgrounds and perspectives to apply for our open positions. We are interested in all qualified individuals and ensure that all employment decisions are based on job-related factors such as skills, experience, and qualifications.