Lead Data Warehouse Engineer

AMERICAN MEDICAL INFORMATICS; New York (NY); On-site; USD 180,000 - 230,000; Full time

Job summary

Mount Sinai Health System seeks a Lead Data Warehouse Engineer to drive integration of multi-modal clinical data into the AIR.MS data warehouse. You will assess data integration opportunities, develop plans to integrate metadata, and execute improvements working with the AIR.MS platform teams.

The role requires 12–15 years of related experience, including 7 years in data warehousing design and development, with strong SQL, 3NF/dimensional modeling, and healthcare data familiarity.

Qualifications

• Bachelors degree in a technical discipline; Masters preferred.
• 12-15 years preferred of related experience, including 7 years of experience designing, developing, and maintaining relational databases, data pipelines, and dimensional/OLAP warehouses.

Responsibilities

• Design databases and pipelines balancing functionality, performance, cost, and development time; evaluate options with the product manager.
• Build, test, and maintain data pipelines to extract, transform, and load data into a multi-modal repository.
• Lead a team; contribute to project planning, work breakdown, and release management.
• Promote standards, patterns, DevOps/SDLC practices, and operational procedures for pipelines and warehouse maintenance.
• Mentor junior engineers in data warehousing and operational support.
• Lead data management processes, including loading flat files (csv, tsv, JSON).
• Lead design sessions, code reviews, and produce technical documentation.
• Tune database objects, stored procedures, and pipelines to optimize performance and costs.
• Monitor operations; provide after-hours support as needed.
• Collaborate with DBAs on backups, performance tuning, and patching.
• Provide high-quality service to researchers, clinicians, and internal partners.

Skills

Database design

Data pipelines

Team leadership

DevOps/SDLC

SQL

ETL

Performance tuning

Big data (Hadoop/Spark)

Data modeling (OMOP/4NF)

Data security/compliance

Education

Bachelor's degree

Master's preferred

Tools

SAP HANA

MySQL

SQL scripting

UDFs

Git

JIRA

Confluence

Job description

The Scientific Computing and Data team at the Icahn School of Medicine at Mount Sinai partners with scientists and clinicians to accelerate scientific discovery.The AI-Ready.MountSinai (AIR.MS)platformcontainspatientdata generated fromthe clinical careprocesses atthe MountSinaiHealthSystem.AIRMS is a cloud-based,high-performanceSAP HANAdata platformwith electronic health record (EHR) data in an OMOP data format. It also contains metadata from and links to raw data sets in other modalities, such as radiology, genomics and pathology. Researchers can access the AIR.MS data throughdirectdatabaseaccess, AIagents, cohort query tools and the Minerva high-performance computer.AIR.MS ingests OHDSI’s Observational Medical Outcomes Partnership (OMOP)-formatted EHR data from the Mount Sinai Data Warehouse. AIR.MS is integrated with Minerva, ahigh-performance computerwith>21 petaflops of raw computational power and the raw radiology, genomics and pathology data sets. Anexpert team of 20+ PhD/MDcomputational scientists, biomedical informaticists and computerscientistspartner with researchers and clinicians toeffectively and efficientlyutilizethese resourcesfor translational science.Lead Data Warehouse Engineeris a senior technical specialist responsible for leading the ongoing integration of multi-modal clinical data into the AIR.MS data warehouse. TheLead Data Warehouse Engineerwill assess the state of data integration, identify opportunities for data integration based on researcher’s priorities, develop a plan to integrate metadata and data, and execute on the data integration. The incumbent will work collaboratively with other members of the AIR.MS data platform and Mount Sinai Data Warehouse teams to lead technical efforts for the integration of multi-modal data sets resulting in expanded AIR.MS functionality. The AIR.MS data warehouse is built on the SAP HANA technology stack and MSDW is built on MySQL.

Responsibilities

• Design databases and pipelines that balance functionality, performance, cost, and development time; evaluate technical options with the product manager.
• Design, build, test, and maintain data pipelines that extract/capture data from source systems, transform and augment those data, and integrate it into a multi-modal data repository.
• Serve as a team leader; contribute to project planning, work breakdown, dependency sequencing, and release management.
• Develop and promote standards, conventions, design patterns, DevOps/SDLC best practices, and operational procedures for pipelines and warehouse maintenance.
• Mentor junior engineers in data warehousing, data engineering skills, and operational support.
• Design, build, and maintain data management processes, including loading flat files (csv, tsv, pipe-delimited, JSON).
• Lead design sessions, code walkthroughs, peer reviews, and produce technical documentation.
• Tune database objects, stored procedures, and pipelines to optimize performance and minimize compute and storage costs.
• Monitor database and pipeline operations; lead troubleshooting and remediation of failures; provide occasional after-hours on-call support.
• Collaborate with DBAs and system administrators on backups, performance tuning, statistics/index maintenance, and patching.
• Provide high-quality customer service to researchers, clinicians, and internal partners; maintain a sciencedriven, customer-focused approach.
• Ensure patient privacy and data security in compliance with IRB & cybersecurity policies, HIPAA, 42 CFR Part 2, NYS Article 27-F, and other regulations.
• Stay current with emerging technologies to improve capabilities, efficiency, quality, or cost.
• Identify improvements in procedures, technology, compliance, and data privacy/security.
• Periodically assist DBAs with user provisioning, backups, restorations, capacity planning, and performance monitoring.
• Perform related duties as assigned.

Qualifications

• Bachelors degree in a technical discipline; Masters degree preferred
• 12-15 years preferred of related experience, including 7 years of experience designing, developing, and maintaining relational databases, data pipelines, and dimensional/OLAP warehouses.

Preferred:

• Expert knowledge of data warehousing: 3NF & dimensional modeling (fact table types, SCDs), change data capture, incremental loads, data lineage, source-to-target mappings, pattern-based & parameter-driven development.
• Experience working with healthcare data
• Expert-level experience with data engineering technologies: SQL, indexing, stored procedures, UDFs, sequences, dynamic SQL, data transformation tools, job orchestration tools for data processing.
• Experience with DevOps/SDLC best practices; Agile (Scrum, Kanban) with JIRA and Confluence; version control with git.
• Strong communication and customer service skills for working with researchers, clinicians, administrators, and IT staff.
• Excellent critical thinking, problem-solving, multitasking, and collaboration skills; ability to work independently in a fast-paced environment.
• Preferred experience with healthcare data (EHR, billing/claims, cost accounting), Epic Clarity/Caboodle, data models (OMOP, i2b2, PCORnet).
• Experience with database administration: configuration, performance tuning, partitioning, materialized views, permissions, backups & restorations.
• Knowledge of Hadoop, Spark, Kafka and other big data technology stacks and streaming tools.

About Us

Strength through Unity and Inclusion

The Mount Sinai Health System is committed to fostering an environment where everyone can contribute to excellence. We share a common dedication to delivering outstanding patient care. When you join us, you become part of Mount Sinai’s unparalleled legacy of achievement, education, and innovation as we work together to transform healthcare. We encourage all team members to actively participate in creating a culture that ensures fair access to opportunities, promotes inclusive practices, and supports the success of every individual.

At Mount Sinai, our leaders are committed to fostering a workplace where all employees feel valued, respected, and empowered to grow. We strive to create an environment where collaboration, fairness, and continuous learning drive positive change, improving the well-being of our staff, patients, and organization. Our leaders are expected to challenge outdated practices, promote a culture of respect, and work toward meaningful improvements that enhance patient care and workplace experiences. We are dedicated to building a supportive and welcoming environment where everyone has the opportunity to thrive and advance professionally. Explore this opportunity and be part of the next chapter in our history.

About the Mount Sinai Health System:

Mount Sinai Health System is one of the largest academic medical systems in the New York metro area, with more than 48,000 employees working across eight hospitals, more than 400 outpatient practices, more than 300 labs, a school of nursing, and a leading school of medicine and graduate education. Mount Sinai advances health for all people, everywhere, by taking on the most complex health care challenges of our time — discovering and applying new scientific learning and knowledge; developing safer, more effective treatments; educating the next generation of medical leaders and innovators; and supporting local communities by delivering high-quality care to all who need it. Through the integration of its hospitals, labs, and schools, Mount Sinai offers comprehensive health care solutions from birth through geriatrics, leveraging innovative approaches such as artificial intelligence and informatics while keeping patients’ medical and emotional needs at the center of all treatment. The Health System includes more than 9,000 primary and specialty care physicians; 13 joint-venture outpatient surgery centers throughout the five boroughs of New York City, Westchester, Long Island, and Florida; and more than 30 affiliated community health centers. We are consistently ranked by U.S. News & World Report's Best Hospitals, receiving high "Honor Roll" status, and are highly ranked: No. 1 in Geriatrics, top 5 in Cardiology/Heart Surgery, and top 20 in Diabetes/Endocrinology, Gastroenterology/GI Surgery, Neurology/Neurosurgery, Orthopedics, Pulmonology/Lung Surgery, Rehabilitation, and Urology. New York Eye and Ear Infirmary of Mount Sinai is ranked No. 12 in Ophthalmology. U.S. News & World Report’s “Best Children’s Hospitals” ranks Mount Sinai Kravis Children's Hospital among the country’s best in several pediatric specialties. The Icahn School of Medicine at Mount Sinai is ranked No. 11 nationwide in National Institutes of Health funding and in the 99th percentile in research dollars per investigator according to the Association of American Medical Colleges. Newsweek’s “The World’s Best Smart Hospitals” ranks The Mount Sinai Hospital as No. 1 in New York and in the top five globally, and Mount Sinai Morningside in the top 20 globally.

Equal Opportunity Employer

The Mount Sinai Health System is an equal opportunity employer, complying with all applicable federal civil rights laws. We do not discriminate, exclude, or treat individuals differently based on race, color, national origin, age, religion, disability, sex, sexual orientation, gender, veteran status, or any other characteristic protected by law. We are deeply committed to fostering an environment where all faculty, staff, students, trainees, patients, visitors, and the communities we serve feel respected and supported. Our goal is to create a healthcare and learning institution that actively works to remove barriers, address challenges, and promote fairness in all aspects of our organization.