Data Engineer II - Enterprise Data & Analytics - Digital and Technology Partners - Remote

Mount Sinai Health System; New York (NY); Remote; USD 90,000 - 136,000; Full time

Job summary

An established industry player is seeking a Data Engineer II to join their dynamic team. In this remote role, you will design and maintain scalable data pipelines, ensuring seamless automation of data processes. You will leverage cutting-edge technologies like Docker, Kubernetes, and orchestration tools to develop AI workflows and implement DevOps best practices. This position offers the opportunity to work on impactful projects that align with scientific and clinical objectives, all while ensuring compliance with data governance policies. If you are passionate about data engineering and eager to make a difference in healthcare, this role is for you.

Qualifications

• 4+ years in data engineering and pipeline development.
• Proficiency in SQL/NoSQL, Python, and orchestration tools.

Responsibilities

• Design and maintain scalable data pipelines using Airflow and Dataiku.
• Deploy containerized applications with Docker and Kubernetes.

Skills

Data Pipeline Development

SQL/NoSQL Databases

Python

Docker

Kubernetes

AI Workflows

DevOps Practices

Cloud Services

Data Governance

HIPAA Compliance

Education

Bachelor's Degree in Computer Science

Advanced Degree

Tools

Airflow

Dataiku

Azure

Hadoop

Spark

Git

Job description

Data Engineer II - Enterprise Data & Analytics - Digital and Technology Partners - Remote

Join to apply for the Data Engineer II - Enterprise Data & Analytics - Digital and Technology Partners - Remote role at Mount Sinai Health System

Responsibilities

• Design, develop, and maintain scalable and reliable data pipelines using orchestration engines such as Airflow and Dataiku, ensuring seamless automation of data ingestion, transformation, and delivery processes.
• Deploy and maintain containerized applications and pipelines, employing technologies like Docker and Kubernetes to achieve resilient and maintainable data workflows.
• Develop, deploy, and operationalize AI workflows, including image processing, data categorization, and natural language processing (NLP) models, ensuring production-level reliability and performance.
• Implement DevOps best practices, including version control with Git, CI/CD pipelines, automated testing frameworks, and unit testing to facilitate rapid, reliable, and high-quality software deployments.
• Create and manage a scalable data architecture, including designing reference tables and deploying AI-driven mechanisms for data accuracy and currency.
• Develop and maintain data dictionaries, enforce data quality metrics, implement anomaly detection, and establish rollback processes for data errors.
• Build robust data ingestion pipelines for diverse sources like AI-generated data, flat files, and RESTful APIs.
• Collaborate with agile teams, participating in planning, stand-ups, and retrospectives.
• Create documentation, diagrams, and metadata catalogs to facilitate knowledge sharing.
• Design, implement, and manage data system monitoring, backups, and disaster recovery plans.
• Engage with stakeholders, delivering solutions aligned with scientific, research, and clinical objectives.
• Ensure compliance with HIPAA and data governance policies.
• Maintain current industry knowledge and adapt to emerging technologies.

Qualifications

Education: Bachelor’s degree in Computer Science or related; advanced degree preferred.

Experience: 4+ years in data engineering, pipeline development, and workflows, preferably in Linux environment. Proficiency with SQL/NoSQL databases, programming languages (Python, Scala, Java, Go), containerization (Docker, Kubernetes), orchestration tools (Airflow, Dataiku), AI workflows, DevOps practices, and cloud services (Azure, Hadoop, Spark). Knowledge of healthcare data standards and visualization tools is a plus.

Additional Information

This is a full-time, mid-senior level role with a salary range of $90,000 - $135,285 annually, depending on experience and qualifications.