Data Engineer II: Big Data, Cloud, & Impact (Remote)

Mount Sinai Medical Center; New York (NY); Hybrid; USD 90,000 - 135,000; Full time

Job summary

Mount Sinai Health System is seeking a Data Engineer II to design and implement data pipelines, data collection architectures, and scalable analytics infrastructure to accelerate translational research and improve clinical care. The role emphasizes cloud-based data platforms (Azure), big data stacks, and collaboration with data scientists.

You will work in a hybrid setting with on-site requirements as needed, contributing to data governance, security, and performance monitoring across multiple

Qualifications

• Bachelor's degree in Computer Science or related field; advanced degree preferred.
• 4+ years of professional data engineering experience.
• Deep knowledge of the Azure cloud platform including Azure Data Factory and Databricks.
• Strong SQL and NoSQL database knowledge: Oracle, PostgreSQL/MySQL, and MongoDB (or similar).
• Proficiency on installation and configuration of big data software and technology
• Knowledge of Hadoop, Spark, Kafka and other big data technology stacks and streaming tools.
• Familiarity with healthcare data, HL7, and Mirth are also a big plus.
• Experience with CI/CD and Observability tools. Also, experience with version control (Git).
• Strong skills in data structures, data/file formats, algorithms and object oriented design.
• Proficiency in at least 2 programming languages among Scala/Python/ Java. Must be flexible and fast to pick up new languages.
• Proficiency in Restful service development, preferably with Node JS, Django and PHP.
• Experience with micro-services and SOA.
• Knowledge of Hadoop, Spark, Kafka and other big data technology stacks and streaming tools.
• Familiarity with open source technologies and tools.
• Working knowledge of cloud architecture and implementation on Azure or AWS; serverless, VMs, cloud security, and other cloud services.

Responsibilities

• Facilitates data collection from a variety of sources, formats, and ensures data quality and accessibility for downstream users.
• Ensures data streams/pipelines are scalable, repeatable, and secure for multiple users.
• Develops as a core member of an Agile team, collaborating with application and data scientists.
• Creates infrastructure that provides insight from raw data and handles diverse sources.
• Enables big data and real-time analytical solutions leveraging emerging technologies.
• Develops prototypes and proofs of concept for selected solutions and implements complex big data projects.
• Translates business requirements into modern data pipeline solutions and documents them.

Skills

SQL

Python

Java/Scala

NoSQL databases

Agile

REST APIs

Docker/Kubernetes

Education

Bachelor's degree in Computer Science

Advanced degree preferred

Tools

Azure Data Factory

Databricks

Hadoop

Kafka

Spark

Spark Streaming

RESTful services

Node.js

Job description

Mount Sinai Health System is seeking a Data Engineer II to design and implement data pipelines, data collection architectures, and scalable analytics infrastructure to accelerate translational research and improve clinical care. The role emphasizes cloud-based data platforms (Azure), big data stacks, and collaboration with data scientists.

You will work in a hybrid setting with on-site requirements as needed, contributing to data governance, security, and performance monitoring across multiple