Data Engineer II - Digital and Technology Partners - Hybrid/Remote

JobCubby; New York (NY); Hybrid; USD 90,000 - 135,000; Full time

Job summary

Mount Sinai Health System in New York City seeks a data engineering professional to design and implement scalable data pipelines, ensuring data quality and security across cloud and on‑premises environments.

You will collaborate in an Agile team with developers, data scientists, and DBAs, and work with Azure Data Factory, Databricks, Spark, and streaming technologies to support research and clinical data initiatives.

Qualifications

• Bachelor's degree in Computer Science or related discipline; Advanced degree preferred.
• 4+ years relevant professional development experience, preferably in a Data Engineering role.
• In depth knowledge of the Azure cloud platform including Azure Data Factory and Databricks.
• Strong SQL and NoSQL Database Knowledge: Oracle, PostgreSQL/MYSQL, and Mongo DB (or similar).
• Proficiency on installation and configuration of big data software and technology
• Knowledge of Hadoop, Spark, Kafka and other big data technology stacks and streaming tools.
• Knowledge of healthcare data, HL7, and Mirth are also a big plus.
• Experience with CI/CD and Observability tools. Also, experience with version control (Git).
• Strong skills in data structures, data/file formats, algorithms and object oriented design.
• Proficiency with at least 2 programming languages among Scala/Python/ Java. Must be flexible and fast to pick up new languages.
• Proficiency in Restful service development, preferably with Node JS, Django and PHP.
• Experience with micro-services and SOA.
• Familiarity with open source technologies and tools.
• Working knowledge of cloud architecture and implementation on Azure or AWS; cloud services is a plus.
• Experience in Agile methodology; JIRA is a plus.

Responsibilities

• Facilitates data collection from various sources, ensuring data quality and quick access for downstream users.
• Ensures data streams/pipelines are scalable, repeatable, and secure for multiple users.
• Develops as a core member of an Agile team with Developers, DB Developers, and Data Scientists.
• Creates infrastructure that provides insight from raw data and handles diverse data sources.
• Enables big data and real-time analytical solutions leveraging emerging technologies.
• Develops prototypes and proofs of concept for selected solutions.

Skills

Data pipeline design

Agile teamwork

RESTful services

Cloud architecture

Big data processing

Education

Bachelor's degree in Computer Science

Advanced degree preferred

Tools

Azure Data Factory

Databricks

Hadoop

Spark

Kafka

Node.js

Django

PHP

Git

JIRA

Job description

DescriptionThis is a remote/hybrid position, may require onsite for interviews and occasionally onsite as needed

Location: 150 E 42nd Street, New York, NY

Responsibilities

• Facilitates data collection from a variety of different sources, getting it in the right formats, assuring that it adheres to data quality standards, and assuring that downstream users can get that data quickly and with a common standard interface.
• Ensures that data streams/pipelines are scalable, repeatable, and secure, and can serve multiple users within the Institute.
• Develops as a core member of an Agile team, using Agile tools and methodology. Work closely with other team members including Application Developers, Database Developers, and Data Scientists.
• Responsible for creating the infrastructure that provides insight from raw data and handles diverse sources of data seamlessly.
• Enables big data and batch/real-time analytical solutions that leverage emerging technologies.
• Additional responsibilities include developing prototypes and proof of concepts for the selected solutions, and implementing complex big data projects with a focus on collecting, parsing, and managing large sets of data using multiple platforms to allow for Research and Data Science initiatives.
• Translates business requirements into modern data pipeline solutions. Create centralized documents and diagrams of all solutions.
• Creates a data catalog store of all metadata.
• Designs and implements monitoring, backup, and disaster recovery of data systems.
• Approaches all relationships with a world-class customer service approach. Maintains a customer-focused approach with users to provide solutions that are science/research-driven.
• Responsible for the integrity and security of data in all forms of storage throughout the Data Architecture.
• Works with other IT professionals through Mount Sinai effectively. Comply with the Institutional Review Board and HIPAA to follow all applicable policies and procedures.
• Assists in the development of standards and procedures affecting data management, design and maintenance.Documents all standards and procedures.
• Provides presentations and training to other team members in the above.
• Possesses an extremely flexible attitude. Willing to work with multiple types of technologies and languages with an open mind and without technology bias. Continuous interest in updating skill sets and knowledge of trends in the Big Data Technology space.
• Other duties as assigned.

Qualifications

• Bachelors degree in Computer Science or a related discipline; Advanced degree preferred.
• 4+ years relevant professional development experience, preferably in a Data Engineering role.
• In depth knowledge of the Azure cloud platform including Azure Data Factory and Databricks.
• Strong SQL and NoSQL Database Knowledge: Oracle, PostgreSQL/MYSQL, and Mongo DB (or similar).
• Proficiency on installation and configuration of big data software and technology
• Knowledge of Hadoop, Spark, Kafka and other big data technology stacks and streaming tools.
• Knowledge of healthcare data, HL7, and Mirth are also a big plus.
• Experience with CI/CD and Observability tools. Also, experience with version control (Git).
• Strong skills in data structures, data/file formats, algorithms and object oriented design.
• Proficiency with at least 2 programming languages among Scala/Python/ Java. Must be flexible and fast to pick up new languages.
• Proficiency in Restful service development, preferably with Node JS, Django and PHP.
• Experience with micro-services and SOA.
• Proficiency on installation and configuration of big data software and technology
• Knowledge of Hadoop, Spark, Kafka and other big data technology stacks and streaming tools.
• Familiarity with and the ability to leverage a wide variety of open source technologies and tools.
• Working knowledge of cloud architecture and implementation on Azure or AWS, is a big plus. Experience with serverless computing, creating VMs, cloud security, and other cloud services is also a big plus.
• Experience working in an Agile methodology. Experience working with JIRA is a plus.

Employer DescriptionStrength through Unity and Inclusion

The Mount Sinai Health System is committed to fostering an environment where everyone can contribute to excellence. We share a common dedication to delivering outstanding patient care. When you join us, you become part of Mount Sinai’s unparalleled legacy of achievement, education, and innovation as we work together to transform healthcare. We encourage all team members to actively participate in creating a culture that ensures fair access to opportunities, promotes inclusive practices, and supports the success of every individual.

At Mount Sinai, our leaders are committed to fostering a workplace where all employees feel valued, respected, and empowered to grow. We strive to create an environment where collaboration, fairness, and continuous learning drive positive change, improving the well‑being of our staff, patients, and organization. Our leaders are expected to challenge outdated practices, promote a culture of respect, and work toward meaningful improvements that enhance patient care and workplace experiences. We are dedicated to building a supportive and welcoming environment where everyone has the opportunity to thrive and advance professionally. Explore this opportunity and be part of the next chapter in our history.

About the Mount Sinai Health System

Mount Sinai Health System is one of the largest academic medical systems in the New York metro area, with more than 48,000 employees working across eight hospitals, more than 400 outpatient practices, more than 300 labs, a school of nursing, and a leading school of medicine and graduate education. Mount Sinai advances health for all people, everywhere, by taking on the most complex health care challenges of our time — discovering and applying new scientific learning and knowledge; developing safer, more effective treatments; educating the next generation of medical leaders and innovators; and supporting local communities by delivering high-quality care to all who need it. Through the integration of its hospitals, labs, and schools, Mount Sinai offers comprehensive health care solutions from birth through geriatrics, leveraging innovative approaches such as artificial intelligence and informatics while keeping patients’ medical and emotional needs at the center of all treatment. The Health System includes more than 9,000 primary and specialty care physicians; 13 joint‑venture outpatient surgery centers throughout the five boroughs of New York City, Westchester, Long Island, and Florida; and more than 30 affiliated community health centers. We are consistently ranked by U.S. News & World Report’s Best Hospitals, receiving high "Honor Roll" status, and are highly ranked: No. 1 in Geriatrics, top 5 in Cardiology/Heart Surgery, and top 20 in Diabetes/Endocrinology, Gastroenterology/GI Surgery, Neurology/Neurosurgery, Orthopedics, Pulmonology/Lung Surgery, Rehabilitation, and Urology. New York Eye and Ear Infirmary of Mount Sinai is ranked No. 12 in Ophthalmology. U.S. News & World Report’s “Best Children’s Hospitals” ranks Mount Sinai Kravis Children’s Hospital among the country’s best in several pediatric specialties. The Icahn School of Medicine at Mount Sinai is ranked No. 11 nationwide in National Institutes of Health funding and in the 99th percentile in research dollars per investigator according to the Association of American Medical Colleges. Newsweek’s “The World’s Best Smart Hospitals” ranks The Mount Sinai Hospital as No. 1 in New York and in the top five globally, and Mount Sinai Morningside in the top 20 globally.

Equal Opportunity Employer

The Mount Sinai Health System is an equal opportunity employer, complying with all applicable federal civil rights laws. We do not discriminate, exclude, or treat individuals differently based on race, color, national origin, age, religion, disability, sex, sexual orientation, gender, veteran status, or any other characteristic protected by law. We are deeply committed to fostering an environment where all faculty, staff, students, trainees, patients, visitors, and the communities we serve feel respected and supported. Our goal is to create a healthcare and learning institution that actively works to remove barriers, address challenges, and promote fairness in all aspects of our organization.

Compensation

The Mount Sinai Health System (MSHS) provides salary ranges that comply with the New York City Law on Salary Transparency in Job Advertisements. The salary range for the role is $90000 - $135285 Annually. Actual salaries depend on a variety of factors, including experience, education, and operational need. The salary range or contractual rate listed does not include bonuses/incentive, differential pay or other forms of compensation or benefits.