Data & ML Engineer (Azure)

Quantum Technologies LLC; New York (NY); On-site; USD 120,000 - 180,000; Full time

Job summary

Quantum Technologies. LLC is seeking an experienced Data & ML Engineer to design, develop, and implement scalable data pipelines and ML solutions in an Azure cloud environment. You will collaborate with engineering and data science teams to deliver production-ready systems.

The role emphasizes ETL/ELT pipelines, Azure-based data platforms, CI/CD and MLOps practices, and robust monitoring to ensure reliable, secure analytics workloads in a modern cloud architecture.

Qualifications

• Hands-on experience as a Data Engineer, ML Engineer, or Cloud Platform Engineer.
• Strong experience with Microsoft Azure data engineering services.
• Proficiency in Python, SQL, and/or Scala.
• Experience developing and supporting production-grade data pipelines.
• Experience with Spark, Databricks, or comparable big data processing frameworks.
• Familiarity with CI/CD pipelines, DevOps, and MLOps practices.
• Ability to independently design, develop, troubleshoot, and deliver technical solutions.
• Strong communication skills and experience working with cross-functional teams.

Responsibilities

• Design, develop, and maintain scalable data ingestion, transformation, and ETL/ELT pipelines.
• Build and optimize Azure-based data platforms using Databricks, Data Lake, Synapse, and related technologies.
• Support deployment and operationalization of machine learning models through CI/CD and MLOps practices.
• Implement monitoring, logging, troubleshooting, and performance optimization for data and ML workloads.
• Collaborate with data scientists and engineering teams to translate requirements into production-ready solutions.
• Develop reusable, automated components following cloud engineering best practices.

Skills

Azure data services

Python

SQL

Scala

Spark/Databricks

CI/CD & MLOps

Cross-functional collaboration

Production pipelines

Tools

Databricks

Azure Data Factory

Docker

Kubernetes

Delta Lake

Parquet

Job description

US Citizen, Green Card, H-1B, OPT-EAD, CPT, Other

Work Authorization:US Citizen, Green Card, OPT-EAD, CPT, H-1B, H4-EAD, Other

Job Description:

Seeking an experienced, hands-on Data & ML Engineer to support the design, development, and implementation of scalable data pipelines and machine learning solutions within an Azure cloud environment. The consultant will work collaboratively with internal engineering and data science teams to deliver reliable, secure, and production-ready solutions.

Key Responsibilities:

• Design, develop, and maintain scalable data ingestion, transformation, and ETL/ELT pipelines.
• Build and optimize Azure-based data platforms using Databricks, Data Lake, Synapse, and related technologies.
• Support deployment and operationalization of machine learning models through CI/CD and MLOps practices.
• Implement monitoring, logging, troubleshooting, and performance optimization for data and ML workloads.
• Collaborate with data scientists and engineering teams to translate requirements into production-ready solutions.
• Develop reusable, automated components following cloud engineering best practices.

Required Qualifications:

• Proven hands-on experience as a Data Engineer, ML Engineer, or Cloud Platform Engineer.
• Strong experience with Microsoft Azure data engineering services.
• Proficiency in Python, SQL, and/or Scala.
• Experience developing and supporting production-grade data pipelines.
• Experience with Spark, Databricks, or comparable big data processing frameworks.
• Familiarity with CI/CD pipelines, DevOps, and MLOps practices.
• Ability to independently design, develop, troubleshoot, and deliver technical solutions.
• Strong communication skills and experience working with cross-functional teams.

Preferred Qualifications:

• Experience with Azure Data Factory, Synapse Analytics, and Azure Data Lake.
• Experience with MLflow, Docker, Kubernetes/AKS, and containerized workloads.
• Knowledge of Delta Lake, Parquet, and other common data formats.
• Experience with infrastructure as code, orchestration tools, and cloud monitoring.
• Exposure to enterprise data integrations, including SAP CDC.
• Experience working in Agile development environments

We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.