Azure Data & ML Engineer: ETL, Databricks & MLOps

Valiant Technologies LLC; New York (NY); On-site; USD 120,000 - 165,000; Full time

Job summary

Valiant Technologies LLC in New York is seeking a Data & ML Engineer to design, build, and operate scalable data pipelines and ML solutions in Azure. You will collaborate with data scientists and engineers to deliver reliable, production-ready components.

The role emphasizes CI/CD, MLOps, monitoring, and performance tuning for data and ML workloads, with hands-on delivery of end-to-end data platforms using Databricks, Synapse, and Data Lake.

Qualifications

• Proven hands-on experience as a Data Engineer, ML Engineer, or Cloud Platform Engineer.
• Strong experience with Microsoft Azure data engineering services.
• Proficiency in Python, SQL, and/or Scala.
• Experience developing and supporting production-grade data pipelines.
• Experience with Spark, Databricks, or comparable big data processing frameworks.
• Familiarity with CI/CD pipelines, DevOps, and MLOps practices.
• Ability to independently design, develop, troubleshoot, and deliver technical solutions.
• Strong communication skills and experience working with cross-functional teams.

Responsibilities

• Design, develop, and maintain scalable data ingestion, transformation, and ETL/ELT pipelines.
• Build and optimize Azure-based data platforms using Databricks, Data Lake, Synapse, and related technologies.
• Support deployment and operationalization of machine learning models through CI/CD and MLOps practices.
• Implement monitoring, logging, troubleshooting, and performance optimization for data and ML workloads.
• Collaborate with data scientists and engineering teams to translate requirements into production-ready solutions.
• Develop reusable, automated components following cloud engineering best practices.

Skills

Python

SQL

Scala

Databricks

Spark

CI/CD

MLOps

Azure

Data pipelines

Cross-functional

Tools

Azure Data Factory

Synapse Analytics

MLflow

Docker

Kubernetes

Delta Lake

Parquet

Job description

Valiant Technologies LLC in New York is seeking a Data & ML Engineer to design, build, and operate scalable data pipelines and ML solutions in Azure. You will collaborate with data scientists and engineers to deliver reliable, production-ready components.

The role emphasizes CI/CD, MLOps, monitoring, and performance tuning for data and ML workloads, with hands-on delivery of end-to-end data platforms using Databricks, Synapse, and Data Lake.