Data & ML Engineer (Azure)

Valiant Technologies LLC; New York (NY); On-site; USD 120,000 - 165,000; Full time

Job summary

Valiant Technologies LLC in New York is seeking a Data & ML Engineer to design, build, and operate scalable data pipelines and ML solutions in Azure. You will collaborate with data scientists and engineers to deliver reliable, production-ready components.

The role emphasizes CI/CD, MLOps, monitoring, and performance tuning for data and ML workloads, with hands-on delivery of end-to-end data platforms using Databricks, Synapse, and Data Lake.

Qualifications

• Proven hands-on experience as a Data Engineer, ML Engineer, or Cloud Platform Engineer.
• Strong experience with Microsoft Azure data engineering services.
• Proficiency in Python, SQL, and/or Scala.
• Experience developing and supporting production-grade data pipelines.
• Experience with Spark, Databricks, or comparable big data processing frameworks.
• Familiarity with CI/CD pipelines, DevOps, and MLOps practices.
• Ability to independently design, develop, troubleshoot, and deliver technical solutions.
• Strong communication skills and experience working with cross-functional teams.

Responsibilities

• Design, develop, and maintain scalable data ingestion, transformation, and ETL/ELT pipelines.
• Build and optimize Azure-based data platforms using Databricks, Data Lake, Synapse, and related technologies.
• Support deployment and operationalization of machine learning models through CI/CD and MLOps practices.
• Implement monitoring, logging, troubleshooting, and performance optimization for data and ML workloads.
• Collaborate with data scientists and engineering teams to translate requirements into production-ready solutions.
• Develop reusable, automated components following cloud engineering best practices.

Skills

Python

SQL

Scala

Databricks

Spark

CI/CD

MLOps

Azure

Data pipelines

Cross-functional

Tools

Azure Data Factory

Synapse Analytics

MLflow

Docker

Kubernetes

Delta Lake

Parquet

Job description

US Citizen, Green Card, H-1B, OPT-EAD, CPT, Other

Work Authorization:US Citizen, Green Card, OPT-EAD, CPT, H-1B, H4-EAD, Other

Job Description:

Seeking an experienced, hands-on Data & ML Engineer to support the design, development, and implementation of scalable data pipelines and machine learning solutions within an Azure cloud environment. The consultant will work collaboratively with internal engineering and data science teams to deliver reliable, secure, and production-ready solutions.

Key Responsibilities:

• Design, develop, and maintain scalable data ingestion, transformation, and ETL/ELT pipelines.
• Build and optimize Azure-based data platforms using Databricks, Data Lake, Synapse, and related technologies.
• Support deployment and operationalization of machine learning models through CI/CD and MLOps practices.
• Implement monitoring, logging, troubleshooting, and performance optimization for data and ML workloads.
• Collaborate with data scientists and engineering teams to translate requirements into production-ready solutions.
• Develop reusable, automated components following cloud engineering best practices.

Required Qualifications:

• Proven hands-on experience as a Data Engineer, ML Engineer, or Cloud Platform Engineer.
• Strong experience with Microsoft Azure data engineering services.
• Proficiency in Python, SQL, and/or Scala.
• Experience developing and supporting production-grade data pipelines.
• Experience with Spark, Databricks, or comparable big data processing frameworks.
• Familiarity with CI/CD pipelines, DevOps, and MLOps practices.
• Ability to independently design, develop, troubleshoot, and deliver technical solutions.
• Strong communication skills and experience working with cross-functional teams.

Preferred Qualifications:

• Experience with Azure Data Factory, Synapse Analytics, and Azure Data Lake.
• Experience with MLflow, Docker, Kubernetes/AKS, and containerized workloads.
• Knowledge of Delta Lake, Parquet, and other common data formats.
• Experience with infrastructure as code, orchestration tools, and cloud monitoring.
• Exposure to enterprise data integrations, including SAP CDC.
• Experience working in Agile development environments

We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.