Senior Azure Data Engineer: Lakehouse & Spark

ValueMomentum; New Jersey; On-site; USD 140,000 - 190,000; Full time

Job summary

ValueMomentum is seeking an experienced Azure Data Engineer to design, build, and support scalable data pipelines on Microsoft Azure. You will work on batch and near-real-time ingestion, data lake/warehouse integration, and operational data workloads using Azure-native services.

Strong PySpark, Python, SQL, and Azure stack expertise are required. The role emphasizes producing reliable, secure, production-ready data pipelines with a focus on data quality and efficiency.

Qualifications

• 9+ years of experience in data engineering or large-scale data processing.
• Hands-on experience designing, developing, testing, and maintaining Azure-based data pipelines.
• Strong PySpark and Python for large-scale data transformation and automation.
• Experience with Azure Data Factory, Databricks, ADLS Gen2, Synapse, and Cosmos DB.

Responsibilities

• Design, build, and support scalable Azure data engineering solutions.
• Ingest batch and near-real-time data from diverse sources and structure data pipelines.
• Develop reusable PySpark components, metadata-driven processing, and quality controls.
• Collaborate with stakeholders to convert requirements into production data workflows.
• Ensure security, performance, and operational stability of data pipelines.

Skills

PySpark

Python

SQL

Azure Data Factory

Azure Databricks

Azure Data Lake Storage Gen2

Azure Synapse Analytics

Azure Cosmos DB

Tools

Git

Azure DevOps

CI/CD

Job description

ValueMomentum is seeking an experienced Azure Data Engineer to design, build, and support scalable data pipelines on Microsoft Azure. You will work on batch and near-real-time ingestion, data lake/warehouse integration, and operational data workloads using Azure-native services.

Strong PySpark, Python, SQL, and Azure stack expertise are required. The role emphasizes producing reliable, secure, production-ready data pipelines with a focus on data quality and efficiency.