Senior Azure Data Engineer: Build AI-Driven Pipelines

Tiger Analytics Inc.; Jersey City (NJ); On-site; USD 140,000 - 210,000; Full time

Benefits offered by this job

Medical

Dental

Vision

Retirement Accounts

Discretionary time-off

HSA/FSA Accounts

Job summary

Tiger Analytics is seeking a Senior Azure Data Engineer to design, develop, and maintain scalable data pipelines on Microsoft Azure. You will work with Python, Scala, and Apache Spark to support enterprise data platforms.

You will build and optimize ETL/ELT processes, leverage Azure Data Factory, Databricks, and Data Lake Storage, and collaborate with data architects to deliver reliable data solutions across the organization.

Qualifications

• Bachelor’s degree in CS/IT/Engineering or related field.
• 10+ years of data engineering experience.
• Hands-on with Azure data services (ADF, Databricks, Data Lake Storage).
• Proficient in Python and Scala for data processing.
• Strong Spark and distributed data processing experience.
• Experience with Hadoop and big data technologies.
• Solid ETL/ELT design and data pipeline architecture.
• Strong SQL with performance tuning and production support.
• Experience with Control-M for batch scheduling and monitoring.
• Familiar with version control, CI/CD, and deployment practices.
• Excellent analytical and collaborative skills.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure data services, Python, Scala, and Spark.
• Develop and optimize ETL/ELT processes across diverse data sources.
• Build pipelines with Azure Data Factory, Databricks, and Data Lake Storage.
• Develop and optimize Spark applications for performance and scalability.
• Work with Hadoop and related big data technologies where applicable.
• Monitor pipeline execution, troubleshoot failures, and ensure reliable data delivery.
• Implement data validation and quality checks to ensure accuracy.
• Collaborate with data architects, analysts, and cross-functional teams.
• Follow coding standards, testing practices, version control, and CI/CD.
• Support production deployments, incident resolution, and maintenance.

Skills

Azure Data Factory

Azure Databricks

Data Lake Storage

Python

Scala

Apache Spark

Hadoop

ETL/ELT

SQL

CI/CD

Control-M

Git

Education

Bachelor’s degree in Computer Science, IT, or related field

Tools

Azure Data Factory

Azure Databricks

Azure Data Lake Storage

Git

Control-M

Job description

Tiger Analytics is seeking a Senior Azure Data Engineer to design, develop, and maintain scalable data pipelines on Microsoft Azure. You will work with Python, Scala, and Apache Spark to support enterprise data platforms.

You will build and optimize ETL/ELT processes, leverage Azure Data Factory, Databricks, and Data Lake Storage, and collaborate with data architects to deliver reliable data solutions across the organization.