Senior Azure Data Engineer

Tiger Analytics; Hyderabad, Chennai District, Bengaluru; Hybride; INR 2 500 000 - 4 000 000; Plein temps

Résumé du poste

Tiger Analytics is seeking a data engineering expert to design, build, and optimize scalable data pipelines on Azure. You will implement ETL/ELT processes, leverage Delta Lake and Synapse, and work with Medallion Architecture to deliver reliable data platforms for reporting and analytics.

You will transform data with PySpark and Python, ensure data quality and performance, and collaborate with engineers, analysts, and stakeholders to meet business requirements.

Qualifications

• 5+ years of experience in Data Engineering on Azure-based platforms.
• Hands-on with ADF, Azure Databricks, PySpark, Python and SQL.
• Experience with Delta Lake, Azure Synapse Analytics, and data warehousing concepts.
• Understanding of ETL/ELT, data modeling, data pipelines and data architecture.
• Experience with Bronze, Silver and Gold data layers/Medallion Architecture.
• Knowledge of Spark/SQL performance tuning.
• Fabric and Unity Catalog experience is a plus.
• Strong problem-solving and analytical abilities.
• Good communication and cross-functional collaboration.

Responsabilités

• Design, develop, and maintain scalable data pipelines using Azure tools.
• Build and optimize ETL/ELT pipelines for large data volumes.
• Develop data solutions with Delta Lake and Synapse Analytics.
• Implement data transformations using PySpark and Python.
• Work with Medallion Architecture to build data platforms.
• Develop data models to support reporting and analytics.
• Ensure data quality, reliability, and performance.
• Troubleshoot and optimize existing pipelines and jobs.
• Collaborate with engineers, analysts, and stakeholders.

Connaissances

Azure Data Factory

Azure Databricks

PySpark

Python

SQL

Delta Lake

Azure Synapse

Medallion Architecture

Data Modeling

Data Warehousing

Spark Tuning

Fabric Catalog

Outils

Microsoft Fabric

Unity Catalog

Description du poste

Role & Responsibilities

• Design, develop, and maintain scalable and reliable data pipelines using Azure Data Factory, Databricks, PySpark, and SQL.
• Build and optimize ETL/ELT pipelines for processing large volumes of structured and unstructured data.
• Develop data solutions using Azure Databricks, Delta Lake, and Azure Synapse Analytics.
• Implement data transformation and processing using PySpark and Python.
• Work with Medallion Architecture (Bronze, Silver, Gold) to build efficient and maintainable data platforms.
• Develop and maintain data models to support reporting, analytics, and downstream business requirements.
• Ensure data quality, accuracy, reliability, and performance across data pipelines.
• Troubleshoot and optimize existing pipelines and data processing jobs.
• Collaborate with data engineers, architects, analysts, and business stakeholders to understand requirements and deliver effective data solutions.
• Follow best practices for security, governance, version control, and CI/CD in data engineering projects.
• Mentor junior data engineers and contribute to technical discussions and solution design.

Preferred Candidate Profile

• 5+ years of experience in Data Engineering, with strong experience in Azure-based data platforms.
• Strong hands-on experience with Azure Data Factory, Azure Databricks, PySpark, Python, and SQL.
• Good experience in Delta Lake, Azure Synapse Analytics, and data warehousing concepts.
• Strong understanding of ETL/ELT, data modeling, data pipelines, and data architecture.
• Experience working with Bronze, Silver, and Gold data layers / Medallion Architecture.
• Good understanding of performance tuning and optimization of Spark and SQL workloads.
• Experience with Microsoft Fabric and Unity Catalog would be an added advantage.
• Strong problem-solving and analytical skills.
• Good communication skills with the ability to work effectively with cross-functional teams.
• Prior experience mentoring team members or taking ownership of data engineering initiatives is preferred.