Data Engineer

Tiger Analytics; Hyderabad, Chennai, Bengaluru; Ibrido; INR 1.800.000 - 2.600.000; Tempo pieno

Descrizione del lavoro

Tiger Analytics is seeking a data platform engineer to design and implement scalable, metadata-driven data pipelines on Azure Databricks. You will work across batch and streaming data, build end-to-end solutions, and mentor peers in best practices.

Responsibilities include owning complex modules, ensuring robust observability, and collaborating with data architects, scientists, and application teams to deliver analytics products. Proficiency in Python, SQL, and Azure-native services is required.

Competenze

• 3+ years of overall technical experience with at least 2 years hands-on in Microsoft Azure and Databricks.
• Delivery of at least one end-to-end Data Lakehouse solution on Azure Databricks using Medallion Architecture.
• Strong knowledge of the Databricks ecosystem including PySpark, Notebooks, Structured Streaming, Unity Catalog, Delta Live Tables, Workflows and SQL Warehouse.
• Advanced programming, unit testing, and debugging skills in Python and SQL.
• Hands-on experience with Azure-native services such as Azure Data Factory, ADLS Gen2, Azure SQL Database, and Event Hub.
• Solid understanding of data modeling techniques, including both Dimensional and 3NF models.
• Exposure to developing LLM/Generative AI-powered applications.
• Excellent understanding of CI/CD workflows using Azure DevOps.
• Bonus: knowledge of Azure infrastructure including provisioning, networking, security, and governance.

Mansioni

• Design and implement scalable, metadata-driven data ingestion, quality, and transformation frameworks for batch and streaming datasets.
• Develop and optimize end-to-end data pipelines to create analytical data products.
• Build robust exception handling, logging, and monitoring for observability and operations.
• Take ownership of complex modules and lead development of critical data workflows and components.
• Provide guidance to data engineers and peers on best practices.
• Collaborate with cross-functional teams including business consultants, data architects, scientists, and application developers to deliver impactful analytics solutions.

Conoscenze

Python

SQL

Unit testing

Debugging

CI/CD

Strumenti

Azure Databricks

Azure Data Factory

Azure SQL Database

Delta Live Tables

Unity Catalog

PySpark

Notebooks

Structured Streaming

Workflows

SQL Warehouse

Event Hub

Descrizione del lavoro

Key Responsibilities:

Design and implement scalable, metadata-driven frameworks for data ingestion, quality, and transformation across both batch

and streaming datasets.

Develop and optimize end-to-end data pipelines to process structured and unstructured data, enabling the creation of analytical

data products.

• Build robust exception handling, logging, and monitoring mechanisms for better observability and operational support.
• Take ownership of complex modules and lead the development of critical data workflows and components.
• Provide guidance to data engineers and peers on best practices.

Collaborate with cross-functional teamsincluding business consultants, data architects, scientists, and application

developers—to deliver impactful analytics solutions.

EXPERTISE AND QUALIFICATIONS

3+ years of overall technical experience, with a minimum of 2 years of hands‑on experience with Microsoft Azure and

Databricks.

Proven experience delivering at least one end-to-end Data Lakehouse solution on Azure Databricks using the Medallion

Architecture.

Strong working knowledge of the Databricks ecosystem, including: PySpark, Notebooks, Structured Streaming, Unity Catalog,

Delta Live Tables, Workflows, and SQL Warehouse.

• Advanced programming, unit testing, and debugging skills in Python and SQL.

Hands‑on experience with Azure‑native services such as: Azure Data Factory, ADLS Gen2, Azure SQL Database, and Event

Hub.

• Solid understanding of data modeling techniques, including both Dimensional and Third Normal Form (3NF) models.
• Exposure to developing LLM/Generative AI-powered applications.
• Must have excellent understanding of CI/CD workflows using Azure DevOps.
• Bonus: Knowledge of Azure infrastructure, including provisioning, networking, security, and governance.