TCJ2420

techcarrot FZ; Hyderabad, Dadri, Chennai District; Sur place; INR 1 400 000 - 2 100 000; Plein temps

Résumé du poste

tech carrot is actively hiring a data engineer to design and implement scalable data pipelines on the Azure platform. You will own end-to-end data processing tasks, building robust ingestion and transformation frameworks using ADF and Databricks.

The role requires strong Python/PySpark skills and experience with SQL, Delta Lake and Unity Catalog. Hyderabad/Noida/Chennai offices support hybrid collaboration.

Qualifications

• 4–6 years of experience in Data Engineering / Data Platform development.
• Preferably 4+ years with Microsoft Azure Data Platform technologies.
• Hands-on experience with Azure Data Factory, Databricks, SQL and Python/PySpark.
• Bachelor's degree in CS/IT/Engineering or related field.
• Experience delivering end-to-end Azure data engineering projects.
• Strong knowledge of data warehousing, data lake and modern data platforms.

Responsabilités

• Understand business and technical requirements and translate them into scalable data engineering solutions.
• Own assigned projects and data pipelines with minimal supervision.
• Design and deploy data pipelines on the Azure Data Platform.
• Build scalable ingestion and transformation frameworks using ADF and Databricks.
• Develop data transformation logic using Python, PySpark and SQL.
• Handle data from diverse sources and support batch and near-real-time processing.
• Optimize Spark workloads and Databricks jobs for performance.
• Create reusable components to improve development efficiency.
• Collaborate with architects, analysts, BI developers and source-system teams.
• Support deployment and troubleshoot post-production data issues.

Connaissances

Azure Data Factory

Databricks

Python

PySpark

SQL

Delta Lake

Unity Catalog

REST API

Power BI

Azure DevOps

CI/CD

Formation

Bachelor's degree in Computer Science or related field

Outils

Azure Data Factory

Azure Databricks

Apache Spark

Delta Lake

REST APIs

Unity Catalog

Power BI

Description du poste

Hyderabad, Noida/ Chennai, India | Posted on 08/31/2026

tech carrot is a leading global ITservice provider and partner in digital transformation offering cost-effective,customer-centric, and cutting-edge digital solutions to clients worldwide.

founded in 2016, tech carrot isheadquartered in Dubai, UAE. Being a people-oriented organization with aphilosophy of "We care" as our core DNA, tech carrot continuallyfocuses on new initiatives and strategies to increase overall employeeexperience by providing a plethora of platforms to drive employee development,overall efficiency, increase overall employee satisfaction. With over 200+technology specialists who are engaged and motivated we are growing at 20% a year-on-year since inception.

We offer cutting-edge services across multipleindustries and deliver seamless customer experiences, ensuring high value andrapid growth for our clients. tech carrot has earned wide recognition as areliable partner for organizations across the globe because of its professionalapproach and strong delivery capabilities. The Middle East and Africa arehandled by UAE operations, while APAC is handled by India operations. We havealso expanded in the USA and the Netherlands as well.

Office Location

Hyderabad, Noida/ Chennai

Job Description

• Independently understand business and technical requirements and translate them into scalable data engineering solutions.
  
• Take end-to-end ownership of assigned projects, modules and data pipelines with minimal supervision.
  
• Design, develop, test and deploy robust end-to-end data pipelines on the Azure Data Platform.
  
• Build scalable and reusable data ingestion and transformation frameworks using Azure Data Factory and Azure Databricks.
  
• Develop data transformation and processing logic using Python, PySpark and SQL.
  
• Work with structured, semi-structured and unstructured data from databases, APIs, files and other enterprise data sources.
  
• Design and implement batch and, where required, near-real-time data processing solutions.
  
• Develop optimized Databricks workloads using Apache Spark and Delta Lake.
  
• Perform data analysis, profiling and validation to identify data quality, completeness and consistency issues.
  
• Implement appropriate data quality checks, error handling, logging and monitoring within data pipelines.
  
• Troubleshoot data pipeline failures, performance issues and data discrepancies independently.
  
• Optimize SQL queries, Spark jobs and data pipelines for performance and scalability.
  
• Create reusable components and frameworks to improve development efficiency across projects.
  
• Participate in solution design and technical discussions and provide recommendations on implementation approaches.
  
• Deploy and manage ADF, Databricks and related Azure data components across development, test and production environments.
  
• Prepare appropriate technical design, data mapping and operational documentation.
  
• Collaborate with solution architects, analysts, BI developers, source-system teams and other engineering teams.
  
• Support production deployments and troubleshoot post-production issues when required.
  

Requirements

Strong hands-on experience with:

• Azure Data Factory (ADF) – pipelines, datasets, linked services, triggers, parameterization and reusable frameworks
  
• Azure Monitor / Log Analytics or equivalent monitoring capabilities
  
• Azure Event Hubs – exposure/experience is preferred
  

Databricks & Data Processing

• Strong hands-on experience with Databricks and Apache Spark
  
• Experience with Delta Lake / Delta tables
  
• Understanding of Medallion or similar layered data architectures
  
• Experience implementing incremental and full-load processing patterns
  
• Ability to troubleshoot and optimize Spark workloads
  
• Experience with Databricks Workflows/Jobs
  
• Knowledge of Unity Catalog is preferred
  

SQL & Python

• Ability to write and optimize complex SQL queries
  
• Experience with joins, CTEs, window functions and analytical queries
  
• Strong working knowledge of Python
  
• Ability to develop reusable Python/PySpark modules and utilities
  

DevOps / CI-CD

• Hands-on experience with Azure DevOps
  
• Git-based source control
  
• Branching and code-management practices
  
• CI/CD implementation for Azure Data Factory and Databricks
  

Qualifications & Experience

• 4–6 years of overall experience in Data Engineering / Data Platform development.
  
• Preferably 4+ years of hands-on experience with Microsoft Azure Data Platform technologies.
  
• Strong hands-on experience with Azure Data Factory, Databricks, SQL and Python/PySpark.
  
• Bachelor's degree in Computer Science, Information Technology, Engineering or a related discipline.
  
• Experience delivering at least one or more end-to-end Azure data engineering projects.
  
• Good understanding of data warehousing, data lake and modern data platform concepts.
  
• Strong analytical and problem-solving skills.
  
• Ability to work independently and manage multiple technical activities.
  
• Good written and verbal communication skills.
  

Preferred Skills

• Experience with Unity Catalog and Databricks governance.
  
• Experience with REST API-based data ingestion.
  
• Knowledge of dimensional modelling and data warehouse concepts.
  
• Experience working with Power BI or downstream analytics platforms.
  
• Exposure to streaming/event-driven data processing.
  
• Experience working in enterprise-scale Azure environments.