Data Engineer

techcarrot; Hyderabad; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

techcarrot is seeking a data engineer to design and implement scalable data pipelines using Azure Databricks, Data Factory, and Synapse. You will analyze structured and unstructured data, build prototypes, and collaborate with cross-functional teams to integrate solutions into core analytics products.

The role requires 4–6 years of relevant experience and a strong background in Python and SQL within the Azure cloud stack.

Qualifications

• Advanced working knowledge of relational and non-relational databases.
• Experience building and optimizing Azure data pipelines and data sets.
• Strong analytics working with unstructured data.
• Proficiency with Azure Data Factory, Databricks, Event Hub, Python, PySpark, Azure Synapse, SQL, and Azure DevOps.
• Experience deploying ADF pipelines with Azure DevOps.
• 4–6 years of experience.
• Knowledge of Azure cloud stack and Databricks experience >4 years.

Responsibilities

• Review and analyze structured, semi-structured, and unstructured data for quality and value.
• Design, prototype, implement, and test data solutions and present results to diverse audiences.
• Develop robust, reusable data pipelines using Microsoft and Databricks stack.

Skills

Analytical skills

Team collaboration

Unstructured data handling

Education

Bachelor’s degree

Tools

Azure Data Factory

Databricks

Event Hub

Python

PySpark

Azure Synapse

SQL

Azure DevOps

Azure cloud stack

Azure Data Platform

Job description

• Detailed reviewing and analyzing structured, semi-structured and unstructured data sources for quality, completeness and business value.
• Design, architect, implement and test rapid prototypes that demonstrate value of the data and present them to diverse audiences.
• Participate in early state design and feature definition activities.
• Responsible for implementing robust data pipeline using Microsoft, Databricks Stack
• Responsible for creating reusable and scalable data pipelines
• You are a Team-Player, collaborating with team members across multiple engineering teams to support the integration of proven prototypes into core intelligence products.

• Detailed reviewing and analyzing structured, semi-structured and unstructured data sources for quality, completeness and business value.
• Design, architect, implement and test rapid prototypes that demonstrate value of the data and present them to diverse audiences.
• Participate in early state design and feature definition activities.
• Responsible for implementing robust data pipeline using Microsoft, Databricks Stack
• Responsible for creating reusable and scalable data pipelines
• You are a Team-Player, collaborating with team members across multiple engineering teams to support the integration of proven prototypes into core intelligence products.

Requirements

• Advanced working knowledge and experience with relational and non-relational databases
• Experience building and optimizing Azure Data data pipelines, architectures and data sets
• Strong analytic skills related to working with unstructured datasets
• Strong experience in Azure Data factory, Databricks, Eventhub, Python, PySpark , Azure Synapse and SQL, Azure DevOps
• Azure DevOps experience to deploy the ADF pipelines.
• Create end to end Azure databricks
• Minimum 4-6 years of experience
• Bachelor’s degree
• Knowledge/Experience with Azure cloud stack.
• Azure Data Platform & Databricks exp should be more than 4 years