Data Engineer

Krato Solutions; Chennai; On-site; INR 900,000 - 1,500,000; Full time

Job summary

Krato Solutions in Chennai is seeking a Data Engineer with 4–5 years of experience to design and maintain scalable data pipelines and data integration solutions. The ideal candidate brings strong expertise in PySpark, Databricks, Azure Data Factory, Python, and SQL, with hands-on experience on Azure cloud platforms.

You will design ETL/ELT pipelines, manage data workflows, ensure data quality and governance, and collaborate with data scientists and analysts.

Qualifications

• 4–5 years of data engineering experience with PySpark and Databricks.
• Strong expertise in Python and SQL with performance tuning.
• Hands-on experience with Azure Data Factory and Azure cloud services.
• Experience building scalable ETL/ELT pipelines and data governance.

Responsibilities

• Design and develop scalable ETL/ELT pipelines using PySpark and Databricks.
• Build and manage workflows with Azure Data Factory.
• Integrate data from APIs, databases, flat files, and cloud storage.
• Optimize and maintain large-scale data processing systems.
• Ensure data quality, validation, and governance across pipelines.
• Collaborate with data scientists and analysts.
• Monitor, troubleshoot, and improve pipeline performance.
• Implement reusable data engineering frameworks and best practices.

Skills

PySpark

Databricks

Azure Data Factory

Python

SQL

Azure Cloud

Data warehousing

Data modeling

Tools

Azure Synapse

Delta Lake

Azure DevOps

Job description

Data Engineer (4–5 Years Experience)

Location: Chennai

Experience: 4–5 Years

Employment Type: Full-time

Job Summary

We are seeking a Data Engineer with 4–5 years of experience to build and maintain scalable data pipelines and data integration solutions. The ideal candidate should have strong expertise in PySpark, Databricks, Azure Data Factory (ADF), Python, and SQL, with hands‑on experience in cloud-based data platforms.

Key Responsibilities

• Design and develop scalable ETL/ELT pipelines using PySpark and Databricks
• Build and manage data workflows using Azure Data Factory (ADF)
• Integrate data from multiple sources such as APIs, databases, flat files, and cloud storage
• Optimize and maintain large-scale data processing systems
• Ensure data quality, validation, and governance across pipelines
• Collaborate with cross-functional teams including data scientists and analysts
• Monitor, troubleshoot, and improve pipeline performance
• Implement reusable frameworks and best practices for data engineering
• Support deployment using CI/CD pipelines

Required Skills

• Strong experience with Azure Data Factory (ADF) for pipeline orchestration
• Hands-on experience with Databricks and PySpark
• Proficiency in Python
• Advanced SQL skills including complex queries and performance tuning
• Experience working on Azure Cloud (ADLS, Azure SQL, etc.)
• Good understanding of data warehousing and data modeling concepts

Preferred Qualifications

• Experience with Azure ecosystem including ADF, ADLS, Synapse, and Key Vault
• Knowledge of workflow orchestration and scheduling tools
• Familiarity with Delta Lake architecture
• Experience handling large datasets and performance tuning

Nice to Have

• Experience with streaming tools such as Kafka or Event Hubs
• Knowledge of DevOps practices and CI/CD using Azure DevOps
• Exposure to data governance and security practices

Soft Skills

• Strong analytical and problem-solving skills
• Good communication and teamwork abilities
• Ability to work in a fast-paced environment