Data Engineer

DataPhi; Pune District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

DataPhi is seeking a skilled Data Engineer to join our Data team in Pune, India. You will design, construct, and maintain scalable data pipelines and robust data architectures (data lakes/warehouses) that ingest and transform large data sets from multiple sources, enabling BI dashboards, analytics, and ML initiatives across the organization.

You will work with Azure Data Factory, Python, PySpark/Scala, and cloud architectures to deliver high-quality, accessible data.

Qualifications

• 2+ years of end-to-end Data Engineering experience.
• Experience with data pipelines, ETL/ELT processes and data warehouses/lakes.
• Hands-on with Azure Data Factory or SSIS or AWS Glue.

Responsibilities

• Develop and maintain robust, scalable data pipelines for batch and real-time data.
• Design, build, and optimize data warehouse and data lake solutions.
• Implement ETL/ELT processes to transform raw data into usable formats.
• Ensure data quality, governance, and security across pipelines.
• Collaborate with data scientists, analysts, and stakeholders to deliver data solutions.
• Automate data processes and build data models.

Skills

End-to-end Data Eng

Azure Data Factory / AWS Glue

Python

PySpark / Scala

Azure Functions

Data Modeling

Data Quality

Tools

Azure Data Factory

SSIS

AWS Glue

Azure Functions

Python

PySpark

Scala

Job description

“We are seeking a skilled Data Engineer to join our growing Data team and be instrumental in building the backbone of our data ecosystem. You will be responsible for designing, constructing, and maintaining scalable data pipelines and robust data architectures (like data lakes/warehouses) that efficiently collect, process, and transform vast amounts of raw data from disparate sources. This role is crucial for delivering high-quality, accessible data that fuels our business intelligence dashboards, advanced analytics, and machine learning initiatives, enabling data-driven decisions across the organization. If you’re passionate about data infrastructure, solving complex data challenges, and enabling data science, we want to hear from you

Key Responsibilities

• Develop and maintain robust, scalable data pipelines for batch and real-time data.
• Design, build, and optimize data warehouse s and data lakes .
• Implement ETL/ELT processes to transform raw data into usable formats.
• Ensure data quality, integrity, security, and governance
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions.
• Automate data processes and build data models.

Skills & Experience: Key Requirements

Required

• Minimum 2 years of end-to-end Data Engineering Development experience – Total professional experience does not matter much.
• 2 to 3 years of experience in working on data engineering/ETL tools like Azure Data Factory (ADF) or SSIS or AWS Glue.
• Should have worked with Azure Function and Python to perform serverless code execution.
• Experience in handling large data sets using ADF and Pyspark / Scala code.

Nice to Have

• Experience working with non-structured data sets in Azure.
• Experience with other analytics tools like Power BI, Azure Analysis service.
• Experience with private and public cloud architectures.
• Min 3 Full Cycle Data Engineering Implementations

Qualifications

• Excellent written and verbal communication skills.