Data Engineer (Python & PySpark)

Techknomatic Services; Pune District; On-site; INR 700,000 - 1,200,000; Full time

Job summary

Techknomatic Services seeks a Data Engineer in Pune to build scalable data pipelines and distributed processing solutions using Python, PySpark, and SQL. In this role, you'll design and optimize large-scale data processing pipelines, collaborate with teams, and contribute to strategic data engineering initiatives.

Ideal candidates should have 3+ years of relevant experience and a Bachelor's degree in a related field, alongside skills in data modeling and ETL processes.

Qualifications

• 3+ years of experience in Data Engineering or Analytics platforms.
• Strong hands-on experience with Python and PySpark for distributed data processing.
• Expertise in SQL, including joins, aggregations, CTEs, and window functions.
• Experience building ETL/ELT pipelines and handling large-scale datasets.
• Good understanding of data modeling, warehousing, and lakehouse concepts.
• Exposure to Spark optimization and performance tuning techniques.

Responsibilities

• Design and implement scalable batch and incremental data pipelines using Python and PySpark.
• Develop and optimize PySpark jobs for distributed data processing.
• Write complex SQL queries for analytics, transformation, and validation.
• Contribute to data architecture and modeling decisions.
• Ensure data quality and reliability across data workflows.
• Collaborate with cross-functional teams to improve data platform efficiency.

Skills

Python

PySpark

SQL

ETL/ELT pipelines

Data modeling

Education

Bachelor's degree in Computer Science, Engineering, IT, or related field

Tools

Azure

AWS

GCP

Job description

Build scalable data pipelines and distributed processing solutions using Python, PySpark, and SQL while contributing to enterprise data architecture and platform optimization.

About the role

As a Data Engineer in Pune, you will design and optimize large-scale data processing pipelines using Python, PySpark, and SQL. You will work closely with data architects and analytics teams to build scalable, high-performance data platforms and contribute to strategic data engineering initiatives.

Responsibilities

• Design and implement scalable batch and incremental data pipelines using Python and PySpark.
• Develop and optimize PySpark jobs for distributed data processing and large-scale transformations.
• Write complex SQL queries for analytics, transformation, and validation use cases.
• Contribute to data architecture, modeling, and storage strategy decisions.
• Optimize pipeline performance using partitioning, caching, and execution tuning techniques.
• Ensure data quality, consistency, governance, and reliability across data workflows.
• Troubleshoot pipeline failures, bottlenecks, and processing issues.
• Collaborate with cross-functional teams to improve scalability and data platform efficiency.

Job Requirement

• 3+ years of experience in Data Engineering or Analytics platforms.
• Strong hands-on experience with Python and PySpark for distributed data processing.
• Expertise in SQL, including joins, aggregations, CTEs, and window functions.
• Experience building ETL/ELT pipelines and handling large-scale datasets.
• Good understanding of data modeling, warehousing, and lakehouse concepts.
• Exposure to Spark optimization, partitioning, and performance tuning techniques.
• Familiarity with cloud platforms such as Azure, AWS, or GCP is a plus.
• Bachelor's degree in Computer Science, Engineering, IT, or a related field.