Pyspark developer

Zohorecruit; Pune District; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

Aligned Automation Services Pvt Ltd in Pune seeks a Senior PySpark Data Engineer with 7–8 years of experience to design, develop, and optimize large-scale data solutions. You will work with PySpark, Python, SQL, Databricks, cloud platforms, and modern data architectures; hands-on development, client interaction, and mentoring are required.

You will design ETL/ELT pipelines, build data lakes, optimize Spark jobs, and collaborate with stakeholders to deliver scalable data solutions and CI/CD for

Qualifications

• 7–8 years of data engineering experience.
• Hands-on development of large-scale data pipelines.
• Mentoring junior engineers and collaborating with stakeholders.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using PySpark.
• Build high-performance data pipelines for large data volumes.
• Develop reusable data ingestion, transformation, validation, and monitoring frameworks.
• Optimize Spark jobs via partitions, joins, caching, and memory tuning.
• Architect Data Lake/Lakehouse using Delta Lake.
• Write complex SQL queries and procedures.
• Collaborate with stakeholders and participate in architecture reviews.
• Mentor engineers and participate in Agile ceremonies.
• Implement CI/CD for data pipelines and monitor production jobs.

Skills

PySpark

Python

SQL

Databricks

Cloud platforms

ETL/ELT pipelines

Data Lake / Lakehouse

CI/CD

Mentoring

Tools

Databricks

Delta Lake

Spark

SQL tooling

Job description

Aligned Automation Services Pvt Ltd | Full time

At Aligned Automation, we live by our"Better Together"philosophy to build a better world. As a strategic service provider to Fortune 500 companies, we help digitize enterprise operations and drive impactful business strategies. Our purpose goes beyond projects—we strive to deliver meaningful, sustainable change that shapes a more optimistic and equitable future.

Our culture is deeply rooted in our4Cs—Care,Courage,Curiosity, andCollaboration—ensuring that each employee is empowered to grow, innovate, and thrive in an inclusive workplace.

Experience: 7–8 Years

Location: Pune (Work from office)

Job Summary

We are seeking a highly skilled Senior PySpark Data Engineer with 7–8 years of experience in designing, developing, and optimizing large-scale data engineering solutions. The ideal candidate should have extensive experience with PySpark, Python, SQL, Databricks, cloud platforms , and modern data architectures. The role requires hands-on development, solution design, client interaction, and mentoring junior team members.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using PySpark .
• Build high-performance data pipelines to process large volumes of structured and semi-structured data.
• Develop reusable frameworks for data ingestion, transformation, validation, and monitoring.
• Optimize Spark jobs by tuning partitions, joins, caching, and memory configurations.
• Design and implement Data Lake/Lakehouse architectures using Delta Lake .
• Write complex SQL queries, stored procedures, CTEs, and window functions.
• Collaborate with business stakeholders, architects, and data analysts to understand business requirements.
• Participate in architecture discussions and recommend scalable data solutions.
• Perform code reviews and enforce coding standards and best practices.
• Monitor production pipelines, troubleshoot issues, and implement performance improvements.
• Work with DevOps teams to implement CI/CD for data pipelines.
• Mentor junior engineers and provide technical leadership.
• Estimate effort, prepare technical documentation, and participate in Agile ceremonies.