Pyspark developer

Aligned Automation

Pune District

On-site

INR 2,200,000 - 3,400,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Aligned Automation Services Pvt Ltd is seeking a Senior PySpark Data Engineer in Pune to design and optimize large-scale data pipelines. The role requires 7-8 years of hands-on experience with PySpark, Python, SQL, Databricks, and modern data architectures. You will lead development, interact with clients, and mentor junior team members.

The position is based in Pune (work from office) and involves hands-on development, architecture discussions, and CI/CD collaboration with DevOps teams.

Qualifications

  • 7-8 years of hands-on data engineering experience.
  • Strong expertise with PySpark, Python, SQL, and Databricks.
  • Experience designing scalable data pipelines and lakehouse architectures using Delta Lake.

Responsibilities

  • Design, develop, and maintain scalable ETL/ELT pipelines using PySpark.
  • Build high-performance data pipelines for large volumes of data.
  • Develop reusable data ingestion, transformation, validation, and monitoring frameworks.
  • Optimize Spark jobs with partitioning, joins, caching, and memory tuning.
  • Design and implement Data Lake/Lakehouse architectures using Delta Lake.
  • Write complex SQL queries, stored procedures, and window functions.
  • Collaborate with stakeholders and mentors junior engineers; participate in Agile ceremonies.

Skills

PySpark
Python
SQL
Databricks
Delta Lake

Tools

Databricks
Delta Lake

Job description

Aligned Automation Services Pvt Ltd | Full time

At Aligned Automation, we live by our\"Better Together\"philosophy to build a better world. As a strategic service provider to Fortune 500 companies, we help digitize enterprise operations and drive impactful business strategies. Our purpose goes beyond projects—we strive to deliver meaningful, sustainable change that shapes a more optimistic and equitable future.

Our culture is deeply rooted in our4Cs-Care,Courage,Curiosity, andCollaboration-ensuring that each employee is empowered to grow, innovate, and thrive in an inclusive workplace.

Experience: 7-8 Years
Location: Pune (Work from office)
Job Summary

We are seeking a highly skilled Senior PySpark Data Engineer with 7-8 years of experience in designing, developing, and optimizing large-scale data engineering solutions. The ideal candidate should have extensive experience with PySpark, Python, SQL, Databricks, cloud platforms , and modern data architectures. The role requires hands-on development, solution design, client interaction, and mentoring junior team members.

Key Responsibilities

  • Design, develop, and maintain scalable ETL/ELT pipelines using PySpark .
  • Build high-performance data pipelines to process large volumes of structured and semi-structured data.
  • Develop reusable frameworks for data ingestion, transformation, validation, and monitoring.
  • Optimize Spark jobs by tuning partitions, joins, caching, and memory configurations.
  • Design and implement Data Lake/Lakehouse architectures using Delta Lake .
  • Write complex SQL queries, stored procedures, CTEs, and window functions.
  • Collaborate with business stakeholders, architects, and data analysts to understand business requirements.
  • Participate in architecture discussions and recommend scalable data solutions.
  • Perform code reviews and enforce coding standards and best practices.
  • Monitor production pipelines, troubleshoot issues, and implement performance improvements.
  • Work with DevOps teams to implement CI/CD for data pipelines.
  • Mentor junior engineers and provide technical leadership.
  • Estimate effort, prepare technical documentation, and participate in Agile ceremonies.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Pyspark developer
Pyspark developer

Aligned Automation Services • Pune District

On-site
INR 2,500,000 - 4,000,000
Pyspark Developer (2 To 8 Years)
Pyspark Developer (2 To 8 Years)

Infosys • Dadri, Chennai District, Bengaluru

Remote
INR 900,000 - 1,800,000
Contractor - PySpark Engineer
Contractor - PySpark Engineer

Vivantify • Hyderabad

On-site
INR 1,800,000 - 2,600,000
Data Engineer Pyspark and Mongo DB
Data Engineer Pyspark and Mongo DB

Aligned Automation, LLC • Pune District

On-site
INR 1,500,000 - 2,300,000
Databricks, PySpark, Python - Data Engineer
Databricks, PySpark, Python - Data Engineer

Infosys • Thiruvananthapuram

On-site
INR 1,400,000 - 2,100,000
Data Engineer
Data Engineer

Cnx • Bengaluru

On-site
INR 1,200,000 - 1,500,000
Role For Python Developer
Role For Python Developer

Altimetrik • Pune District, Chennai District, Bengaluru

On-site
INR 2,500,000 - 3,500,000
Data Architect
Data Architect

Agilisium • Pune District

On-site
INR 1,500,000 - 2,100,000
Contractor - PySpark Engineer
Contractor - PySpark Engineer

Vivantify Technology Solutions India Pvt. Ltd. • Hyderabad

Remote
INR 1,800,000 - 3,200,000
SPARK Data Onboarding Engineer - Chennai
SPARK Data Onboarding Engineer - Chennai

Photon • India

On-site
INR 2,500,000 - 4,000,000