PySpark Developer / Senior Data Engineer

Alignity Solutions; Hyderabad; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Alignity Solutions is seeking a PySpark Developer / Senior Data Engineer to design, build, and optimize large-scale data pipelines using PySpark and Apache Spark. The role requires hands-on experience with distributed computing and transforming big datasets in production environments.

The candidate will collaborate with data architects, analysts, and platform teams to ensure robust ETL/ELT workflows, data validation, and efficient Spark job tuning.

Qualifications

• We are looking for an experienced PySpark Developer with strong hands-on expertise in big data processing, distributed computing, and data engineering.
• The ideal candidate will have deep experience building scalable data pipelines, transforming large datasets, and working with Spark-based ecosystems in production environments.

Responsibilities

• Design, build, and maintain scalable data pipelines using PySpark and Apache Spark
• Develop ETL/ELT workflows for batch and near-real-time processing
• Optimize Spark jobs for performance, reliability, and cost efficiency
• Work with large structured and unstructured datasets
• Integrate data from multiple sources such as databases, APIs, files, and cloud storage
• Write reusable, modular PySpark code
• Troubleshoot job failures, data quality issues, and performance bottlenecks
• Collaborate with data architects, analysts, platform teams, and business stakeholders
• Implement data validation, monitoring, and logging frameworks
• Support deployment, scheduling, and orchestration of data pipelines
• Participate in design reviews, code reviews, and technical discussions
• Mentor junior engineers and contribute to team best practices

Skills

PySpark

Apache Spark

SQL

Data pipelines

ETL/ELT pipelines

Performance tuning

Linux/Unix

Git

Data engineering

Tools

PySpark tooling

Job description

PySpark Developer / Senior Data Engineer

Seri Lingampally, India | Posted on 08/18/2026

Do you love a career where you Experience, Grow & Contribute at the same time, while earning at least 10% above the market? If so, we are excited to have bumped onto you.

Learn how we are redefining the meaning of work , and be a part of the team raved by Clients, Job-seekers and Employees.

If you are a PySpark Developer / Senior Data Engineer looking for excitement, challenge and stability in your work, then you would be glad to come across this page.

We are an IT Solutions Integrator/Consulting Firm helping our clients hire the right professional for an exciting long-term project. Here are a few details.

Check if you are up for maximizing your earning/growth potential, leveraging our Disruptive Talent Solution.

Role: PySpark Developer / Senior Data Engineer
Location:HYDERABAD | BANGALORE | PUNE | CHENNAI
Experience: 6+ Years
Employment Type: Contract to hire
Notice Period:0-30 days(If you have negotiable notice period or buyout option please apply)

Requirements

We are looking for an experienced PySpark Developer with strong hands-on expertise in big data processing, distributed computing, and data engineering. The ideal candidate will have deep experience building scalable data pipelines, transforming large datasets, and working with Spark-based ecosystems in production environments.

Key Responsibilities

• Design, build, and maintain scalable data pipelines using PySpark and Apache Spark
  
• Develop efficient ETL/ELT workflows for batch and near-real-time processing
  
• Optimize Spark jobs for performance, reliability, and cost efficiency
  
• Work with large structured and unstructured datasets
  
• Integrate data from multiple sources such as databases, APIs, files, and cloud storage
  
• Write reusable, modular, and maintainable PySpark code
  
• Troubleshoot job failures, data quality issues, and performance bottlenecks
  
• Collaborate with data architects, analysts, platform teams, and business stakeholders
  
• Implement data validation, monitoring, and logging frameworks
  
• Support deployment, scheduling, and orchestration of data pipelines
  
• Participate in design reviews, code reviews, and technical discussions
  
• Mentor junior engineers and contribute to team best practices
  

Required Skills

• Strong hands-on experience with PySpark and Apache Spark
  
• Deep understanding of Spark concepts such as RDDs, DataFrames, datasets, partitioning, caching, shuffling, joins, and window functions
  
• Experience with SQL and relational databases
  
• Knowledge of big data concepts and distributed data processing
  
• Hands-on experience with ETL/ELT pipeline development
  
• Good understanding of performance tuning and optimization techniques in Spark
  
• Experience with version control tools like Git
  
• Familiarity with Linux/Unix environments