Pyspark Developer

Leading Global Technology Services Company; Chennai District, Bengaluru; Hybrid; INR 1,200,000 - 1,800,000; Full time

Job summary

Leading Global Technology Services Company is seeking a Python / PySpark Developer with 4-8 years of experience to design, develop, and support scalable data processing solutions. The role emphasizes Python, PySpark, and data engineering concepts within an Agile environment, delivering data migration, transformation, and analytics initiatives.

The candidate will design and maintain data processing applications, build ETL/ELT pipelines, optimize Spark jobs for performance, and collaborate with

Qualifications

• 4-8 years of hands-on experience in software development and data engineering.
• Strong programming expertise in Python.
• Strong experience in PySpark and Apache Spark framework.
• Experience in developing and supporting ETL/data migration solutions.
• Strong understanding of SQL and relational databases.
• Experience with large datasets and distributed computing environments.
• Knowledge of data modeling, data quality, and performance tuning techniques.
• Familiarity with Linux/Unix environments and shell scripting.
• Experience with version control tools such as Git.
• Strong analytical, debugging, and problem-solving skills.

Responsibilities

• Design, develop, and maintain data processing applications using Python and PySpark.
• Build scalable ETL/ELT pipelines for processing large volumes of structured and unstructured data.
• Develop and optimize Spark jobs for performance and reliability.
• Analyze business requirements and translate them into technical solutions.
• Work closely with Business Analysts, Solution Architects, and cross-functional teams.
• Perform code reviews and ensure adherence to coding standards and best practices.
• Troubleshoot production issues and provide timely resolutions.
• Participate in Agile ceremonies, sprint planning, and estimation activities.
• Create technical documentation, unit test cases, and deployment guides.
• Support data migration and modernization initiatives across enterprise platforms.

Skills

Python

PySpark

SQL

Data engineering

Linux

Agile

Tools

Apache Spark

Git

Azure

Databricks

Job description

Job Description: Python / PySpark Developer (4 - 8 Years Experience)

Job Title: Python / PySpark Developer
Experience: 4 - 8 Years
Location: Chennai / Bangalore
Employment Type: Full-Time

Job Summary

We are seeking a highly skilled Python / PySpark Developer with 4-8 years of experience in designing, developing, and supporting scalable data processing solutions. The ideal candidate should possess strong expertise in Python, PySpark, and data engineering concepts, with the ability to work in an Agile environment and deliver high-quality solutions for large-scale data migration, transformation, and analytics initiatives.

Key Responsibilities

• Design, develop, and maintain data processing applications using Python and PySpark.
• Build scalable ETL/ELT pipelines for processing large volumes of structured and unstructured data.
• Develop and optimize Spark jobs for performance and reliability.
• Analyze business requirements and translate them into technical solutions.
• Work closely with Business Analysts, Solution Architects, and cross-functional teams.
• Perform code reviews and ensure adherence to coding standards and best practices.
• Troubleshoot production issues and provide timely resolutions.
• Participate in Agile ceremonies, sprint planning, and estimation activities.
• Create technical documentation, unit test cases, and deployment guides.
• Support data migration and modernization initiatives across enterprise platforms.

Required Skills

• 4-8 years of hands-on experience in software development and data engineering.
• Strong programming expertise in Python.
• Strong experience in PySpark and Apache Spark framework.
• Experience in developing and supporting ETL/data migration solutions.
• Strong understanding of SQL and relational databases.
• Experience working with large datasets and distributed computing environments.
• Knowledge of data modeling, data quality, and performance tuning techniques.
• Familiarity with Linux/Unix environments and shell scripting.
• Experience with version control tools such as Git.
• Strong analytical, debugging, and problem-solving skills.

Preferred Skills

• Experience with Hadoop ecosystem technologies.
• Exposure to cloud platforms such as Azure, AWS, or GCP.
• Experience with Azure Data Factory, Databricks, or similar data platforms.
• Knowledge of CI/CD pipelines and DevOps practices.
• BFSI domain experience will be an added advantage.