PySpark Developer (2 To 8 Years)

Infosys; Dadri, Chennai District, Bengaluru; Hybrid; INR 900,000 - 1,800,000; Full time

Job summary

Infosys seeks a PySpark Developer with 2-8 years of experience to design, develop, and optimize large-scale data processing solutions. The role focuses on PySpark, Python, Spark SQL, ETL development, and Big Data technologies to build scalable data pipelines.

You will transform datasets, support analytics, and collaborate with data engineers, analysts, and business stakeholders. The position is full-time and PAN India, with opportunities to contribute to production deployments and documentation.

Qualifications

• 2-8 years of PySpark development experience.
• Strong Python and Spark SQL skills.
• Experience with ETL design and data transformation.
• Familiarity with Big Data ecosystems and distributed processing.

Responsibilities

• Design, develop, and maintain scalable data processing applications using PySpark.
• Build and optimize ETL/ELT pipelines for structured and unstructured data.
• Develop data transformation logic using Spark SQL and DataFrames.
• Work with large-scale datasets in distributed computing environments.
• Collaborate with Data Engineers, Data Analysts, and Business stakeholders to understand data requirements.
• Monitor, troubleshoot, and optimize Spark jobs for performance and reliability.
• Ensure data quality, integrity, and consistency across data pipelines.
• Participate in code reviews and follow coding best practices.
• Support deployments, enhancements, and production issue resolution.
• Create technical documentation and maintain operational procedures.

Skills

PySpark

Python

Spark SQL

ETL development

Big Data technologies

Job description

Job Title: PySpark Developer
Experience: 2-8 Years
Location: PAN India
Employment Type: Full-Time

Job Summary

We are seeking a skilled PySpark Developer with 2-8 years of experience in designing, developing, and optimizing large-scale data processing solutions. The ideal candidate should have hands-on experience with PySpark, Python, Spark SQL, ETL development, and Big Data technologies. The role involves building scalable data pipelines, transforming large datasets, and supporting analytics and reporting requirements.

Key Responsibilities

• Design, develop, and maintain scalable data processing applications using PySpark.
• Build and optimize ETL/ELT pipelines for structured and unstructured data.
• Develop data transformation logic using Spark SQL and DataFrames.
• Work with large-scale datasets in distributed computing environments.
• Collaborate with Data Engineers, Data Analysts, and Business stakeholders to understand data requirements.
• Monitor, troubleshoot, and optimize Spark jobs for performance and reliability.
• Ensure data quality, integrity, and consistency across data pipelines.
• Participate in code reviews and follow coding best practices.
• Support deployments, enhancements, and production issue resolution.
• Create technical documentation and maintain operational procedures.