Python Developer - F2F discussion - Chennai

Tata Consultancy Services; Chennai District; On-site; INR 900,000 - 1,500,000; Full time

Job summary

Tata Consultancy Services (TCS) is seeking a PySpark developer to design and maintain scalable data processing applications. The role involves building ETL/ELT pipelines and handling both structured and unstructured data with Apache Spark.

You will collaborate with data engineers, data scientists, and business stakeholders to gather requirements, optimize Spark jobs, and ensure data quality across distributed environments. Familiarity with AWS, Azure, or GCP is expected.

Qualifications

• Develop and maintain scalable data processing applications using PySpark.
• Design, build, and optimize ETL/ELT pipelines for large datasets.
• Process structured and unstructured data using Apache Spark.
• Collaborate with data engineers, data scientists, and business stakeholders to understand requirements.
• Write efficient Spark jobs and optimize performance for distributed environments.
• Integrate data from multiple sources such as databases, APIs, and cloud storage.
• Perform data quality checks, validation, and troubleshooting.
• Monitor and maintain production data pipelines.
• Implement best practices for coding, testing, deployment, and documentation.
• Work with cloud platforms such as AWS, Azure, or GCP.

Responsibilities

• Work as part of a team to design and implement Spark-based data pipelines.
• Monitor production jobs, troubleshoot issues, and optimize performance.
• Collaborate with stakeholders to understand data requirements and ensure quality.

Skills

PySpark

Apache Spark

ETL

Cloud platforms

Job description

Greetings from Tata Consultancy Services (TCS)

What we are looking for : Pyspark developer

Interview Mode : F2F discussion

Experience : 4-15 Years

• Develop and maintain scalable data processing applications using PySpark.
• Design, build, and optimize ETL/ELT pipelines for large datasets.
• Process structured and unstructured data using Apache Spark.
• Collaborate with data engineers, data scientists, and business stakeholders to understand requirements.
• Write efficient Spark jobs and optimize performance for distributed environments.
• Integrate data from multiple sources such as databases, APIs, and cloud storage.
• Perform data quality checks, validation, and troubleshooting.
• Monitor and maintain production data pipelines.
• Implement best practices for coding, testing, deployment, and documentation.
• Work with cloud platforms such as AWS, Azure, or GCP.