Python Pyspark Developer

Quess IT Solutions; Chennai District, Bengaluru; On-site; INR 900,000 - 1,800,000; Full time

Job summary

Quess IT Solutions in Chennai district, India is seeking a data engineer with strong Python development and PySpark expertise to build scalable data pipelines and ETL processes. You will work with large distributed datasets, apply data warehousing concepts and contribute to data lake ecosystems.

Proficiency in SQL, Git, and cloud platforms such as Azure, AWS or GCP is expected. Experience with distributed storage like MINIO, ADLS using S3, and Iceberg tables is a plus for optimizing data lakes

Qualifications

• Strong hands-on experience with Python development.
• Extensive experience with PySpark / Apache Spark.
• Expertise in Data Engineering, ETL development and data processing frameworks.
• Knowledge of distributed storage and data lake ecosystems (MINIO, ADLS using S3, Iceberg).
• Strong SQL skills and large-scale distributed data processing.
• Familiarity with cloud platforms such as Azure, AWS, or GCP.
• Experience with Git and version control.

Skills

Python Development

PySpark

Data Engineering

ETL Development

SQL

Big Data Concepts

Tools

Git

Azure

AWS

GCP

MINIO

ADLS (using S3)

Iceberg

Job description

• Strong hands-on experience in Python Development.
• Extensive experience with PySpark / Apache Spark.
• Expertise in Data Engineering, ETL development, and data processing frameworks. (OR) Big Data & eco systems knowledge, distributed storage like MINIO, ADLS using S3, Iceberg tables

• Strong knowledge of SQL and database concepts.
• Experience working with large-scale datasets in distributed environments.
• Understanding of Data Warehousing concepts and dimensional data modeling.
• Exposure to cloud platforms such as Azure, AWS, or GCP.
• Experience with version control tools such as Git.