Data Engineer – PySpark/Python | Big Data & Cloud | 6+ Years Experience | Any UST Location | Im[...]

UST; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

UST in Bengaluru is seeking a Data Engineer with over 6 years of experience to develop scalable data pipelines and optimize cloud data processing.

The ideal candidate should have strong expertise in PySpark or Python, along with hands-on experience in Hadoop, Spark, and Hive. Knowledge of AWS Glue, AWS EMR, GCP Dataproc, and GCP BigQuery is essential.

Qualifications

• 6+ years of experience in Data Engineering.
• Strong expertise in PySpark or Python.
• Hands-on experience with Hadoop, Spark, and Hive.
• Experience with AWS Glue and AWS EMR.
• Experience with GCP Dataproc and GCP BigQuery.
• Strong ETL and SQL development experience.
• Experience in building scalable data pipelines.
• Strong understanding of cloud-based data processing.

Skills

Data Engineering

PySpark or Python

Hadoop

Spark

Hive

AWS Glue

AWS EMR

GCP Dataproc

GCP BigQuery

ETL

SQL

Data Pipelines

Cloud-based Data Processing

Job description

Candidates ready to join immediately can share their details via email for quick processing.

[e-mail]

Must-Have Skills

• 6+ years of experience in Data Engineering
• Strong expertise in PySpark or Python
• Hands-on experience with Hadoop, Spark, Hive
• Experience with AWS Glue, AWS EMR
• Experience with GCP Dataproc and GCP BigQuery
• Strong ETL and SQL development experience
• Experience in building scalable and high-performance data pipelines
• Strong understanding of cloud-based data processing and optimization

Good-to-Have Skills

• Experience with Airflow or scheduling/orchestration tools
• Exposure to Kafka or streaming technologies
• Knowledge of Docker/Kubernetes
• Experience in Agile/Scrum environment
• Understanding of data warehousing concepts