PySpark Data Engineer - Big Data ETL & Analytics

Acesoft Labs (India) Pvt Ltd; Dubai; Hybrid; AED 35,000 - 50,000; Full time

Job summary

Acesoft Labs (India) Pvt Ltd in Bangalore is seeking a skilled PySpark Developer with hands-on experience in big data processing, data engineering, and distributed computing. You will develop, optimize, and maintain scalable data pipelines using PySpark, Python, and related technologies.

The role requires building ETL pipelines, tuning performance, and working with cloud-based data platforms. You will collaborate with data engineers, architects, and analysts to ensure data quality and timely

Qualifications

• Hands-on PySpark development experience.
• Strong Python programming skills.
• Good knowledge of Spark DataFrames, Spark SQL, RDDs, and transformations.
• Experience building ETL/ELT pipelines.
• SQL skills with relational databases.
• Familiarity with data warehousing concepts and large datasets.

Responsibilities

• Develop and maintain scalable ETL/ELT data pipelines using PySpark and Python.
• Design data processing solutions for large and complex datasets.
• Write optimized Spark transformations and manage DataFrames/Spark SQL.
• Perform data cleansing, validation, transformation and aggregation.
• Tune Spark jobs for performance and resource utilization.
• Work with structured and semi-structured data from multiple sources.

Skills

PySpark

Python

Big Data

ETL

SQL

Spark Architecture

Data Warehousing

Delta Lake

Airflow

CI/CD

Education

Bachelor's or Master's in CS/IT/Engineering

Tools

Databricks

Cloud platforms (AWS/Azure/GCP)

Job description

Acesoft Labs (India) Pvt Ltd in Bangalore is seeking a skilled PySpark Developer with hands-on experience in big data processing, data engineering, and distributed computing. You will develop, optimize, and maintain scalable data pipelines using PySpark, Python, and related technologies.

The role requires building ETL pipelines, tuning performance, and working with cloud-based data platforms. You will collaborate with data engineers, architects, and analysts to ensure data quality and timely