PySpark Developer

V2 Solutions; Hinoba-an; On-site; PHP 600,000 - 900,000; Full time

Job summary

V2 Solutions is seeking a data analytics professional to design and maintain large-scale data pipelines using PySpark. You'll work with distributed data platforms like Hadoop/Hive and cloud storage, turning raw data into actionable insights for business teams.

The role emphasizes data quality and governance, ETL/ELT workflow design, and performance tuning of Spark jobs. You will collaborate with engineers and analysts, write clean code, and automate workflows with tools such as Airflow or Oozie.

Qualifications

• Bachelor-level engineering or related degree required.
• Degrees in technology fields such as BTech/BSc/BCA/MTech/MSc/MCA recognized.
• Data analytics focus is preferred.

Responsibilities

• Develop and maintain data pipelines using PySpark
• Process and analyze large-scale datasets in distributed environments
• Design and implement ETL/ELT workflows
• Optimize Spark jobs for performance and scalability
• Work with data stored in HDFS, Hive, or cloud storage (S3, ADLS)
• Collaborate with data engineers, analysts, and business teams
• Ensure data quality, integrity, and governance
• Debug and troubleshoot data processing issues
• Automate workflows using scheduling tools (Airflow, Oozie, etc.)
• Write clean, scalable, and efficient code

Skills

Python PySpark

Apache Spark

HDFS Hive

ETL Pipeline

SQL & Databases

Data Formats

Distributed Computing

Git

Education

B.E./BTech

BSc

BCA

MTech

MSc

MCA

Job description

Educational Requirements

• Bachelor of Engineering
• BTech
• BSc
• BCA
• MTech
• MSc
• MCA

Service Line

Data Analytics

Unit Responsibilities

• Develop and maintain data pipelines using PySpark
• Process and analyze large-scale datasets in distributed environments
• Design and implement ETL/ELT workflows
• Optimize Spark jobs for performance and scalability
• Work with data stored in HDFS, Hive, or cloud storage (S3, ADLS)
• Collaborate with data engineers, analysts, and business teams
• Ensure data quality, integrity, and governance
• Debug and troubleshoot data processing issues
• Automate workflows using scheduling tools (Airflow, Oozie, etc.)
• Write clean, scalable, and efficient code

Technical and Professional Requirements

Technical Skills

• Strong proficiency in Python and PySpark
• Good experience with Apache Spark (RDDs, DataFrames, Spark SQL)
• Knowledge of Hadoop ecosystem (HDFS, Hive)
• Experience in ETL pipeline development
• Familiarity with SQL and database concepts
• Experience with data formats (Parquet, ORC, JSON, CSV)
• Basic understanding of distributed computing concepts
• Exposure to version control tools (Git)

Technical and Professional Requirements

Technology->Big Data - Data Processing->PySpark Preferred Skills Technology->Big Data - Data Processing->PySpark