Senior PySpark Data Engineer & ETL Architect

V2 Solutions; Hinoba-an; On-site; PHP 500,000 - 900,000; Full time

Job summary

V2 Solutions in the Philippines is seeking senior data engineering expertise to build and deploy BigData applications using SparkSQL and SparkStreaming in Python. Design and implement Python-based data pipelines, and convert PowerCenter mappings to PySpark jobs.

Create PySpark frames to load data from multiple databases into Amazon S3. Expertise in graph algorithms, advanced recursion, and handling XML/JSON documents along with REST API interactions is required.

Qualifications

• 4-8 years in build & deployment of BigData apps using SparkSQL/SparkStreaming in Python
• 5+ years in design, build and deployment of Python-based apps
• ETL patterns design using Python on Spark; convert PowerCenter mappings to PySpark jobs
• Create PySpark frame to move data from databases to Amazon S3; experience with XML/JSON and REST APIs

Skills

SparkSQL

SparkStreaming

Python

PySpark

PowerCenter

XML

JSON

REST API

Amazon S3

ETL

Graph algorithms

Tools

PowerCenter

PySpark

Job description

V2 Solutions in the Philippines is seeking senior data engineering expertise to build and deploy BigData applications using SparkSQL and SparkStreaming in Python. Design and implement Python-based data pipelines, and convert PowerCenter mappings to PySpark jobs.

Create PySpark frames to load data from multiple databases into Amazon S3. Expertise in graph algorithms, advanced recursion, and handling XML/JSON documents along with REST API interactions is required.