Pyspark Developer - Virtual Interview - 2 June(Tuesday)

V2 Solutions; Hinoba-an; On-site; PHP 500,000 - 900,000; Full time

Job summary

V2 Solutions in the Philippines is seeking senior data engineering expertise to build and deploy BigData applications using SparkSQL and SparkStreaming in Python. Design and implement Python-based data pipelines, and convert PowerCenter mappings to PySpark jobs.

Create PySpark frames to load data from multiple databases into Amazon S3. Expertise in graph algorithms, advanced recursion, and handling XML/JSON documents along with REST API interactions is required.

Qualifications

• 4-8 years in build & deployment of BigData apps using SparkSQL/SparkStreaming in Python
• 5+ years in design, build and deployment of Python-based apps
• ETL patterns design using Python on Spark; convert PowerCenter mappings to PySpark jobs
• Create PySpark frame to move data from databases to Amazon S3; experience with XML/JSON and REST APIs

Skills

SparkSQL

SparkStreaming

Python

PySpark

PowerCenter

XML

JSON

REST API

Amazon S3

ETL

Graph algorithms

Tools

PowerCenter

PySpark

Job description

• Minimum 4-8 years of experience in build & deployment of Bigdata applications using SparkSQL, SparkStreaming in Python;
• 2. Minimum 5 years of extensive experience in design, build and deployment of Python-based applications;
• 3. Design and develop ETL integration patterns using Python on Spark. Develop framework for converting existing PowerCenter mappings and to PySpark(Python and Spark) Jobs.
• 4. Create Pyspark frame to bring data from multiple databases to Amazon S3. Expertise on graph algorithms and advanced recursion techniques. Hands-on experience in generating/parsing XML, JSON documents, and REST API request/responses.