Data Engineer

Platoon Consulting; Abu Dhabi; On-site; AED 120,000 - 200,000; Full time

Job summary

A consulting firm based in Abu Dhabi is seeking a skilled candidate to design and launch data extraction and transformation processes. Responsibilities include web crawling, data cleaning, and maintaining data pipelines. Strong programming skills in Python, Shell, and Java are required, along with experience in AWS services like EC2 and S3. This role offers a unique opportunity to work with complex data sets in a dynamic environment.

Qualifications

• Solid coding skills in Python, Shell, and Java.
• Experience with AWS services like EC2, S3, SQS.
• Familiarity with data ingestion and ETL processes.

Responsibilities

• Design, build and launch new data extraction processes.
• Perform web crawling and data cleaning.
• Maintain and create data pipelines.

Skills

Python

Shell

Java

AWS

ETL

CI/CD

Data Processing

Job description

Responsibilities

Design, build and launch new data extraction, transformation & loading processes in production. Web crawling, data cleaning, data annotation, data ingestion and data processing Reading & collating complex data sets Creating maintaining data pipelines

Required Candidate profile

• Programming solid coding skills in Python Shell & Java
• Solution architecture data ingestion query optimization data segregation ETL ELT AWS EC2 S3 SQS lambda Elastic Search Redshift CI/CD frameworks