Senior Data Engineer: Spark, Airflow & AWS for Big Data

JobCubby; Hinoba-an; On-site; PHP 900,000 - 1,500,000; Full time

Job summary

Proximity is seeking a Senior Data Engineer to join its Data Platform team, building scalable, high-performance data platforms powering analytics and downstream applications.

You will design, develop, and optimize large-scale data pipelines using Spark, Scala, and Airflow, with AWS and S3 for storage. Collaboration with cross-functional teams is essential to deliver reliable data solutions at scale.

Qualifications

• 5+ years of experience in Data Engineering, Big Data, or a similar role.
• Strong hands-on experience with Apache Spark and Scala.
• Experience designing, building, and maintaining large-scale ETL pipelines.
• Strong hands-on experience with AWS, particularly Amazon S3.
• Hands-on experience with Apache Airflow for workflow orchestration and scheduling.
• Strong SQL skills and a solid understanding of distributed data processing concepts.
• Experience working with batch and/or streaming data pipelines.
• Excellent debugging, problem-solving, and performance optimization skills.
• Strong communication and collaboration skills.

Responsibilities

• Design, develop, and maintain scalable ETL and data processing pipelines for large-scale datasets.
• Build and optimize distributed data applications using Apache Spark and Python/Scala.
• Develop reliable, high-performance data pipelines for batch and streaming workloads.
• Design and manage data workflows using Apache Airflow.
• Build and operate data workloads on AWS, with strong usage of Amazon S3 for large-scale data storage.
• Work with large datasets to ensure data quality, consistency, reliability, and performance.
• Collaborate with engineering, product, analytics, and other platform teams to deliver robust data solutions.
• Optimize data workflows for scalability, reliability, performance, and cost efficiency.
• Troubleshoot production issues, identify bottlenecks, and continuously improve platform performance.

Skills

Apache Spark

Scala

Airflow

SQL

AWS

Python

Tools

Databricks

Amazon S3

Job description

Proximity is seeking a Senior Data Engineer to join its Data Platform team, building scalable, high-performance data platforms powering analytics and downstream applications.

You will design, develop, and optimize large-scale data pipelines using Spark, Scala, and Airflow, with AWS and S3 for storage. Collaboration with cross-functional teams is essential to deliver reliable data solutions at scale.