AWS Data Platform Engineer

Synechron; Sydney; On-site; AUD 120,000 - 140,000; Full time

Job summary

A technology consulting company in Sydney is seeking an experienced AWS Data Platform Engineer. The successful candidate will design and implement scalable data solutions, optimize data lake performance, and work with data scientists to manage efficient data processing. Ideal candidates should have deep knowledge of AWS data services and Apache Iceberg, alongside strong coding skills in languages such as Python, Java, or Scala.

Qualifications

• Extensive experience with AWS data services and strong understanding of Apache Iceberg.
• Proven track record in building and optimizing data lakes.
• Proficiency in working with big data processing tools.

Responsibilities

• Implement and manage data lake solutions using Apache Iceberg.
• Optimize data storage and querying by integrating Iceberg tables into AWS services.
• Monitor, troubleshoot, and optimize performance of data solutions.

Skills

AWS data services

Apache Iceberg

Python

Java

Scala

Infrastructure as Code (IaC)

Data orchestration

Analytical skills

Problem-solving

Communication

Tools

Apache Airflow

AWS CloudFormation

Terraform

Job description

Overview

We are seeking a seasoned AWS Data Platform Engineer. The successful candidate will be responsible for designing and implementing scalable and efficient data storage and retrieval systems, optimizing data lake performance, and ensuring the smooth handling of our data workloads on AWS.

Key Responsibilities

• Implement and manage data lake solutions using Apache Iceberg to handle time-travel, schema evolution, and transaction support.
• Optimize data storage and querying by integrating Iceberg tables into AWS services like S3, Glue, Redshift, and EMR.
• Work closely with data scientists and analysts to ensure seamless data access and efficient data processing workflows.
• Monitor, troubleshoot, and optimize performance of data solutions, managing scalability and cost-effectiveness.
• Keep up-to-date with the latest AWS offerings and Apache Iceberg features, continuously improving our data infrastructure.
• Develop best practices for data governance, security, and compliance within our AWS and Iceberg environments.
• Document system designs, architectures, and operational manuals for data solutions.

Qualifications

• Extensive experience with AWS data services and a strong understanding of Apache Iceberg.
• Proven track record in building and optimizing data lakes and data warehouse solutions.
• Proficiency in working with big data processing tools and frameworks.
• Strong coding skills in languages such as Python, Java, or Scala.
• Experience with infrastructure as code (IaC), such as AWS CloudFormation or Terraform.
• Familiarity with data orchestration tools like Apache Airflow.
• Excellent analytical, problem-solving, and teamwork skills.
• Strong communication abilities for both technical and non-technical audiences.

Interested candidates EASY APPLY.