Data Engineer: AWS, Spark & AI Automation Expert

Ethos BeathChapman; Sydney; On-site; AUD 110,000 - 160,000; Full time

Job summary

Ethos BeathChapman is seeking an experienced Data Engineer to join a Sydney-based FS company, building and scaling an AWS data platform with Redshift, S3, Athena and Glue, and developing Spark jobs for batch and real-time processing.

You will also contribute to AI agent initiatives to automate processes and increase productivity, wearing multiple hats across data reporting, modelling, and platform engineering as required by the team.

Qualifications

• Experience with AWS data services (Redshift, S3, Athena, Glue).
• Spark experience for data processing, preferably in Scala.
• Experience with BI tools such as Power BI.
• Experience building AI agents to automate data engineering tasks is a plus.

Responsibilities

• Work on an AWS data platform within a Sydney-based team.
• Build Spark jobs for batch and real-time data processing.
• Collaborate with data analytics and platform engineering teams.
• Contribute to AI agent initiatives to automate processes.

Skills

AWS cloud services

Spark data processing

Scala programming

Data engineering

Tools

Spark (Scala)

Power BI

AWS services (Redshift S3 Athena Glue)

Job description

Ethos BeathChapman is seeking an experienced Data Engineer to join a Sydney-based FS company, building and scaling an AWS data platform with Redshift, S3, Athena and Glue, and developing Spark jobs for batch and real-time processing.

You will also contribute to AI agent initiatives to automate processes and increase productivity, wearing multiple hats across data reporting, modelling, and platform engineering as required by the team.