Senior Big Data Engineer – Real-Time Analytics & Spark

Pyramid Global Technologies

New South Wales

On-site

AUD 120,000 - 180,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Pyramid Global Technologies in New South Wales, Australia seeks an experienced Data Engineer to join our analytics team and drive data engineering and warehousing initiatives. You will work across ingestion, processing, and storage layers using Spark, Kafka, Hive, and related technologies to build scalable analytics platforms.

Strong SQL, ETL, and data profiling skills are essential to deliver reliable data pipelines.

Qualifications

5+ years experience working in Data Engineering and Warehousing.
3-5 years’ experience integrating data into analytical platforms.
Experience in ingestion technologies (e.g. Sqoop, flume), processing technologies (Spark/Scala), and storage (e.g. HDFS, HBase, Hive).
Experience in data profiling, source-target mappings, ETL development, SQL optimization, testing, and implementation.
Expertise in streaming frameworks (Kafka/Spark Streaming/Storm) essential.
Experience building Spark Sql and Spark Data Frame API-based applications.
Experience managing structured and unstructured data types.
Experience in requirements engineering, solution architecture, design, and development/deployment.
Experience in creating big data or analytics IT solution.
Track record of implementing databases and data access middleware and high-volume batch and near real-time processing.