Data Engineer

Roy Morgan Research; Melbourne; On-site; AUD 120,000 - 150,000; Full time

Benefits offered by this job

Vibrant work culture

Competitive salary package

Company social events

EAP service

MYKI pass discounts

Job summary

Roy Morgan is seeking a Data Engineer to join the Data Engineering team, building a scalable data platform on AWS SageMaker Unified Studio. You’ll enable data science and analytics users with governed, self-service access, while setting standards for Redshift and data storage.

The role emphasizes designing data pipelines, cubes for analytics, and modernizing legacy pipelines with security, governance and observability from day one.

Qualifications

• 3-5+ years' experience in Data Engineering or similar role, preferably in big data or market research.
• Strong data pipeline design, ETL/ELT, and data modelling for lake/warehouse.
• Python and SQL programming skills; R a plus.
• Hands-on with AWS data services: Redshift, Glue, S3, Lambda, Fargate, Lake Formation, EMR, SageMaker.

Responsibilities

• Build data platform on SageMaker Unified Studio for governed self-service access.
• Set and apply Redshift storage standards, schemas, partitioning, performance.
• Design and maintain OLAP/semantic cubes for analytics.
• Develop data APIs and file-based data access patterns.
• Design and build data pipelines (ingestion, ETL/ELT, transformation).
• Develop data lake/warehouse with suitable engines for performance.
• Move legacy batch pipelines to modern cloud platforms.
• Incorporate data quality, lineage, observability, security and governance.

Skills

Data engineering

Big data

Python

SQL

AWS

Redshift

SageMaker

ETL/ELT

Data modelling

OLAP

CI/CD

Data governance

REST/JSON

Security/compliance

Education

Bachelor's degree in IT/Data Science

Tools

Redshift

Glue

S3

Lambda

Fargate

Lake Formation

EMR

SageMaker Unified Studio

Job description

Roy Morgan is Australia's most reputable and longest established market research company, with an unparalleled reputation for reliable, accurate, meaningful, revealing market research. At the forefront of data and research innovation since 1941, today Roy Morgan continues to provide accurate and independent information insights to leading Australian and international companies and governments, as well as partnering with these same companies on bespoke data applications and technology development.

About The Role

The Technology department is responsible for positioning Roy Morgan as an efficient and innovative Research Company that employs best-in-breed technologies and methodologies to generate powerful insights for its customers.

Over the past year we've laid the foundations for API-enabled access to our data for clients, backed by Amazon and other Cloud Engineering. The big push now is platform enablement: building a single AWS data platform on Amazon SageMaker Unified Studio that serves all of Roy Morgan's data needs. Our Data Engineering team sits at the core of that platform, supporting our existing Data Science and Data Analyst teams with reliable, well-governed data.

Reporting to the Head of Information Technology, you will join the Data Engineering team and work hands-on across the whole data platform. You will:

• Build and enable our data platform on Amazon SageMaker Unified Studio, giving our Data Science and Data Analyst teams governed, self-service access to the data they need
• Set and apply Redshift data storage standards, covering modelling, schemas, naming, partitioning and performance
• Design, build and maintain cubes that give analysts and clients fast, consistent views of our data
• Build and improve the queries and data contracts behind API-enabled client data access
• Design and build data pipelines (ingestion, ETL/ELT, transformation and modelling) on AWS
• Build out our data lake and warehouse, including selecting and tuning engines for wide columnar storage and fast query performance
• Move legacy, server-scheduled batch processing onto modern managed, serverless and containerised pipeline platforms
• Build data quality, lineage, reconciliation and observability into the platform from day one
• Apply data-security best practice, including role-based access, masking, encryption, auditing and secrets management
• Bring an R&D mindset by testing and introducing new data tools and engines that fit Roy Morgan's needs

You will work closely with the Head of IT, Tech Lead, engineering teams, Data Science and Data Analyst teams, Research and Sales. The aim is to deliver data products that cut the cost of processing our data and open up new ways of getting it to clients. This is a unique opportunity to help shape how one of Australia's most respected data sets is stored, processed and delivered.

About You

Ideally, your skills and experience will include:

• 3- 5+ years' experience in Data Engineering or a similar role, preferably in "big data" and/or market research
• A strong background in data pipeline design, ETL/ELT processes and data modelling for lake and warehouse architectures
• Strong practical programming skills in Python and SQL (R a plus)
• Hands-on experience with AWS data services, including Redshift, Glue, S3, Lambda, Fargate, Lake Formation, EMR, Amazon SageMaker and SageMaker Unified Studio
• Experience defining and applying data storage standards in Redshift or a similar cloud data warehouse
• Experience designing and building cubes (OLAP or semantic layer) for analytical reporting
• Experience enabling a shared data platform for Data Science and Data Analyst users
• Experience re-platforming legacy batch pipelines onto modern cloud pipeline platforms
• DevOps principles applied to data engineering: version control, CI/CD, automated testing of data work, infrastructure as code and secrets management
• Experience with API (REST/JSON) and file-based (SFTP) integration patterns
• Familiarity with data governance, lineage, data quality and privacy/security obligations
• Exposure to AI and machine learning in real-world data solutions (a plus)
• Strong analytical, problem-solving and research skills
• Well-developed communication skills, with the ability to explain complex concepts to technical and non-technical stakeholders and to work as a consultative partner to the business
• A Bachelor's degree in Information Technology, Data Science or a related discipline

Our Offer to You

We believe in investing in our people and when you join us, you can expect:

• Vibrant work culture
• Competitive salary package
• Company social events, including Friday night activities
• Company EAP service
• Benefits including entertainment events, annual flu vaccine, discounted MYKI passes, and corporate discount for health insurance

To be considered for this role you must have full Australian working rights at the time of application.

Privacy Statement

Roy Morgan is committed to protecting your privacy. You can read our privacy policy by clicking on the following link www.roymorgan.com/legal/privacy. The information provided by you will be used to assess and process your application for recruitment. To do this, we may share your information with any recruitment agency engaged to assist with our recruitment processes. Should you, at any time, choose not to proceed with your application you may exit the recruitment process by emailing **@roymorgan.com and advising us in writing of your decision.

We value diversity and encourage applications from people of all backgrounds, including Aboriginal and Torres Strait Islander peoples.

Background Screening

Successful applications will be subject to pre-employment and in some circumstances, ongoing background checks which include a criminal history check.