Data Engineer

Boltin; City of Melbourne; On-site; AUD 120,000 - 170,000; Full time

Job summary

Boltin is seeking an experienced Data Engineer in Melbourne to build and optimize scalable data pipelines using Informatica and cloud data platforms. You will design and manage BigQuery datasets, write high-performance SQL, and automate repetitive tasks with Python and Unix shells, collaborating with analysts to deliver reliable analytics.

This role is on-site in Melbourne with opportunities to influence data strategy and governance.

Qualifications

• Experience designing and supporting production data pipelines.
• Strong SQL, data validation, and performance tuning skills.
• Experience with cloud migration or hybrid data integrations is desirable.

Responsibilities

• Build and maintain scalable ETL/ELT workflows using Informatica.
• Manage BigQuery datasets, partitions, clustering for large-scale analytics.
• Write and tune advanced SQL scripts to boost performance and reduce costs.
• Use Unix shell scripting to automate file transfers and batch jobs.
• Develop Python scripts to parse files and interact with cloud APIs.
• Schedule, monitor, and troubleshoot workflows across cron jobs and cloud schedulers.
• Migrate legacy on-prem data assets to cloud via hybrid ETL pipelines.
• Implement data validation, logging, and error handling in SQL/Python.
• Diagnose bottlenecks in Informatica mappings, BigQuery queries, and shell scripts.
• Maintain technical mapping specs and runbooks, collaborating with analysts and stakeholders.

Skills

Advanced SQL

Python programming

Unix shell scripting

Data integration

Agile delivery

Education

Bachelor's degree in Computer Science, Information Technology, Data Engineering

Tools

Informatica

Google BigQuery

Job description

• Build and maintain scalable data extraction, transformation, and loading workflows using Informatica
• Manage tables, datasets, partitions, and clustering inside Google BigQuery for large-scale analytics.
• Write and tune advanced SQL scripts and queries to boost execution speed and lower compute costs.
• Use Unix shell scripting to automate file transfers, pre- and post-load routines, and batch job sequencing.
• Develop custom Python scripts to parse files, orchestrate data movement, and interact with cloud APIs.
• Schedule, monitor, and troubleshoot integrated workflows across Unix cron jobs and cloud schedulers.
• Move legacy on-premise data assets into cloud environments via hybrid ETL pipelines.
• Implement error-handling, logging, and data validation rules using SQL and Python checks.
• Diagnose bottlenecks in Informatica mappings, BigQuery queries, and shell scripts.
• Maintain technical mapping specs, system architecture diagrams, and partner with analysts or data

Key Skills & Experience

• 4-7 years of hands-on data engineering experience designing, developing, and supporting production data pipelines.
• Experience building ETL/ELT workflows using Informatica, including mapping development, workflow scheduling, monitoring, and troubleshooting.
• Proven capability working with Google BigQuery, including dataset management, table design, partitioning, clustering, and query performance optimisation.
• Advanced SQL skills, including complex joins, window functions, stored procedures, query tuning, and data validation routines.
• Practical experience with Unix shell scripting for file handling, batch orchestration, job sequencing, and automation of operational tasks. (desired)
• Proficiency in Python for data processing, file parsing, API integration, automation, and development of reusable data engineering utilities.
• Experience supporting cloud migration or hybrid data integration initiatives, including movement of legacy on-premise data assets to cloud platforms. (desired)
• Good understanding of data quality, reconciliation, error handling, logging, and operational monitoring practices across batch and scheduled workloads.
• Ability to diagnose and resolve technical issues across Informatica mappings, SQL queries, shell scripts, and cloud data pipelines.
• Experience maintaining technical documentation, mapping specifications, operational runbooks, and working closely with analysts, testers, and business stakeholders.
• Familiarity with Agile delivery practices, code version control, release processes, and collaborative delivery across onshore and offshore teams.
• Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related discipline; relevant cloud, data, or Informatica certifications are desirable.