Data Platform Engineer

ShopGrok; Chippendale (MD); Hybrid; USD 85,000 - 127,000; Full time

Benefits offered by this job

Free coffee

Weekly lunches

Team events

Hybrid working environment

Job summary

ShopGrok is seeking a Data Engineer to advance the Bedrock Platform and migrate customers from the Legacy Platform. You will shape modern data architecture, automate pipelines, and drive efficiency in a lean team.

You’ll work across data platform engineering and operations, balancing maintenance with building out scalable Bedrock pipelines. Candidates should be comfortable with legacy SQL and Alteryx ETL workflows while delivering reliable data flows.

Qualifications

• 2+ years of deep experience writing complex, performant SQL (Snowflake experience is highly regarded).
• Demonstrated ability to interpret legacy SQL code and refactor them safely.
• Experience architecting data pipelines, medallion architecture, data lineage, and test suites.

Responsibilities

• Data Architecture & Automation: Build Bedrock Platform, medallion layers, data models.
• Legacy Maintenance & Migration: balance legacy maintenance with new architecture build and migration.
• Data Operations & Quality: ensure high data quality and resolve anomalies across stages.

Skills

SQL Expertise

Legacy SQL Troubleshooting

Pipeline Architecture & Testing

Pragmatic Mindset

Data Transformation

Communication

Tools

dbt

Git

Alteryx

Tableau

Job description

About ShopGrok

ShopGrok is a fast-growing, bootstrapped SaaS company providing leading retailers and consumer brands with real-time competitive price intelligence and product insights. We track millions of price points every week, processing massive datasets to power analytics that drive critical pricing decisions. We run a lean, high-performing team that prioritises engineering pragmatic solutions over bureaucracy.

About the Role

At ShopGrok, data is at the core of everything we build. We have recently built our modern data architecture (our "Bedrock" Platform) and are mid-way through migrating our enterprise customers across to it.

In this role, you will work across both data platform engineering and data operations. A key objective is to shift operational effort into the Bedrock Platform by driving efficiency, automation, scalability and AI readiness.

There will be an active maintenance element supporting customers on our Legacy Platform for at least the next 12 months. Success in this role means striking the right balance: keeping the legacy engine running reliably while dedicating focused effort to building out the new architecture and migrating customers across. We need an engineer who is comfortable with the imperfect nature of a legacy environment, pragmatic enough to pick the low-hanging fruit in a lean team, and skilled at converting legacy SQL workflows into clean, automated Bedrock pipelines.

Key Responsibilities

1. Data Architecture & Automation

• Modern Data Architecture: Build out and refine our next-generation Bedrock Platform, utilising a Medallion Data Architecture (Bronze, Silver, Gold layers) to structure scalable data models.
• Transparent Lineage & Layered Testing: Establish transparent data lineage across all pipelines and design automated test suites at each layer of the data architecture to ensure end-to-end data quality.
• Platform Automation: Automate operational tasks and build resilient data infrastructure to eliminate repetitive break-fix work and prevent recurring issues.
• SDLC Practices: Apply modern software development practices (git version control, modular code, pull requests, continuous testing) to data pipelines.

2. Legacy Maintenance & Migration (6 to 12 Months)

• Balancing Priorities: Successfully balance ongoing maintenance and bug fixes on the Legacy Platform with dedicated build time for the new architecture.
• Legacy Code Wrangling: Dive into less structured legacy SQL logic and Alteryx ETL workflows, unpicking complex data pipelines to troubleshoot discrepancies or alter logic.
• Pragmatic Prioritisation: Comfortably navigate imperfect legacy code, identifying and executing on low-hanging fruit to deliver immediate reliability wins without getting bogged down.
• Platform Migration: Systematically refactor legacy datasets and logic into the Bedrock Platform as customers are transitioned across.

3. Data Operations & Quality

• Ensure high data quality and accuracy across incoming raw data, intermediate transformations, and outgoing analytical models.
• Trace and resolve data anomalies quickly across collection, transformation, and delivery stages.

Eligibility

• Residing in Sydney, Australia with full Australian working rights (candidates not already residing in Sydney, Australia; on temporary / part-time work visas; or requiring sponsorship will not be considered for this role)

Essential Skills

• SQL Expertise: 2+ years of deep experience writing complex, performant, and scalable SQL (Snowflake experience is highly regarded).
• Legacy SQL Troubleshooting: Demonstrated ability to interpret messy or unstructured legacy SQL code, understand existing workflows, and refactor them safely.
• Pipeline Architecture & Testing: Experience architecting data pipelines, implementing medallion architecture patterns, creating data lineage, and designing test suites across each layer.
• Pragmatic Mindset: Comfort working in a fast-paced, lean team with a knack for balancing legacy maintenance against new feature build, picking off low-hanging fruit efficiently.
• Data Transformation: High proficiency in data manipulation and cleansing techniques (joins, transposes, crosstabs, regex, aggregations).
• Communication: Clear written and verbal communication to explain data logic, lineage, and system trade-offs to internal stakeholders.

Highly Desirable (Or Demonstrated Willingness to Learn)

• dbt & Semantic Layers: Experience using dbt (or a similar tool) to build scalable data architectures, transparent lineage, and semantic layers.
• Data SDLC: Commercial experience using software engineering principles like Git, branching, and automated testing for data workflows.
• Note: If you do not have direct dbt or formal Git/SDLC experience in data, you must demonstrate strong technical aptitude and a willingness to learn quickly.
• ETL Tooling: Experience with Alteryx or similar visual ETL tools is a strong plus (though not a strict necessity).

Nice to Have

• Data Visualisation & Views: Basic familiarity or experience with Tableau or similar analytics/dashboarding tools.
• Retail Data: Prior experience handling retail, FMCG, or pricing datasets.
• Cloud Environments: Experience with GCP (Google Cloud Platform) or equivalent

About You

• Pragmatic & Resourceful: You enjoy bringing order to chaos, balancing competing priorities, and finding smart, simple solutions.
• Automation-Oriented: You dislike doing manual tasks twice and actively seek ways to automate repetitive data ops into platform capabilities.
• Software Engineering Mindset: You treat data engineering with software engineering rigour, valuing clear data lineage, testing at every layer of the medallion architecture, Git hygiene, and clean, maintainable logic.
• Collaborative: You enjoy working alongside software developers, platform engineers, and customer teams to keep data flowing smoothly

Core Tech Stack

• Core Platforms: Snowflake (Medallion Architecture), GCP (Google Cloud Platform)
• Data & Engineering: SQL, dbt, Git
• Legacy & Visualisation: Alteryx, Tableau

• Direct Impact: Play a pivotal role in migrating our core technology and scaling a platform used by leading brands.
• Career Growth: Opportunity to mature into a Senior Data Engineering role as our engineering organisation expands.
• Vibrant Culture: Hybrid working environment based in Chippendale, Sydney, with free coffee, weekly lunches, and regular team events.
• Competitive Package: Competitive salary with startup perks in a profitable, sustainable SaaS business.