Data Platform Engineer — Databricks, AWS & Tableau Cloud

GCS Recruitment; Pittsburgh (Allegheny County); On-site; USD 100,000 - 140,000; Full time

Job summary

GCS Recruitment is seeking an experienced Data Engineer to join the Supply/Yield Data Engineering team. You will design and maintain data pipelines in Databricks, enable analytics in Tableau Cloud, and support Looker-to-Tableau migrations while modernizing legacy data solutions.

You will work closely with stakeholders, build cloud-native pipelines on AWS, and implement IaC with Terraform. This role emphasizes reliability, scalability, and collaboration across the data platform, through 2026 and

Qualifications

• Strong data engineering and ETL/ELT development experience.
• Hands-on Databricks experience with large datasets.
• Experience building data pipelines in AWS and Snowflake.
• Proficient in Spark / PySpark.
• Hands-on Terraform and IaC experience.
• Knowledge of CI/CD practices and tools such as GitHub Actions, Concourse, Jenkins.
• Experience supporting and troubleshooting production data pipelines.
• Strong testing, data quality, and operational support skills.
• Excellent problem-solving and collaboration abilities.

Responsibilities

• Design, develop, and maintain ETL/ELT pipelines into Databricks.
• Build scalable data pipelines using AWS, Databricks, Snowflake.
• Collaborate with stakeholders to translate requirements into data solutions.
• Develop data assets and datasets for analytics and reporting in Tableau Cloud.
• Support Looker to Tableau migration efforts.
• Create technical documentation and implementation guides.
• Perform testing, validation, and QA for data accuracy.
• Provide post-delivery support and incident resolution for production workloads.
• Implement IaC solutions using Terraform and manage CI/CD with GitHub Actions or Concourse.

Job description

Overview

We are looking for an experienced Data Engineer to join the Supply/Yield Data Engineering team and support a large-scale data modernization and analytics transformation programme through 2026.

This role will focus on designing, developing, and maintaining data pipelines that bring data into Databricks, support Inventory and Yield analytics initiatives, and enable business-critical reporting within Tableau Cloud. The successful candidate will also contribute to the migration of reporting assets from Looker to Tableau while helping modernize existing legacy data engineering solutions.

Key Responsibilities

• Design, develop, and maintain ETL/ELT pipelines to ingest and transform data into Databricks.
• Build and support scalable data pipelines using AWS, Databricks, and Snowflake.
• Work closely with business and technical stakeholders to understand requirements and translate them into robust data solutions.
• Develop core data assets and datasets that support analytics and reporting consumed through Tableau Cloud.
• Support the Inventory ART programme by delivering Data Engineering solutions and increasing team capacity during migration activities.
• Enhance and optimize existing legacy data engineering jobs and processes.
• Contribute to the migration of reporting solutions from Looker to Tableau.
• Create and maintain technical documentation, specifications, and implementation guides.
• Perform testing, validation, and quality assurance activities to ensure data accuracy and reliability.
• Provide post-delivery support, incident resolution, and troubleshooting for production workloads.
• Implement Infrastructure as Code (IaC) solutions using Terraform.
• Build and manage CI/CD pipelines using Concourse or GitHub Actions to enable reliable and automated deployments.
• Help identify and mitigate risks, including reducing Single Points of Failure (SPOF) through documentation, automation, and knowledge sharing.
• Collaborate with fellow engineers to establish and promote engineering best practices.

Required Skills & Experience

• Strong experience in Data Engineering and ETL/ELT development.
• Hands-on experience with Databricks.
• Strong SQL skills with experience working on large datasets.
• Experience building data pipelines within AWS environments.
• Experience with Snowflake data warehousing.
• Experience with Spark/PySpark.
• Hands-on experience with Terraform and Infrastructure as Code (IaC).
• Knowledge of CI/CD practices and tools such as GitHub Actions, Concourse, Jenkins, or similar.
• Experience supporting and troubleshooting production data pipelines.
• Strong understanding of testing, data quality, and operational support processes.
• Excellent problem-solving and collaboration skills.

Nice to Have

• Experience with Tableau or Tableau Cloud.
• Experience supporting Looker to Tableau migrations.
• Experience working within Supply Chain, Inventory, Yield, or Manufacturing domains.
• Data modelling and analytics engineering experience.
• Experience modernizing legacy data platforms and applications.

Technology Stack

• Databricks
• AWS
• Snowflake
• SQL
• Spark / PySpark
• Terraform
• GitHub Actions / Concourse
• Tableau Cloud
• Looker (Migration Support)

Project Highlights

• Work on a strategic data modernization programme running through 2026.
• Build the core data pipelines powering Inventory, Supply, and Yield analytics.
• Support a major Looker-to-Tableau reporting migration.
• Develop cloud-native solutions using modern data engineering technologies.
• Play a key role in improving automation, reliability, scalability, and operational excellence across the data platform.

GCS is acting as an Employment Business in relation to this vacancy.