Cloud Data Engineer: Build Scalable BigQuery Pipelines

OutsideCapital; Cape Town; On-site; ZAR 700,000 - 1,200,000; Full time

Job summary

OutsideCapital in Cape Town is seeking a senior data engineer to design, build and optimize cloud data pipelines and architectures. You will turn varied operational data into dependable analytical products with a focus on reliability, scale and cost efficiency.

You will work hands-on with Google Cloud Platform, BigQuery, Python and SQL, owning ETL processes, data models and data quality, lineage, and performance improvements while collaborating with data users across the business.

Qualifications

• Delivery of production data pipelines using GCP, BigQuery, Python and SQL.
• Design and maintain ETL processes transforming varied source data into analytical datasets.
• Experience building data models with data quality, validation and lineage.
• Diagnose and improve pipeline performance, scalability or cost.
• Hands-on exposure to Dataflow, Pub/Sub and Cloud Storage is advantageous.

Skills

BigQuery

Python

SQL

Data pipelines

ETL

Data modeling

Data quality

Cost optimization

Education

Bachelor's degree in a relevant field

Tools

Dataflow

Pub/Sub

Cloud Storage

Job description

OutsideCapital in Cape Town is seeking a senior data engineer to design, build and optimize cloud data pipelines and architectures. You will turn varied operational data into dependable analytical products with a focus on reliability, scale and cost efficiency.

You will work hands-on with Google Cloud Platform, BigQuery, Python and SQL, owning ETL processes, data models and data quality, lineage, and performance improvements while collaborating with data users across the business.