Senior Data Engineer (AI Engineering)

HR Talent Partner SA; United States; Remote; USD 120,000 - 180,000; Full time

Job summary

HR Talent Partner SA is seeking a Senior Data Engineer (AI Engineering) for a fully remote role with optional hybrid office time in South Africa. The role focuses on building and operating pipelines feeding daily reporting, attribution and LTV, collaborating with Analytics, Marketing and Data Science.

Requirements include experience with Python, SQL, Spark, Databricks, Airflow, and GCP, plus strong English communication and agile methods. Nice-to-have: dbt and ads/payments APIs.

Qualifications

• Experience with Python, SQL, Spark, Databricks (Unity Catalog and Delta), Apache Airflow and GCP (GCS and BigQuery).
• Comfort owning production data pipelines end-to-end.
• Curiosity about marketing spend, subscriptions and attribution, not only the pipelines.

Responsibilities

• Ingest data from ad platforms, payment systems and product backends into Databricks.
• Orchestrate jobs in Airflow and keep silver and gold tables accurate and on time.
• Own production: scheduled jobs, data quality checks, alerting and getting a failed daily run back on track.

Skills

Python

SQL

Spark

Databricks

Airflow

GCP

BigQuery

Unity Catalog

Delta

English (Advanced)

Tools

Databricks

Airflow

GCP

BigQuery

Job description

Senior Data Engineer (AI Engineering) – Remote (South Africa).

A well establish international software development company is looking for a Senior Data Engineer (AI Engineering)to join their team in either Somerset West, Pretoria or Johannesburg.

This is a fully remote position, with the option to work from the office once a week if you prefer a hybrid working arrangement. You also have the option of choosing between permanent or contract employment, with a requirement to work 8 normal working hours per day.

Overview:

A growing data platform team is looking for a Senior Data Engineer to align reliable marketing, payments and product data with business goals.

You will build and operate the pipelines that feed daily reporting, attribution and LTV, working closely with Analytics, Marketing and Data Science.

If you enjoy solving complex problems and caring about the business behind the tables, this role is for you.

Requirements:

• Experience with Python, SQL, Spark, Databricks (Unity Catalog and Delta), Apache Airflow and GCP (GCS and BigQuery).
• Comfort owning production data pipelines end to end.
• Curiosity about marketing spend, subscriptions and attribution, not only the pipelines that move the data.
• Excellent analytical and problem-solving abilities with a strategic approach.
• Strong collaboration and communication skills across all levels.
• A solid understanding of agile methodologies.
• Advanced spoken and written English.
• Nice-to-have:
• • dbt.
  • Ads and payments APIs such as Google Ads, Stripe or RevenueCat.

Responsibilities:

• Ingest data from ad platforms, payment systems and product backends into Databricks.
• Orchestrate jobs in Airflow and keep silver and gold tables accurate and on time.
• Own production: scheduled jobs, data quality checks, alerting and getting a failed daily run back on track.
• Proactively identify, troubleshoot and resolve issues to improve internal and external processes.
• Balance multiple projects without sacrificing quality, in line with brand standards and regulatory requirements.
• Work with product, engineering and marketing stakeholders to gather requirements, evaluate optionsand build tools and processes.

Interview Process:

They do not have an extensive interview process and normally it is only three parts:

• A technical interview.
• An online interview.
• After successful completion, contract negotiation will take place.