Senior Data Engineer

SmartChoice International GCC; Riyadh; Vor Ort; SAR 320.000 - 520.000; Vollzeit

Zusammenfassung

SmartChoice International GCC is seeking a Senior Data Engineer to help build and scale its data platform. This hands-on role covers ingestion, processing, storage, modelling and governance, delivering reliable data infrastructure used by multiple applications.

You will shape engineering standards as the platform evolves and collaborate with cross-functional teams to improve data quality, security and performance. Strong Python/SQL, AWS data services and lakehouse experience are essential.

Qualifikationen

• 8+ years of professional data engineering experience.
• Strong Python and SQL skills with distributed processing.
• Experience with AWS data services (S3, Glue, Athena, Redshift).
• Experience designing data lake or lakehouse architectures.
• Experience with ETL/ELT and modern data tools (dbt, Airbyte, Fivetran)

Aufgaben

• Design, build and operate scalable batch and streaming data pipelines.
• Develop data architecture across ingestion, processing, storage and transformation.
• Build datasets and data models for downstream applications.
• Implement data quality, validation, monitoring, lineage and governance.
• Design for reliability with idempotency, deduplication, schema evolution, recovery.
• Implement access controls and protection for sensitive data.
• Work with engineering and infrastructure to operate production data services.
• Optimize performance, scalability and costs.
• Contribute to technical standards, documentation and code reviews.

Kenntnisse

Python

SQL

PySpark

Data governance

Data quality

Tools

S3

Glue

Glue Data Catalog

Athena

Redshift

Airbyte

Fivetran

Meltano

Kinesis

MSK

SQS

SNS

DMS

Debezium

dbt

Parquet

Iceberg

Delta Lake

Hudi

MWAA

Step Functions

Dagster

Prefect

Jobbeschreibung

We are partnering with a growing technology organisation to appoint a Senior Data Engineer to help build and scale its data platform.

This is a hands-on role with ownership across data ingestion, processing, storage, modelling and governance. You will build reliable data infrastructure used across multiple technical and business applications, while helping shape engineering standards as the platform evolves.

Key Responsibilities

• Design, build and operate scalable batch and streaming data pipelines.
• Develop data architecture across ingestion, processing, storage and transformation layers.
• Build reliable, reusable datasets and data models for downstream applications and teams.
• Implement data quality, validation, monitoring, lineage and governance.
• Design for reliability through idempotency, deduplication, schema evolution, recovery and historical accuracy.
• Implement appropriate access controls and protection for sensitive data.
• Work closely with engineering and infrastructure teams to operate production data services.
• Optimise performance, scalability and infrastructure costs.
• Contribute to technical standards, documentation and code reviews.

What We're Looking For

• 8+ years of professional data engineering experience.
• Strong Python and SQL skills, including PySpark or comparable distributed processing technologies.
• Strong production experience with AWS data services, particularly S3, Glue, Glue Data Catalog, Athena and Redshift.
• Experience designing and operating data lake or lakehouse architectures, including layered data transformation.
• Experience with ETL/ELT and integration technologies such as Airbyte, Fivetran or Meltano.
• Experience with streaming, event-driven or change-data-capture technologies such as Kinesis, MSK, SQS, SNS, DMS or Debezium.
• Experience with dbt or comparable data transformation and modelling tools.
• Familiarity with Parquet and modern table formats such as Iceberg, Delta Lake or Hudi.
• Experience with workflow orchestration tools such as MWAA, Step Functions, Dagster or Prefect.
• Strong understanding of data quality, governance, security and lineage.

AI & Advanced Data

Experience building data infrastructure for AI or other advanced data