DataOps Engineer: Metadata-Driven Ingestion & Automation

Mars; Warszawa; Hybrid; PLN 180,000 - 320,000; Full time

Benefits offered by this job

Hybrid work option

Career progression

Job summary

Mars Data Platform Services (DPS) is transforming our enterprise data platform across Azure Databricks and Unity Catalog into an automated, self-service product. We are looking for a DataOps Engineer to lead the technical evolution of our data inflow and landing zone ecosystem.

You will build the central framework that turns pipeline creation, data onboarding, storage provisioning, and data archiving into automated, configuration-driven processes.

Qualifications

• 4+ years of hands-on data engineering experience building reliable batch and streaming pipelines using Python/PySpark, SQL, and cloud data warehouses or lakehouses (Azure Databricks / Delta Lake).
• Proven track record designing or maintaining config-driven/metadata-driven frameworks where pipeline generation, schema creation and Delta merge logic are driven dynamically by central control tables.

Responsibilities

• Build & Scale the Metadata-Driven Ingestion Engine.
• Lead Landing Zone Reliability & Data Quality Safeguards.
• Automate Unity Catalog Assets & Long-Term Archiving.
• Champion DataOps Standards & Operational Enablement.

Skills

Python/PySpark

SQL

Azure Databricks

Delta Lake

Metadata-driven frameworks

CI/CD

Git

Terraform

Tools

Terraform

Git

Databricks APIs

Job description

Mars Data Platform Services (DPS) is transforming our enterprise data platform across Azure Databricks and Unity Catalog into an automated, self-service product. We are looking for a DataOps Engineer to lead the technical evolution of our data inflow and landing zone ecosystem.

You will build the central framework that turns pipeline creation, data onboarding, storage provisioning, and data archiving into automated, configuration-driven processes.