Senior Data Platform Engineer: Lakehouse for ML Data

Socket.dev; Calgary; Hybrid; CAD 110,000 - 170,000; Full time

Job summary

Precision AI is hiring a senior data platform engineer in Calgary to own the data platform for AI training on drone imagery and telemetry. You’ll architect the lakehouse, set up ingestion pipelines, and define data contracts while collaborating with ML engineers to deliver curated, versioned training datasets.

This hybrid role sits at Precision AI’s Calgary HQ three days per week. You will drive data quality, reproducibility, and scalable cost controls, enabling faster model development and

Qualifications

• 4+ years building and operating production data platforms.
• Strong Python with solid software engineering fundamentals: testing, code structure, version control, and code review.
• Deep hands-on AWS experience, particularly S3, Athena or equivalent query engines, IAM, and cost management at scale.
• Production experience with a workflow orchestrator (Airflow, Dagster, Prefect, Step Functions, or similar).
• Advanced SQL and demonstrable data modeling judgment.
• Experience with columnar formats and open table formats: Parquet plus Iceberg, Delta, or Hudi.
• Track record of designing for reproducibility, including data versioning, lineage, and backfill correctness.
• Comfort operating without an existing platform to lean on and the judgment to sequence what gets built first.

Responsibilities

• Design the lakehouse architecture over S3 and Parquet footprint, including table format selection and partitioning strategy.
• Decide what stays as raw imagery and what becomes a managed table, and document why.
• Stand up orchestration from scratch; evaluate tooling and build idempotent, backfillable pipelines.
• Build reliable ingestion for drone imagery and telemetry captured in the field with validation and deduplication.
• Collaborate with AI engineers to turn raw captures into curated, versioned training datasets; own dataset lineage and versioning.
• Define data contracts, automated tests, and freshness and anomaly monitoring; ensure data quality across the pipeline.
• Manage S3 storage class lifecycle, file compaction, and Athena scan cost; optimize storage and layout for scale.
• Enable analytics: dbt, semantic layer for reporting as the platform matures.
• Write production-grade Python; embrace IaC, containers, CI/CD, and code reviews; raise data engineering standards.
• Communicate design decisions clearly to engineers and non-technical partners.

Skills

Python

AWS

SQL

Data modeling

Reproducibility

Data governance

Education

BSc/MSc in CS/CE/SE/Data Science

Tools

Airflow

Dagster

Prefect

Step Functions

dbt

MLflow

DVC

LakeFS

Terraform

Pulumi

Job description

Precision AI is hiring a senior data platform engineer in Calgary to own the data platform for AI training on drone imagery and telemetry. You’ll architect the lakehouse, set up ingestion pipelines, and define data contracts while collaborating with ML engineers to deliver curated, versioned training datasets.

This hybrid role sits at Precision AI’s Calgary HQ three days per week. You will drive data quality, reproducibility, and scalable cost controls, enabling faster model development and