Lead Data Platform Engineer

Black Dog Labs Limited; United States; Remote; USD 140,000 - 210,000; Full time

Benefits offered by this job

Remote contract

Flexible scheduling

Growth opportunities

Job summary

Black Dog Labs is seeking a Lead Data Engineer to drive the design, delivery, and operation of production-grade data platforms in a hands-on role. You will lead a small squad, guiding ingestion, transformation, orchestration, APIs, and observability across time zones.

The role is remote for at least 10 months with potential extension or conversion, requiring strong collaboration across international teams and time-zone coordination.

Qualifications

• Experience building and operating production data platforms.
• Ability to lead delivery and coach engineers.
• Experience with cloud infrastructure and IaC.
• Familiarity with data modeling and BI patterns.
• AWS or Azure experience preferred; knowledge of Kafka helpful.

Responsibilities

• Set technical direction, milestones, dependencies, risks, and ensure delivery.
• Define architectures, data contracts, make buy vs build decisions, RFCs, guardrails.
• Design and operate batch and change-driven pipelines with scheduling and orchestration.
• Build tested transformations and data models; optimize SQL and support analytics.
• Develop or integrate APIs with contracts, auth, validation, idempotency, auditing.
• Provision cloud resources with Terraform, automate CI/CD, instrument logs and metrics, SLOs.

Skills

Data engineering leadership

Leadership

Async communication

Cloud infrastructure

Terraform

CI/CD

Observability

SQL performance

APIs design

Tools

Azure

AWS

Kafka

Job description

Data Engineer India or Latin America preferred

Job details

India or Latin America preferred Eligibility

Lead Experience

Not specified Employment

About this role

Role overview

Lead the design, delivery, and operation of production-grade data platforms in a hands-on engineering role. You will guide a small squad or workstream while contributing directly across ingestion, transformation, orchestration, APIs, infrastructure automation, observability, and reliability. The engagement is remote, runs for at least 10 months with possible extension or conversion, and requires collaboration across time zones.

Responsibilities

• Set technical direction, define milestones, manage dependencies and risks, and keep delivery moving.
• Establish target architectures, data contracts, build-versus-buy decisions, RFCs, and cost, performance, and reliability guardrails.
• Design and operate batch and change-driven pipelines, including scheduling, orchestration, schema evolution, and failure handling.
• Build tested transformations and reusable data models; improve SQL performance and support analytics and product consumption.
• Develop or integrate APIs with clear contracts, authentication, validation, idempotency, and auditing.
• Provision cloud resources with Terraform, automate CI/CD, create operational scripts, and instrument logs, metrics, traces, and service-level objectives.

Requirements

• Strong experience building and operating production data platforms and reliable pipelines.
• Ability to lead delivery, conduct architecture and code reviews, coach engineers, and communicate clearly in async and live settings.
• Practical experience with cloud infrastructure, infrastructure as code, CI/CD, testing, and operational reliability.
• Familiarity with data modeling, BI consumption patterns, platform architecture, governance, cataloging, and lineage concepts.
• Experience with Azure is preferred; strong AWS experience is acceptable. Exposure to Kafka or other messaging systems, streaming, columnar formats, and cloud monitoring is valuable.
• Willingness to learn quickly and use AI-assisted coding tools responsibly while maintaining independent technical judgment.

Benefits and work setup

• Remote contract with flexible scheduling and agreed overlap for key meetings and on-call windows.
• Competitive compensation based on experience.
• Opportunities for technical growth and leadership development, with exposure to modern data-platform practices and production observability.