Senior Data Engineer (Contract)

coconutsoftware; United States; Remote; USD 63,000 - 91,000; Full time

Benefits offered by this job

Health and dental coverage

Virtual care

Disability coverage

Mental health support

Retirement savings matching

Annual wellness allowance

Internet subsidy

Incentive programs

Job summary

coconutsoftware engineers a modern analytics platform by owning data pipelines for workforce forecasting and staffing products. This 12-month contract role operates remotely from Canada, with responsibilities spanning ingestion, transformation, migration, reliability, and comprehensive documentation.

You will collaborate with analytics engineers and product teams, lead migrations from Redshift to Databricks, manage data replication, and ensure data contracts protect schema evolution while

Qualifications

• Five or more years in data engineering, including production Databricks with Delta, Jobs, DLT or Lakeflow, and Unity Catalog.

Responsibilities

• Build and operate production pipelines in Databricks, from customer file ingestion through curated tables for models and reporting, using tools such as Lakeflow, Jobs, and dbt.
• Lead further migration from Redshift to Databricks, including source table landing, product analytics migration, and retirement of legacy warehouse components.
• Manage replication and ingestion from production databases and files across environments, accounting for data residency and personally identifiable information.
• Agree on data contracts with internal teams and external machine learning partners to manage schema changes safely.
• Monitor pipeline health, data quality, and platform costs; support the team’s on-call rotation.
• Estimate effort, explain trade-offs, document designs and runbooks, and help improve engineering practices, including thoughtful use of AI tools.

Skills

Python

SQL

dbt

Databricks

Delta Lake

Unity Catalog

AWS data services

Change data capture

Tools

Lakeflow

Jobs

dbt

Databricks

Delta

Unity Catalog

Terraform

Databricks Asset Bundles

LookML

Looker

AI coding agents

Job description

Role overview

Own data engineering for a team modernizing its analytics platform and building data pipelines for workforce forecasting and staffing products. Over a 12-month contract, the senior engineer will work independently across ingestion, transformation, migration, reliability, and documentation, partnering with analytics engineers, product teams, and other technical groups. The position is remote within Canada and includes participation in an on-call rotation.

Responsibilities

• Build and operate production pipelines in Databricks, from customer file ingestion through curated tables for models and reporting, using tools such as Lakeflow, Jobs, and dbt.
• Lead further migration from Redshift to Databricks, including source table landing, product analytics migration, and retirement of legacy warehouse components.
• Manage replication and ingestion from production databases and files across environments, accounting for data residency and personally identifiable information.
• Agree on data contracts with internal teams and external machine learning partners to manage schema changes safely.
• Monitor pipeline health, data quality, and platform costs; support the team’s on-call rotation.
• Estimate effort, explain trade-offs, document designs and runbooks, and help improve engineering practices, including thoughtful use of AI tools.

Requirements

• Five or more years in data engineering, including at least two years operating Databricks in production with Delta, Jobs, DLT or Lakeflow, and Unity Catalog.
• Strong Python, SQL, and dbt skills, plus comfort with AWS data services.
• Experience replicating operational data through change data capture or events and ingesting imperfect files.
• Participation in at least one data warehouse or lakehouse migration.
• Experience handling personally identifiable or regulated data; financial services experience is particularly relevant.
• Ability to work autonomously, take initiative, lead projects, and focus effort on high-priority work.

Nice to have

• Infrastructure-as-code experience such as Terraform or Databricks Asset Bundles.
• Hands-on use of AI coding agents for data pipelines, dbt, debugging, or troubleshooting.
• LookML or embedded Looker experience, and pipelines that feed forecasting or machine learning models.

Benefits and work setup

• Twelve-month contract, remote anywhere in Canada.
• Listed benefits include health and dental coverage, virtual care, disability coverage, mental health support, retirement savings matching, an annual wellness allowance, internet subsidy, and incentive programs.