Senior Data Engineer

Accion Labs

United States

Remote

USD 120,000 - 170,000

Part time

18 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Accion Labs is seeking a senior Data Engineer for a fully remote, 12-month contract. The role focuses on building production ETL/ELT pipelines with Delta Lake and Databricks, administering Unity Catalog, and driving data governance in a regulated environment.

You will mentor junior engineers and collaborate with business stakeholders. The ideal candidate brings 7+ years in data engineering, deep Databricks expertise, and strong SQL/Python (PySpark) skills, with experience in CI/CD and TDD

Qualifications

  • Bachelor’s degree in Computer Science, Engineering, or related field or equivalent experience.
  • 7+ years of data engineering experience, including 3+ years on Databricks in production.
  • Deep expertise in Delta Lake, Unity Catalog, DLT/Lakeflow, and Workflows.
  • Advanced SQL and strong Python (PySpark) skills.
  • Experience owning a medallion/lakehouse architecture at scale.
  • Experience with data governance and access control in regulated environments (SOX, HIPAA, or similar).
  • Experience with CI/CD, Agile, and Test-Driven Development.
  • Strong communication skills to explain technical concepts to non-technical audiences.

Responsibilities

  • Build and optimize production ETL/ELT pipelines using Delta Lake, Delta Live Tables/Lakeflow, and Databricks Workflows.
  • Tune performance and manage costs, including cluster sizing, Photon, Liquid Clustering, Auto Loader, and DBU governance.
  • Administer Unity Catalog, including access control, lineage, and row/column-level security.
  • Maintain CI/CD for Databricks assets using Agile and TDD practices.
  • Implement data quality, monitoring, and alerting, and resolve production issues.
  • Support SOX compliance through access reviews, change control, and audit documentation.
  • Pilot Databricks AI/ML features such as MLflow, Feature Store, Model Serving, Genie, and vector search/RAG.
  • Mentor mid-level data engineers and partner with business and technical stakeholders.

Skills

SQL
Python PySpark
Delta Lake
Unity Catalog
Databricks
DLT Lakeflow
CI/CD
Agile
TDD
Data governance

Education

Bachelor's degree

Tools

Delta Live Tables
Azure cloud services
Lakehouse architecture

Job description

Fully Remote- Hiring for Data Engineer with strong Databricks, AI/ML and Data Governance experience - 12 months contract

Key Responsibilities
  • Build and optimize production ETL/ELT pipelines using Delta Lake, Delta Live Tables/Lakeflow, and Databricks Workflows
  • Tune performance and manage costs, including cluster sizing, Photon, Liquid Clustering, Auto Loader, and DBU governance
  • Administer Unity Catalog, including access control, lineage, and row/column-level security
  • Maintain CI/CD for Databricks assets (Asset Bundles, Git) using Agile and TDD practices
  • Implement data quality, monitoring, and alerting, and resolve production issues
  • Support SOX compliance through access reviews, change control, and audit documentation
  • Pilot Databricks AI/ML features such as MLflow, Feature Store, Model Serving, Genie, and vector search/RAG
  • Mentor mid-level data engineers and partner with business and technical stakeholders
Required Qualifications
  • Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent experience
  • 7+ years of data engineering experience, including 3+ years on Databricks in production
  • Deep expertise in Delta Lake, Unity Catalog, DLT/Lakeflow, and Workflows
  • Advanced SQL and strong Python (PySpark)
  • Experience owning a medallion/lakehouse architecture at scale
  • Experience with data governance and access control in a regulated environment (SOX, HIPAA, or similar)
  • Experience with CI/CD, Agile, and Test-Driven Development
  • Strong communication skills, with the ability to explain technical concepts to non-technical audiences
Preferred Qualifications
  • Databricks certifications (Data Engineer Professional, ML Associate/Professional)
  • Hands-on MLflow experience, plus exposure to Feature Store, Model Serving, Genie, or RAG
  • Azure cloud services (ADLS, Azure SQL, Synapse/Fabric); AWS/GCP is acceptable
  • Experience with Power BI/Fabric and middleware such as Boomi Scala
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer - Databricks
Data Engineer - Databricks

UNAVAILABLE • McLean (VA)

On-site
USD 120,000 - 170,000
Senior Databricks Engineer
Senior Databricks Engineer

Remote • San Francisco (CA)

On-site
USD 150,000 - 190,000
Senior Data Engineer - Databricks
Senior Data Engineer - Databricks

UNAVAILABLE • McLean (VA)

On-site
USD 120,000 - 160,000
Databricks Developer
Databricks Developer

Vservice It Solutions • United States

Remote
USD 140,000 - 190,000
Senior Data Engineer
Senior Data Engineer

SiloSmashers, Inc. • Alexandria (VA)

On-site
USD 140,000 - 155,000
Senior Data Engineer - Databricks
Senior Data Engineer - Databricks

DATAECONOMY Inc • New Jersey

On-site
USD 140,000 - 180,000
Senior Data Engineer - Databricks
Senior Data Engineer - Databricks

DATAECONOMY Inc • Raleigh (NC)

On-site
USD 120,000 - 190,000
Senior Databricks Data Engineer (Fully Remote)
Senior Databricks Data Engineer (Fully Remote)

Jobgether • United States

Remote
USD 110,000 - 160,000
Healthcare coverage
Meal vouchers
Life insurance
+4
Senior Databricks Data Engineer
Senior Databricks Data Engineer

Virtue America • New York (NY)

On-site
USD 140,000 - 190,000
Databricks Engineer
Databricks Engineer

Insight Global (United States) • United States

Remote
USD 140,000 - 190,000