Lead Digital Data Engineer - AI & Databricks Lakehouse

BJ's Wholesale Club, Inc.; Marlborough (MA); Hybrid; USD 129,000 - 164,000; Full time

Benefits offered by this job

Weekly Pay

Daily Pay

Free BJ’s Memberships

Generous Paid Time Off

Flexible and Affordable HealthBenefits

401(k) Retirement Savings Plan

Employee Stock Purchase Plan

Job summary

BJ’s Wholesale Club is seeking a Lead Digital Data Engineer to own the design, build, and evolution of our Databricks Lakehouse. You will turn digital, fulfillment, and operational data into trusted, governed data products and drive AI-powered analytics adoption across the business.

You will lead a distributed team of up to 10 engineers, collaborate with product managers and senior leaders, and shape the data and analytics roadmap while optimizing performance and cost.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, or equivalent, plus 7+ years of experience in data engineering, including 3+ years of hands-on Databricks experience in production.
• Experience with Spark (PySpark/SQL), Delta Lake, Unity Catalog, and Databricks Workflows.
• Strong ability to build and maintain data pipelines across AWS services (S3, Kafka, DynamoDB, Lambda, IAM) and cloud platforms.

Responsibilities

• Design and maintain a scalable Lakehouse architecture (Delta Lake, Bronze/Silver/Gold) as single source of truth.
• Build batch and streaming pipelines integrating AWS services with Databricks.
• Ingest and unify data from digital and transactional sources; ensure data quality and governance.
• Lead AI/BI dashboards and Databricks agents use cases (Genie Spaces, Mosaic AI, MLflow).
• Oversee CI/CD for data pipelines (Git, Terraform) and drive Agile delivery with distributed teams.

Skills

Apache Spark

PySpark

SQL

Delta Lake

Unity Catalog

Databricks Workflows

SQL Warehouses

AWS S3

Kafka

DynamoDB

Lambda

IAM

Genie Spaces

AI dashboards

Mosaic AI

MLflow

CI/CD

Git workflows

Terraform

Agile Scrum

Data governance

Databricks agents

Education

Bachelor's degree in Computer Science, Information Technology, or equivalent

Tools

Databricks

Terraform

Git

Job description

BJ’s Wholesale Club is seeking a Lead Digital Data Engineer to own the design, build, and evolution of our Databricks Lakehouse. You will turn digital, fulfillment, and operational data into trusted, governed data products and drive AI-powered analytics adoption across the business.

You will lead a distributed team of up to 10 engineers, collaborate with product managers and senior leaders, and shape the data and analytics roadmap while optimizing performance and cost.