JR-227337 Data Engineer Lead

Ctdev

San Mateo (CA)

On-site

USD 150,000 - 190,000

Full time

21 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Customertimes is seeking a Lead Databricks Engineer to drive design, development, and optimization of data pipelines and analytics on the Databricks Lakehouse. This hands-on leader will shape scalable data architectures and data products across global teams.

You will mentor engineers, enforce coding standards, collaborate with data scientists, and ensure secure, compliant deployments across AWS, Azure, and GCP using Terraform and Databricks CLI.

Qualifications

  • BS/MS in CS/IT/Engineering or related field.
  • 5+ years in Data Engineering or Big Data.
  • 2+ years with Databricks, Spark, Delta Lake.
  • Strong cloud knowledge (AWS/Azure/GCP) and Lakehouse/Data Mesh.
  • Proficiency in SQL and Python.
  • Experience with dbt.
  • CI/CD, Git, DataOps.
  • Understanding of data governance and security.
  • Experience leading teams or projects.

Responsibilities

  • Lead end-to-end data pipeline development on Databricks.
  • Design Lakehouse architectures aligned to requirements.
  • Collaborate with data scientists, analysts, and engineers.
  • Define coding standards and performance tuning.
  • Optimize Delta Lake data models and Unity Catalog.
  • Integrate data sources across AWS/Azure/GCP.
  • Lead architecture reviews and mentor engineers.
  • Automate infrastructure deployment with Terraform/Databricks CLI.
  • Ensure security, compliance, and scalability.

Skills

Databricks
Spark
Delta Lake
SQL
Python
Cloud Platforms
dbt
Git
CI/CD
Data Governance
Terraform
Unity Catalog
MLOps

Education

Bachelor's or Master's in CS/IT/Engineering or related

Tools

Databricks
Spark
Python
Terraform
Databricks CLI

Job description

Hello, our name is Customertimes, but you can call us CT. We are a global digital engineering, product development, and technology consulting company. Headquartered in New York, we have a team of 1000+ experts working across 65+ countries.

About this job:

We are seeking a Lead Databricks Engineer to drive the design, development, and optimization of data pipelines and analytics solutions on the Databricks Lakehouse platform. This role is ideal for a hands-on technical leader who is passionate about big data technologies, cloud computing, and enabling business insights through scalable data architectures.

How you'll make an impact:
  • Lead end-to-end development of data pipelines, ETL/ELT processes, and batch/streaming solutions using Databricks and Apache Spark;
  • Design and implement Lakehouse architectures that align with business and technical requirements;
  • Collaborate with data scientists, analysts, and engineers to deliver high-performance data products and ML features;
  • Define and enforce coding standards, best practices, and performance tuning strategies across Databricks notebooks and jobs;
  • Optimize data models in Delta Lake and implement data governance standards using Unity Catalog;
  • Manage integration of data sources across cloud platforms (e.g., AWS, Azure, GCP) using native and third-party connectors;
  • Contribute to and lead technical reviews, architecture sessions, and mentoring of less experienced engineers;
  • Automate infrastructure deployment with tools like Terraform, Databricks CLI, or others;
  • Ensure data platform solutions are secure, compliant, and scalable across global business units.
What makes you a great fit:
  • Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field;
  • 5+ years of experience in Data Engineering or Big Data roles;
  • 2+ years of hands‑on experience with Databricks, Spark (PySpark or Scala), and Delta Lake;
  • Strong knowledge of cloud platforms (AWS, Azure, or GCP) and modern data architectures (Lakehouse, Data Mesh, etc.);
  • Proficiency in SQL, Python, and distributed data processing;
  • Experience with dbt;
  • Experience with CI/CD, version control (Git), and DataOps in a data environment;
  • Deep understanding of data governance, cataloging, and security concepts;
  • Experience leading a team or a project, acting as a team/tech lead of a project.
You are a star if you have:
  • Databricks (Pro, Architect) certification;
  • Experience with machine learning pipelines and MLOps in Databricks.

Other skills:

  • English – excellent written and verbal communication skills;
  • Ability to work in a global company with many cultures and nationalities;
  • Ability to lead conversations with both technical and business representatives;
  • Proven ability to work both independently and as part of an international project team.

If you're looking to contribute to global projects, grow your skills, shape the way you work, and be part of a connected international community — Customertimes is the place to build your next career chapter. Are you ready to spark the tech with us?

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

JR-226355 Data Engineer Senior
JR-226355 Data Engineer Senior

Ctdev • United States

Hybrid
USD 120,000 - 180,000
JR-189588 Project Manager Senior
JR-189588 Project Manager Senior

Customertimes • United States

On-site
USD 130,000 - 190,000
Data Team Lead
Data Team Lead

Incedo Inc. • San Rafael (CA)

Hybrid
USD 130,000 - 150,000
Medical insurance
Vision insurance
401(k)
Databricks Data Engineer
Databricks Data Engineer

Volto Consulting • Irving (TX)

On-site
USD 120,000 - 150,000
Lead Software Engineer - Databricks (Corporate Technology)
Lead Software Engineer - Databricks (Corporate Technology)

ARCHER • Jersey City (NJ)

On-site
USD 140,000 - 210,000
Health Care Coverage
On-Site Health And Wellness Centers
Retirement Savings Plan
+4
Lead Databricks Data Engineer
Lead Databricks Data Engineer

EXL Talent Acquisition Team • New York (NY)

On-site
USD 180,000 - 240,000
Databricks Lead
Databricks Lead

Tata Consultancy Services Limited • Minneapolis (MN)

On-site
USD 130,000 - 150,000
Discretionary Annual Incentive
Comprehensive Medical Coverage: Health
Family Support: Parental Leaves
+3
Staff Fullstack Engineer, Agentic Applications
Staff Fullstack Engineer, Agentic Applications

Neon • Mountain View (CA)

On-site
USD 192,000 - 260,000
Comprehensive health coverage
401(k) Plan
Equity awards
+3
Sr. Solutions Engineer - AI Natives Business
Sr. Solutions Engineer - AI Natives Business

Neon • United States

Remote
USD 152,000 - 209,000
Sr. Staff Fullstack Engineer, Agentic Applications
Sr. Staff Fullstack Engineer, Agentic Applications

Databricks Inc. • Mountain View (CA)

On-site
USD 192,000 - 260,000
Comprehensive health coverage
401(k) Plan
Equity awards
+6