Lead Data Engineer at V4C | Databricks Career Growth

V4C.ai; United States; Hybrid; USD 150,000 - 200,000; Full time

Benefits offered by this job

Unlimited PTO

Life Insurance

AD&D Insurance

Job summary

V4C.ai is seeking a senior Lead Data Engineer in the United States to guide production Lakehouse pipelines on client engagements. You will translate architectures into robust data flows, own design patterns, and drive quality in batch and streaming workloads with Databricks.

Expect to mentor engineers, collaborate with data architects, and communicate trade-offs to stakeholders while navigating a fast-paced consulting environment and travel as needed.

Qualifications

• 7+ years of data engineering and project/team leadership experience.
• Proficiency in Python, SQL, Spark, and Delta Lake.
• Production-grade batch and streaming pipelines on cloud platforms.

Responsibilities

• Lead engineering on customer engagements from design to production handover.
• Build and optimize batch and streaming pipelines with PySpark, SQL, and Delta Lake on Databricks.
• Implement Medallion architecture and set team patterns/standards.
• Own performance and cost: benchmark Spark jobs, tune SQL, and manage compute.
• Mentor mid-level and junior engineers through code review and design feedback.
• Collaborate with Data Architect on design decisions and client-facing explanations.

Skills

Python

SQL

Apache Spark

Delta Lake

Databricks

Leadership

Mentorship

Client communication

Tools

Unity Catalog

Lakeflow

Databricks Asset Bundles

dbt

Airflow

Job description

This is a strong career opportunity for a senior data engineer looking to deepen Databricks expertise within a fast-growing, niche consulting firm. V4C's rapid rise — 850+ certifications and Partner of the Year in under two years — signals strong execution and market traction, but also a high-intensity environment.

Compensation: No salary is listed, which is a yellow flag. For a Lead Data Engineer with 7+ years in the U.S. consulting market, market rates typically range from $150,000–$200,000+ annually depending on location and engagement model. Candidates should push for transparency during the screening process.

Skills You'll Build:

• Deep, production-validated Databricks expertise (Delta Lake, Lakeflow, Unity Catalog) — among the fastest-growing skills in modern data engineering
• Client-facing technical leadership — a rare and highly marketable hybrid skill set
• Mentorship and engineering team leadership experience

Green Flags: Structured certification roadmap, dedicated Employee Success Manager, Unlimited PTO, and clear ownership model at the engagement level are all positive signals. Healthcare + life sciences exposure (preferred) is a high-value specialization.

Red Flags to Watch: The consulting model means context-switching between clients, potential travel demands, and the pressure of real deadlines on external deliverables. Candidates should ask about average engagement length, bench policy, and work-hour expectations before accepting.

About V4C

V4C is a pure-play Databricks partner, and in under two years we have earned 850+ Databricks certifications across 325+ customers and been named Databricks' 2026 Partner of the Year (C&SI Emerging).

As a Lead Data Engineer, you will lead the build of production Lakehouse pipelines on customer engagements, working directly with customers and often Databricks' own field teams. Every technical hire here gets a certification roadmap built for them and taught in-house by Databricks-approved trainers, plus a dedicated Employee Success Manager whose only responsibility is your experience at V4C.

The Role

Lead Data Engineers are the technical backbone of V4C delivery. You lead the engineering workstream on an engagement — translating an architecture into working pipelines, setting the patterns the team follows, and owning the quality of what ships. You will work across batch and streaming, on messy enterprise source systems, under real deadlines, and in front of the customer explaining what you built and why. The difference from a Data Engineer is ownership: you are accountable for a workstream and the engineers on it.

Responsibilities

• Lead engineering on customer engagements, from design through production handover
• Build and optimize batch and streaming pipelines using PySpark, SQL, and Delta Lake on Databricks
• Implement Medallion architecture across cloud ecosystems and set the layer conventions the team follows
• Own performance and cost: benchmark Spark jobs, tune SQL, right-size compute, and explain the trade-offs to the customer
• Implement data quality and validation as code, including Lakeflow expectations and automated testing in CI
• Mentor mid-level and junior engineers through code review, pairing, and design feedback
• Partner with the Data Architect on design decisions, and escape heavy investment to the customer; accelerate remediation when implementation reveals the design will not hold

Requirements

• 7+ years of data engineering plus experience leading projects or teams
• Strong proficiency in Python, SQL, Apache Spark, and Delta Lake
• Production experience building both batch and streaming pipelines at enterprise scale
• Hands-on experience with AWS, Azure, or GCP and its data services
• Experience with data modeling, Git, and CI/CD for data workloads
• Demonstrated experience optimizing Spark performance and managing platform cost
• Strong client communication skills, including the ability to explain technical trade-offs to a non-technical stakeholder

Preferences

• Healthcare or life sciences experience preferred but not required
• Existing Databricks Certified Data Engineer Associate or Professional certification
• Hands-on experience with Unity Catalog, Lakeflow, or Databricks Asset Bundles
• Experience with dbt, Airflow, or equivalent orchestration and transformation tooling
• Experience migrating a legacy ETL estate to a modern cloud platform
• Consulting or client-facing engineering background
• Unlimited PTO
• Life Insurance (Basic, Voluntary & AD&D)

Is the Lead Data Engineer role at V4C fully remote? The job is listed for the United States without a specified city, suggesting location flexibility, but the role involves direct customer engagements and client-facing work which may require some on-site or travel commitments. Candidates should confirm the remote policy directly with V4C. What Databricks certifications are relevant for this role? V4C prefers candidates with existing Databricks Certified Data Engineer Associate or Professional certifications. For those without, V4C provides an in-house certification roadmap taught by Databricks-approved trainers. Is healthcare or life sciences experience required? No, healthcare or life sciences experience is listed as preferred but not required for this role. What is the seniority level and experience requirement for this position? This is a senior-level role requiring 7+ years of data engineering experience, plus demonstrated experience leading projects or engineering teams. Does V4C offer professional development support? Yes — every technical hire receives a personalized certification roadmap, in-house training by Databricks-approved trainers, and a dedicated Employee Success Manager focused solely on the employee's experience at V4C.

Remote work & career advice that actually works.