Senior Data Engineer, Databricks

GAP

United States

Remote

USD 140,000 - 200,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Growth Acceleration Partners seeks a Senior Databricks Data Engineer to design, build, and maintain scalable data pipelines within an Azure/Databricks environment. You will own data modeling, pipeline orchestration, and data quality while partnering with application developers and business stakeholders.

You will operate with autonomy, applying PySpark, SQL, Unity Catalog, and governance practices to deliver secure and governed data for critical enterprise applications.

Qualifications

  • Bachelor’s degree in computer science or related technical field.
  • 5+ years of professional experience in data engineering.
  • Proven hands-on experience with Databricks, PySpark, SQL, and Delta Lake.

Responsibilities

  • Architect, build, and maintain scalable Databricks data pipelines to support existing systems and new application integrations.
  • Design data models that integrate Databricks with Azure SQL and Azure Cosmos DB.
  • Orchestrate data pipelines and schedule workflows using Lakeflow Jobs.
  • Enforce rigorous data quality, testing, and monitoring standards across data engineering processes.
  • Manage deployment lifecycles with Databricks Asset Bundles and GitHub CI/CD workflows.
  • Collaborate with frontend/backend developers and Product Owners to ensure data infrastructure meets needs.
  • Operate within security and governance models using Unity Catalog and Entra ID.

Skills

Databricks
PySpark
SQL
Delta Lake
Unity Catalog
Lakeflow Jobs
GitHub CI/CD
Data modeling

Education

Bachelor’s Degree in Computer Science or related technical field

Tools

Databricks
Unity Catalog
Lakeflow
GitHub
Delta Lake
Azure SQL
Azure Cosmos DB

Job description

WHAT WE DO

Founded in 2007, Growth Acceleration Partners (GAP) is a consulting and technology services company. We consult, design, build and modernize revenue-generating software and data engineering solutions for clients. With modernization services and AI tools, we help businesses achieve a competitive advantage through technology. GAP’s remote, integrated engineering teams use end-to-end solutions to innovate and align with your business goals. We have 600+ English-speaking engineers based in Latin America and approximately 20 U.S.-based engineers. With some of the highest customer satisfaction scores in the industry, GAP’s focus is customer and employee success. GAP is a woman-owned company headquartered in Austin Texas. We are a values-based company focused on growing our people by investing in education, onsite English classes and training in the latest technologies, including AI, data analytics and machine learning. Our goal is to provide solutions for our customers that help them achieve critical business outcomes, while enabling our GAPSters and our communities to attain long‑term success.

Summary

We are looking for a Senior Databricks Data Engineer to design, build, and maintain scalable data pipelines and integrations within a robust Azure and Databricks ecosystem. In this high‑impact role, you will work as part of a dedicated team to deliver large, defined projects, seamlessly integrating Databricks with modern application stacks. You will take technical ownership of data modeling, pipeline orchestration, and data quality while collaborating closely with cross‑functional application developers and business stakeholders. Operating with a high level of autonomy, you will leverage PySpark, SQL, and Unity Catalog to ensure secure, performant, and governed data delivery for critical enterprise applications.

Education

Bachelor’s Degree in Computer Science, Data Engineering, Software Engineering, Information Systems, or a related technical field.

Professional Experience

5+ years of professional experience in Data Engineering, specializing in modern cloud data architectures. Proven hands‑on experience building and maintaining complex data pipelines on Databricks using PySpark and SQL. Deep expertise working with Delta Lake architectures and implementing governance via Unity Catalog. Demonstrated experience integrating Databricks workflows with Azure SQL or Azure Cosmos DB. Experience operating in Git‑based CI/CD environments, specifically managing all work in GitHub where every pull request is linked to a GitHub Issue.

Key Responsibilities
  • Architect, build, and maintain scalable Databricks data pipelines to support existing systems and new application integrations.
  • Design and implement robust data models that integrate seamlessly between Databricks and transactional databases like Azure SQL or Azure Cosmos DB.
  • Orchestrate data pipelines and schedule workflows utilizing Lakeflow Jobs.
  • Enforce rigorous data quality, testing, and monitoring standards across all data engineering processes.
  • Manage deployment lifecycles using Databricks Asset Bundles and Git‑based CI/CD workflows.
  • Collaborate continuously with frontend/backend application developers (React, C# .NET) and Product Owners to ensure data infrastructure meets application requirements.
  • Operate strictly within organizational security and governance models, leveraging Microsoft Entra ID for access and Unity Catalog for data governance.
Required Technical Skills
  • Strong experience in several of the following areas:
  • Data Engineering & Processing: Deep proficiency in Databricks, PySpark, advanced SQL, and Delta Lake.
  • Governance & Orchestration: Unity Catalog (grants, lineage, access control) and Lakeflow Jobs.
  • Cloud & Databases: Seamless integration with Azure SQL and Azure Cosmos DB.
  • DevOps & CI/CD: Databricks Asset Bundles, Git/GitHub workflows (linking PRs to Issues), and automated deployment strategies.
  • Testing & Quality: Proven methodologies for data quality assurance, automated testing, and pipeline monitoring.
Nice to Have
  • Experience with Databricks AI services such as Model Serving, Mosaic AI Agents, Vector Search, or Genie Spaces.
  • Familiarity with Python API development using FastAPI and Pydantic.
  • Exposure to financial domain data, such as Concur expense data, 73Strings data, or fund accounting analytics.
  • Experience with observability tools, including OpenTelemetry logs, spans, and metrics.
Soft Skills
  • Advanced English proficiency (written and verbal).
  • Strong communication skills for working directly with business users and translating business needs into technical solutions.
  • Ability to ramp up quickly on existing codebases and shared enterprise packages.
  • Highly collaborative mindset, capable of integrating smoothly with established development teams and processes.

At Growth Acceleration Partners, we're an equal opportunity employer committed to building a diverse and inclusive team. We value everyone's unique background, and we provide equal opportunities regardless of race, color, creed, religion, sexual orientation, gender identity, age, national origin, disability, marital status, veteran status or any other personal right protected by law. We foster a culture of belonging and strive to provide a welcoming environment where everyone feels safe to contribute and grow.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Databricks Data Engineer
Senior Databricks Data Engineer

GAP Careers • Austin (TX)

On-site
USD 140,000 - 200,000
Senior Data Engineer - Databricks
Senior Data Engineer - Databricks

UNAVAILABLE • McLean (VA)

On-site
USD 120,000 - 160,000
Senior Databricks Data Engineer - Remote & Azure
Senior Databricks Data Engineer - Remote & Azure

GAP Careers • Austin (TX)

On-site
USD 140,000 - 200,000
Databricks Data Engineer
Databricks Data Engineer

Quantum Technologies LLC • Dallas (TX)

On-site
USD 150,000 - 190,000
Data Engineer - Databricks
Data Engineer - Databricks

UNAVAILABLE • McLean (VA)

On-site
USD 120,000 - 170,000
Senior Data Engineer, Azure Databricks
Senior Data Engineer, Azure Databricks

Nextech Solutions • United States

Remote
USD 130,000 - 170,000
Data Engineer, Principal
Data Engineer, Principal

Horizontal Talent • Golden Valley (MN)

On-site
USD 68,000 - 117,000
Medical, dental, vision benefits
Retirement plan
Databricks Engineer
Databricks Engineer

Dataflix • United States

Remote
USD 130,000 - 170,000
Databricks Data Architect
Databricks Data Architect

NewVision Software • United States

Remote
USD 120,000 - 180,000
Databricks Developer
Databricks Developer

Vservice It Solutions • United States

Remote
USD 140,000 - 190,000