Senior Data Engineer – Azure Databricks & Pipelines Lead

Business Needs Inc; Austin (TX); Hybrid; USD 110,000 - 160,000; Full time

Job summary

Business Needs Inc. in Austin, TX is seeking a Data Engineer to design, develop, and maintain scalable data pipelines, ingestion frameworks, and reusable data products using Azure Databricks, PySpark, SQL, and Delta Lake.

You will implement Bronze, Silver, and Gold patterns, support batch and streaming workloads, lead technical reviews, mentor engineers, and collaborate with data architects to ensure secure, governed, cost-conscious solutions.

Qualifications

• Design, build, and maintain scalable data pipelines and products.
• Experience with Azure Databricks, PySpark, SQL, and Delta Lake.
• Lead reviews, mentor engineers, and promote coding standards.

Responsibilities

• Design, develop, and maintain scalable data pipelines and platforms.
• Implement Bronze/Silver/Gold data architecture patterns.
• Support batch, near-real-time, and streaming data workloads.

Skills

Data engineering

Data pipeline design

Mentoring

Observability

Tools

Azure Databricks

PySpark

SQL

Delta Lake

Azure DevOps

Git

Job description

Business Needs Inc. in Austin, TX is seeking a Data Engineer to design, develop, and maintain scalable data pipelines, ingestion frameworks, and reusable data products using Azure Databricks, PySpark, SQL, and Delta Lake.

You will implement Bronze, Silver, and Gold patterns, support batch and streaming workloads, lead technical reviews, mentor engineers, and collaborate with data architects to ensure secure, governed, cost-conscious solutions.