Sr. BI Lead Databricks Engineer

Stefanini, Inc; Humble, Northern (TX, KY); Hybrid; USD 110,000 - 117,000; Full time

Job summary

Stefanini Group is seeking a Sr. BI Lead Databricks Engineer to serve as technical authority for BI and data platform initiatives, combining Databricks engineering with Power BI development and offshore leadership.

You will own the architecture, governance, and delivery roadmap for Databricks Lakehouse and Power BI, guiding an offshore team to ensure quality and on-time delivery. Remote work options are available for qualified candidates with strong SQL, PySpark, and cloud data services.

Qualifications

• Minimum 5 years of hands-on data engineering experience with production Databricks.
• Expert proficiency in PySpark, Spark SQL, and Delta Lake.
• 5+ years Power BI development and enterprise report design.

Responsibilities

• Lead Databricks platform ownership and governance.
• Architect scalable data pipelines and Medallion architecture.
• Oversee offshore BI developers and delivery quality.

Skills

Databricks engineering

Power BI development

Team leadership

Education

Bachelor's degree in Computer Science or related field

Tools

PySpark

Delta Lake

Azure DevOps

Power BI

Job description

Job Details

Job ID#: 64813

Job Category: BI Analyst

Position Type: Full Time

Duration: 1 year

Shift: 1

Remaining Positions: 1

Posted: 9/3/2026

Location: Humble, TX

Stefanini Groupis looking for Sr. BI Lead Databricks Engineer for a globally recognized company!

POSITION OVERVIEW

The Senior BI Lead & Databricks Engineer is responsible for serving as the technical authority and team leader for Business Intelligence and data platform initiatives. This is a dual-function role combining expert-level Databricks platform engineering with senior Power BI development and offshore BI team leadership. The role owns the architecture, governance, and delivery roadmap for both the Databricks Lakehouse environment and the Power BI reporting ecosystem, while guiding an offshore team of BI developers to ensure consistent quality and on-time delivery.

PRINCIPAL RESPONSIBILITIES AND DUTIES

Databricks Platform Ownership

• Serve as the primary subject matter expert (SME) and internal champion for Databricks across the organization.
• Architect, build, and maintain scalable data pipelines using PySpark, Delta Lake, and Databricks Workflows.
• Implement and enforce the Medallion architecture (Bronze / Silver / Gold) for structured data layering.
• Manage Databricks Unity Catalog for data governance, access controls, and data lineage tracking.
• Optimize cluster configurations, job scheduling, and compute costs to ensure platform efficiency.
• Establish CI/CD best practices for Databricks notebooks and jobs using Azure DevOps or equivalent tooling.
• Partner with data engineers, architects, and business stakeholders to define data contracts and SLAs.
• Monitor pipeline health, data quality, and platform performance; resolve issues proactively.

Power BI Leadership & Development

• Lead the design and delivery of enterprise Power BI reports, dashboards, and semantic data models.
• Define and enforce Power BI development standards including naming conventions, DAX best practices, and deployment pipelines.
• Architect and manage the Power BI semantic layer (datasets and dataflows) connected to Databricks as the source of record.
• Review and approve all BI deliverables produced by the offshore development team prior to release.
• Provide hands-on development for complex or high-priority reports and datasets as needed.
• Manage the Power BI Premium or Fabric workspace environment, capacity planning, and licensing.
• Collaborate with business stakeholders to gather requirements, prioritize the BI backlog, and deliver actionable insights.

Offshore Team Leadership

• Provide day-to-day technical direction, task assignment, and code review for offshore BI developers.
• Establish and communicate sprint-level priorities, delivery timelines, and quality standards.
• Conduct regular team check-ins to remove blockers, ensure alignment, and support developer growth.
• Create and maintain BI development documentation, standards templates, and reusable component libraries.
• Onboard new team members and establish repeatable, documented development processes.
• Escalate capacity, quality, or scope issues to leadership proactively and with recommended solutions.

Data Strategy & Stakeholder Engagement

• Act as a strategic advisor to business leaders on data visualization, self-service analytics, and data-driven decision making.
• Participate in roadmap planning for the overall data platform and BI tool ecosystem.
• Translate complex technical concepts into clear, business-facing communication for non-technical audiences.
• Identify and evaluate opportunities to leverage AI/ML capabilities within Databricks and Microsoft Fabric.
• Perform other related duties or special projects as assigned or directed.

EXPERIENCE AND OTHER REQUIREMENTS

• Minimum 5 years of hands-on data engineering experience, with 3 or more years working in Databricks at production scale.
• Expert proficiency in PySpark, Spark SQL, and Delta Lake including schema evolution, file optimization, and VACUUM operations.
• Deep working knowledge of Medallion / Lakehouse architecture patterns and best practices.
• Minimum 5 years of Power BI development experience including complex DAX, data modeling, and enterprise report design.
• Experience with Power BI Premium, Deployment Pipelines, and workspace administration.
• Proven ability to architect and manage enterprise-grade semantic models and datasets connected to cloud data platforms.
• Hands-on experience with Azure Data Lake Storage, Azure Data Factory, or equivalent cloud data services.
• Strong SQL skills including complex joins, CTEs, window functions, and query performance tuning.
• Experience with CI/CD pipelines and version control (Azure DevOps, Git) for data and BI workflows.
• 2 or more years of experience leading, mentoring, or managing BI developers; offshore team management experience strongly preferred.
• Demonstrated ability to manage workload, quality standards, and delivery timelines across distributed teams.
• Excellent written and verbal communication skills with the ability to present technical findings to non-technical stakeholders.
• Ability to manage competing priorities and deliver consistent results in a fast-paced, remote environment.

EDUCATION / CERTIFICATION

• Bachelor's degree in Computer Science, Information Systems, Data Engineering, or a related field preferred; equivalent professional experience considered.
• Databricks Certified Associate or Professional certification (Data Engineer or Data Analyst track) preferred; willingness to obtain certification if not already held.
• Microsoft Certified: Power BI Data Analyst Associate (PL-300) preferred; willingness to obtain certification if not already held.

#LI-AH1 #LI-REMOTE

Pay Range: $ 80.00 - $ 85.00

Already have an account? Log in here