Lead Data Engineer: Cloud Architect & AI Pipelines

Texas Gas Transmission; Houston (TX); On-site; USD 140,000 - 190,000; Full time

Job summary

Boardwalk Pipelines, LP seeks a Lead Data Engineer to architect scalable data pipelines across cloud environments, driving Databricks and Lakehouse adoption in Houston. You will guide data integration, governance and analytics initiatives while mentoring engineers to raise team standards.

Required: 7–10 years in cloud data architecture with 5+ years AWS, plus strong SQL/Python and leadership abilities. This role emphasizes collaboration and delivering business value through data.

Qualifications

• 7–10 years of experience in cloud data architecture; at least 5 years in AWS.
• Proficiency with AWS services: Glue, Redshift, Athena, Lake Formation, SageMaker, Bedrock, Step Functions.
• Extensive experience with Databricks; strong data governance tooling (Alation).
• Strong SQL (T-SQL/PL-SQL) and Python (PySpark/Pandas) skills; knowledge of DAX and M.
• Experience with SQL Server, Oracle, PostgreSQL, Azure SQL, Teradata.
• Proficiency with Power BI, Git, VS Code, SSMS.
• Familiarity with Medallion architecture, data mesh, Lakehouse concepts.
• Excellent communication, collaboration, and leadership abilities.

Responsibilities

• Architect and deliver scalable data pipelines and platforms in cloud environments.
• Lead the adoption of Lakehouse and Databricks architectures.
• Strengthen data integration, governance, and security across systems.
• Support analytics and AI/ML use cases; mentor engineers and analysts.
• Collaborate across teams to drive business value through data.

Skills

Cloud data architecture

Leadership

Communication

Mentoring

Data governance

Collaboration

Analytics/ML

SQL

Python

Education

Bachelor’s degree in CS/DS/Engineering

Masters degree (preferred)

Tools

Databricks

AWS

Alation

SQL Server

Oracle

PostgreSQL

Azure SQL

SSMS

Git

Visual Studio Code

Power BI

Job description

Boardwalk Pipelines, LP seeks a Lead Data Engineer to architect scalable data pipelines across cloud environments, driving Databricks and Lakehouse adoption in Houston. You will guide data integration, governance and analytics initiatives while mentoring engineers to raise team standards.

Required: 7–10 years in cloud data architecture with 5+ years AWS, plus strong SQL/Python and leadership abilities. This role emphasizes collaboration and delivering business value through data.