Analyst, Lead Data Engineering

Enterprise Products; Houston (TX); On-site; USD 180,000 - 240,000; Full time

Job summary

Info Tech in Houston, TX is seeking an experienced Senior Software Engineer to lead the design and delivery of data pipelines and AI-enabled platforms. You will own end‑to‑end development for batch and streaming systems, data catalogs, and GenAI applications, collaborating with stakeholders across technical and business teams.

The role demands deep Python expertise, extensive AWS knowledge, and strong experience in MLOps and AI development workflows.

Qualifications

• 10+ years of software engineering experience with leadership exposure
• Expert level Python with strong software engineering fundamentals
• Experience building data products with Snowflake, DBT, and Airflow
• Delivery of classical ML and GenAI systems to production incl. MLOps
• Deep AWS experience building data and AI platforms (S3, Glue, Athena, SageMaker, Bedrock)
• Experience with Infrastructure as Code (Terraform, AWS CDK, CloudFormation) and CI/CD
• Practical AI coding agent workflows (Claude Code / Cursor)
• Ability to communicate with technical and non-technical stakeholders

Responsibilities

• Design and build reliable batch and streaming pipelines
• Establish data modeling, quality, lineage, and cataloging for data products
• Build feature pipelines and curated datasets for analytics and GenAI use cases
• Productionize ML and GenAI systems with MLOps practices
• Develop GenAI applications like retrieval augmented generation over enterprise docs
• Implement guardrails, observability, and cost controls for LLM-based systems
• Define evaluation approaches for AI quality and regressions
• Create self-service tooling and CI/CD pipelines for data/AI products on AWS and on-prem
• Own infrastructure as code and platform reliability
• Embed security, governance, and compliance by design
• Mentor junior engineers in design and reviews

Skills

Python

Data products

Snowflake

DBT

Airflow

AWS

MLOps

Terraform

AWS CDK

CloudFormation

Tools

Claude Code

Cursor

SageMaker

Bedrock

Lambda

ECS

RDS

IAM

VPC

S3

Job description

Primary Location

USA-Texas-Houston

What you will be doing

Enterprise Products Partners L.P. is one of the largest publicly traded partnerships and a leading North American provider of midstream energy services to producers and consumers of natural gas, NGLs, crude oil, refined products and petrochemicals. Our services include natural gas gathering, treating, processing, transportation and storage; NGL transportation, fractionation, storage and import and export terminals; crude oil gathering, transportation, storage and terminals; petrochemical and refined products transportation, storage and terminals; and a marine transportation business that operates primarily on the United States inland and Intracoastal Waterway systems. The partnership’s assets include approximately 50,000 miles of pipelines; 260 million barrels of storage capacity for NGLs, crude oil, refined products and petrochemicals; and 14 billion cubic feet of natural gas storage capacity.

Responsibilities include, but are not limited to

• Design and build reliable batch and streaming pipelines, including ingestion of industrial time-series and operational data alongside unstructured document sources
• Establish data modeling, quality, lineage, and cataloging practices for data product development
• Build feature pipelines and curated datasets that serve analytics, ML, and GenAI use cases
• Productionize classical ML models following MLOps practices
• Build GenAI applications such as retrieval-augmented generation over enterprise documents, including chunking, embeddings, vector search, prompt management, and evaluation
• Implement guardrails, observability, and cost controls for LLM-based systems
• Define evaluation approaches that make AI quality measurable and regressions visible
• Build self-service tooling, reusable templates, and CI/CD pipelines so teams can ship data and AI products consistently on AWS and on-premises environments
• Own infrastructure as code, observability, and reliability practices for the platform
• Embed security, governance, and compliance requirements into the platform by design
• Define and implement best practices around AI Assisted Software Development Life Cycle
• Mentor junior engineers in design and code reviews and implementation best practices

What Your Background Should Look Like (minimum Qualifications)

• 10+ years of software engineering experience, with at least 2 years in a technical lead role
• Expert level experience in Python with solid software engineering fundamentals including testing, code reviews, branching policies, version control, design patterns, and API design
• Experience building data products leveraging Snowflake, DBT, and Airflow
• Experience delivering classical ML and GenAI systems to production, including MLOps practices such as model registries, pipeline automation, and monitoring
• Deep AWS experience building data and AI platforms including S3 Tables, Glue, Athena, Lake Formation, SageMaker, Bedrock, Lambda, ECS, RDS, IAM, and VPC
• Experience in designing and implementing Infrastructure as Code (IaC) using Terraform, AWS CDK, or CloudFormation along with deployment automation with CI/CD pipelines
• Practical experience with AI-assisted software development workflows using tools such as Claude Code or Cursor to design, implement, test, and refactor code
• Experience in structuring work for AI coding agents such as writing clear specifications, providing context, breaking tasks down, and reviewing generated output critically
• Clear communication skills and the ability to work with both technical and non-technical stakeholders

Shift

Day Job

Travel

Yes, 10 % of the Time

Unposting Date

Ongoing

Organization

Info Tech

Job

Information Technology

Job Level

Individual Contributor