Senior Staff Engineer-Data and AI Platform

H.E.B.; Austin (TX); On-site; USD 180,000 - 230,000; Full time

Job summary

H.E.B. is seeking an experienced Senior Staff Engineer – Data & AI Platform to architect and lead our enterprise-wide cloud data modernization and unified AI Platform strategy.

You will spearhead the migration of our data estate to Google Cloud Platform (GCP) and design standardized ingestion, transformation, and data serving frameworks for scalable ML workloads. You will also build developer tooling, drive IaC best practices, and lead the architecture of our next-generation AI and MLOps

Qualifications

• 10+ years in software/data/platform engineering with senior staff exposure.
• Proven success migrating large-scale cloud and data platforms.
• Deep GCP expertise with hands-on ML/AI platform experience.

Responsibilities

• Lead enterprise data modernization and unified AI platform strategy.
• Architect end-to-end data migration from AWS/Databricks to GCP.
• Design standardized ingestion, transformation, and serving frames.
• Build DevX tooling and CI/CD IaC for data/ML products.
• Mentor senior engineers and align roadmap with business goals.

Skills

GCP expertise

Python

SQL

Spark

dbt

Terraform

Kubernetes

Airflow

MLOps

RBAC

Education

Computer Science degree

Formal training or certifications

Tools

BigQuery

Dataflow

Dataproc

Airflow/Composer

Pub/Sub

GCS

Vertex AI

Kubeflow

MLflow

Feature Stores

Job description

Responsibilities

Job Summary: We are seeking an experienced Senior Staff Engineer – Data & AI Platform to architect and lead our enterprise-wide cloud data modernization and unified AI Platform strategy. In this pivotal technical leadership role, you will spearhead the migration of our data estate from AWS and Databricks to Google Cloud Platform (GCP). You will design and establish standardized ingestion, transformation, and data serving frameworks, build robust developer automation tooling, and lead the architecture of our next-generation enterprise AI and MLOps platform.

Key Responsibilities & Essential Functions:

1. Cloud Migration & Modernization Strategy

• Architect and lead the end-to-end migration of enterprise data workloads and storage from AWS and Databricks to a centralized, modernized GCP ecosystem.
• Establish dual-run strategies, data validation frameworks, and zero-downtime cutover methodologies to ensure business continuity.
• Design multi-tenant, secure, and cost-optimized cloud architectures leveraging BigQuery, Dataflow, Dataproc, and Cloud Storage.

2. Standardized Enterprise Data Frameworks

• Design, build, and evangelize reusable, metadata-driven ingestion, transformation, and serving frameworks.
• Standardize pipeline development with built-in data quality, lineage, automated schema evolution, and enterprise RBAC/governance.
• Create high-performance batch and real-time streaming architectures supporting mission-critical analytics and operational workloads.

3. Developer Experience (DevX) & Automation Tooling

• Build self-service SDKs, templates, and CLI tools that abstract cloud infrastructure complexities for platform consumers (DEs, BIs, DSs).
• Drive CI/CD automation and Infrastructure as Code (IaC) best practices to accelerate time-to-delivery for data and machine learning products.
• Enhance observability, monitoring, alerting, and cost-governance tooling across all data pipelines and AI workloads.

4. AI Platform & MLOps Infrastructure

• Architect and scale enterprise AI and MLOps platforms spanning traditional ML, Deep Learning, and Generative AI (LLMs).
• Implement robust model lifecycle infrastructure including feature stores, model registries, automated CI/CD for ML pipelines, and real-time/batch inference engines.
• Standardize LLMOps foundations, including vector database integration, retrieval-augmented generation (RAG) frameworks, model evaluation pipelines, and fine-tuning infrastructure.

5. Technical Leadership, Governance & Mentorship

• Set technical direction, architectural standards, and engineering guardrails across data and AI engineering teams.
• Mentor senior and staff engineers, fostering technical excellence, innovation, and cross-functional collaboration.
• Partner with Product, Security, Compliance, and Business Leadership to align platform roadmap priorities with core business outcomes.

Technical & Experience Qualifications

• 10+ years of software, data, or platform engineering experience, including 3+ years at the Staff or Senior Staff level.
• Proven success leading large-scale cloud and enterprise data platform migrations.
• Deep expertise in GCP, including BigQuery, Dataflow, Dataproc, Airflow/Composer, Pub/Sub, and GCS.
• Strong experience with Terraform, Kubernetes, and cloud-native infrastructure.
• Hands-on experience building production AI/ML and MLOps platforms using Vertex AI, Kubeflow, MLflow, Feature Stores, and related technologies.
• Knowledge of GenAI, LLMOps, vector databases, RAG, model monitoring, and AI governance.
• Expert proficiency in Python, SQL, Spark, Trino/Presto, and dbt.
• Strong background in distributed systems, data architecture, data modeling, and platform engineering.
• Experience implementing enterprise security, governance, RBAC/ABAC, lineage, auditing, and compliance frameworks.
• Exceptional communication, technical leadership, and stakeholder management skills.

Preferred Qualifications

• Experience with AWS, Databricks, Delta Lake, and Apache Iceberg.
• Experience delivering enterprise-scale cloud modernization and platform transformation initiatives.
• Hands-on experience with production Generative AI, LLM, RAG, and AI Agent solutions.
• Background in retail, supply chain, e-commerce, or other large-scale digital organizations.
• Experience building self-service data and AI platforms and mentoring senior engineering talent.

The responsibilities and essential functions outlined above describe the general nature and level of work assigned to this position. This is not an exhaustive list of all duties, responsibilities, and skills required. Duties and responsibilities may be modified at any time based on business needs. Employees may be required to perform other job-related tasks as requested by their supervisor, subject to reasonable accommodations.

Education:

• Computer Science degree or comparable formal training, certification, or work experience.

Physical Demands & Working Conditions:

• Travel by car or plane with overnight stays
• Work extended hours; sit for extended periods
• Work rotating and on-call schedules, as needed

The work environment characteristics described here are representative of those a Partner encounters while performing the essential functions of this job. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.

Last revised: 11/01/2024