Senior Data Engineer, Customer & Identity Resolution

HEB; Austin (TX); On-site; USD 140,000 - 180,000; Full time

Job summary

HEB in Austin, TX is seeking a Senior Data Engineer to strengthen customer and identity resolution through reliable data patterns, pipelines, and integrations. You will design scalable batch and real-time solutions and mentor junior engineers while partnering with Product, Data Science, Application, and Analytics teams.

You will validate data quality, build data integrations and APIs, and contribute to architecture, security, scalability, reliability, and performance across the data platform.

Qualifications

• Bachelor's degree in CS or related field.
• 5+ years of data engineering experience.
• Experience with Lean Startup or Agile development methodologies.
• Experience with large-scale infrastructure, large data sets, and mission-critical SLAs.
• High proficiency in Python, PySpark, and SQL for data manipulation and pipeline development.

Responsibilities

• Design data patterns for analytics datasets, including cleansing and mapping across sources.
• Validate data quality and write automated tests.
• Maintain end-to-end data pipelines and build advanced pipelines and APIs.
• Collaborate with product, data science, and analytics teams to deliver solutions.
• Mentor junior engineers and contribute to architecture, security, reliability, and performance.
• Design and develop data integrations for application engineering and system integration.

Skills

Agile methodologies

Mentoring

Communication

Data modeling

Learning new tech

Education

Bachelor's degree in CS or related field

Tools

Python

PySpark

SQL

Google Cloud Platform (GCP)

Dataproc

Dataflow

Airflow (Cloud Composer)

Cloud Storage / Pub/Sub

Databricks

BigQuery

FastAPI & REST APIs

ETL/ELT

CI/CD & Git

Job description

HEB is seeking a Senior Data Engineer to strengthen customer and identity resolution through reliable data patterns, pipelines, and integrations. In this onsite role in Austin, you will build and maintain end-to-end data infrastructure that supports analytics needs while partnering with Product, Data Science, Application, and Analytics teams.

You will design scalable batch and real-time solutions, validate data quality, and help translate upstream sources into capabilities the business can use. The role also includes mentoring junior engineers and contributing to architecture, security, scalability, reliability, and performance.

What you'll do

• Design data patterns that enable analytics datasets, including calculations, cleansing, standardization, and mapping or linking data across multiple sources.
• Perform data validation and quality assurance on work from junior peers, including writing automated tests.
• Maintain and streamline existing data pipelines end to end.
• Build and support more advanced pipelines, application programming interfaces (APIs), data integrations, streaming solutions, and predictive model implementations (quality, velocity, and related measures).
• Identify complex data from upstream sources to support new capabilities.
• Test technical solutions to ensure data integrity and system functionality, including end-to-end coverage for your work and broader system behavior.
• Create and maintain large-scale batch and real-time pipelines using big data processing frameworks.
• Design and develop data integrations for application engineering and system integration.
• Design, develop, and maintain large data pipelines, diagnosing and solving production support issues when needed.
• Implement monitoring capabilities based on business SLAs and data quality expectations.
• Contribute to refinement of Digital Engineering tools, standards, and training for the team.
• Collaborate with external technical teams and, when necessary, shared services teams and vendors to deliver timely, high-quality outcomes.
• Drive data infrastructure planning through the full SDLC, including planning, design, development, certification, implementation, and support.
• Partner with Product, Business, and Analyst stakeholders to confirm data quality, align on requirements, and support data testing.
• Conduct peer code reviews and incorporate learnings from review feedback on your own work.
• Build more complex data models to support insightful analytics while maintaining strong data integrity.
• Mentor and support junior Data Engineers.
• Apply knowledge of machine learning concepts as part of data and predictive solutions.
• Support technology documentation and training related to stacks and team standards.

Requirements

• 5+ years of experience related to data engineering.
• A related degree or comparable formal training, certification, or work experience.
• Experience with Lean Startup or Agile development methodologies, including advanced knowledge of Lean Startup and Agile methods.
• Experience working with large-scale infrastructure, large data sets, and mission-critical SLAs.
• High proficiency in Python, PySpark, and SQL for large-scale data manipulation and pipeline development.
• Proven experience with Google Cloud Platform (GCP) and managed services, including Cloud Dataproc (Spark), Cloud Dataflow, Apache Airflow (Cloud Composer), Google Cloud Storage, and Cloud Pub/Sub.
• Knowledge of modern data lakehouse architectures (Databricks, BigQuery) and scalable ETL/ELT pipelines.
• Knowledge of business intelligence, analytics and reporting, and application integration.
• Strong understanding of data architecture and data modeling best practices for data and analytics platforms.
• Demonstrated experience developing and deploying internal data-serving REST APIs using frameworks like FastAPI, integrating APIs with downstream applications and microservices.
• Strong adherence to software engineering best practices, including unit testing, integration testing, code reviews, Git branching strategies, and CI/CD automation.
• Strong prioritization and verbal/written communication and data presentation skills.
• Ability and willingness to learn new technologies as they emerge.
• Ability to calmly work under pressure.
• Ability to work a flexible schedule as needed and collaborate across multiple work locations.
• Ability to work on a team with willingness to take feedback from peers and mentors.
• Knowledge in machine learning concepts.

Technologies

• Python, PySpark, SQL
• Google Cloud Platform (GCP), Cloud Dataproc (Spark), Cloud Dataflow, Apache Airflow (Cloud Composer), Google Cloud Storage, Cloud Pub/Sub
• Databricks, BigQuery
• FastAPI, REST APIs, ETL/ELT
• CI/CD, Git
• Machine learning concepts

Nice-to-have qualifications

• Retail concepts, including background in retail, POS/transactions, and customer/identity resolution.
• Graph ecosystems experience, including Neo4j and the Cypher / ISO GQL query language.

Work style and availability

• Ability to function in a fast-paced environment with the ability to work from office.
• Ability to work extended hours; sit for extended periods.

HEART FOR PEOPLE

• Willing to facilitate solutions with multiple engineers.
• Provides upward communication.
• Mentors others.

HEAD FOR BUSINESS

• Consistently demonstrates and upholds standards of coding, infrastructure, and process.

PASSION FOR RESULTS

• Capable of high-velocity contributions across multiple technical domains.