Senior Data Engineer

Abinbev; Campinas; Vor Ort; BRL 180.000 - 280.000; Vollzeit

Zusammenfassung

AB InBev Growth Group seeks a Senior Data Engineer to own the technical design and delivery of the BEES data model. You will profile source systems, define canonical datasets, and map attributes to physical columns with lineage.

The role emphasizes data quality, entity resolution, and collaboration with data scientists and analysts. You will contribute to the ontology codebase using Python, SQL, Spark on Databricks, YAML contracts, and Git workflows, while mentoring engineers and guiding data

Qualifikationen

• Senior data engineer with experience designing conceptual and logical data models from source systems.
• Ability to turn ambiguous business context into explicit data models with keys, relationships and grain.
• Hands-on master data management including entity resolution and survivorship rules.
• Advanced data quality techniques: uniqueness, referential integrity, validity and timeliness checks.
• Proficiency in Python, SQL, and PySpark; experience on Databricks is a plus.
• Experience with catalog/metadata systems, data contracts and lineage.

Aufgaben

• Own the design and delivery of the data model for the BEES domain.
• Profile source systems and Unity Catalog tables to ensure trustworthy representations.
• Develop canonical logical models and map attributes to columns with lineage documentation.
• Define master-data outcomes, including entity resolution and survivorship rules.
• Define and implement data-quality expectations and contribute to QA tooling.
• Shape models for data scientists and analysts; feed downstream data products.
• Contribute to the ontology codebase with Python, SQL and YAML contracts; review PRs.
• Mentor engineers through design reviews and cross-team collaboration.
• Communicate clearly to stakeholders without needing meetings.

Kenntnisse

Senior data engineer

Data modeling

Entity resolution

Data quality

Python

SQL

PySpark

Databricks

GitHub

Ausbildung

Bachelor's degree or equivalent

Tools

Databricks

Unity Catalog

SQL

Git

YAML

Azure DevOps

Jobbeschreibung

About us

AB InBev is the leading global brewer and one of the world's top 5 consumer product companies. With over 500 beer brands, we're number one or two in many of the world's top beer markets, including North America, Latin America, Europe, Asia, and Africa.

About AB InBev Growth Group

Created in 2022, the Growth Group unifies our business-to-business (B2B), direct-to-consumer (DTC), Sales & Distribution, and Marketing teams. By bringing together global tech and commercial functions, the Growth Group allows us to fully leverage data and drive digital transformation and organic growth for AB InBev around the world.

What you'll do:

Own the technical design and delivery of the data model for a BEES business domain. Start from business intent and from the data sources that exist, and produce authoritative definitions for entities, assets, attributes, relationships, identifiers, and grain.

Profile source systems and Unity Catalog tables (grain, keys, cardinality, overlap, nulls, and how columns are derived) and decide which sources can support a trustworthy representation of the business.

Produce and evolve the domain's canonical logical model and canonical datasets, including identifiers, relationships, and cardinalities. Map attributes to physical columns, with transformation, nullability, and lineage documented so consumers can trust the result.

Design master-data outcomes for the domain, including entity resolution: match approach, blocking keys, duplicate handling, cross-system identity, and survivorship when sources disagree.

Define advanced data-quality expectations on the model (uniqueness at the declared grain, referential integrity, completeness, validity, consistency across sources, reconciliation, and freshness) and contribute to the Data Quality Framework and Tooling so those expectations become executable checks and monitoring.

Shape the model for its consumers. Data scientists and data analysts are the primary consumers. The same model feeds downstream data products such as curated tables, metric views, and feature tables, with a stable grain and documented semantics.

Contribute to the ontology codebase: Python packages, SQL and Spark on Databricks, YAML definition contracts, entity classification, lineage-aware mappings and tests. Review pull requests on models, mappings, and ontology-aligned dataset changes.

Drive entity and attribute lifecycle work: creation, change, split, rename, deprecation and retirement, with compatibility guidance and clear communication to consumers.

Mentor engineers on the team through design review, source-analysis pairing, and hands‑on support.

What you'll need:

Senior experience as a data engineer, data analyst, or any equivalent role in data, who has designed conceptual and logical data models from operational source systems in a complex business.

A record of turning ambiguous business context and imperfect data into a model with explicit entities, grain, keys, relationships, and cardinalities that downstream teams actually use.

Hands-on master data management, including entity resolution: deterministic and probabilistic matching, blocking, duplicate detection, golden‑record or survivorship rules, and identity across systems.

Advanced data quality technique. You have designed and applied uniqueness, referential integrity, validity, completeness, consistency, reconciliation, and timeliness checks, and you have seen them catch real defects before consumers did. SQL, Python, or a quality framework in production counts.

Traditional data engineering: production SQL (joins, aggregations, window functions), batch ELT patterns, and enough PySpark or Spark SQL to profile a source and prototype a conformed dataset on Databricks.

Ability to contribute to a Python codebase: change an existing package, add a test, and carry a pull request through review. Our stack is Python 3.11+, Databricks (Unity Catalog and Delta), SQL, and YAML contracts in Git.

Experience with catalog or metadata systems, mapping business attributes onto physical tables and columns, lineage, and data contracts between producers and consumers.

Experience reviewing other engineers' models and code, and mentoring through design feedback and documentation.

Collaboration in GitHub or Azure DevOps, including review of changes from outside the immediate team.

Clear written and verbal communication with business stakeholders, data scientists, and data analysts. Definitions should be usable without a meeting.

Bachelor's degree in computer science, engineering, information systems, or another quantitative field, or equivalent practical experience.

Nice to have:

Relevant data management and governance experience: business glossaries, critical data elements, stewardship, naming standards, and dataset certification. The team will teach the ontology vocabulary.

Prior work on Azure Databricks or Unity