Senior Data Architect - International Project

DTI Digital; Brasil; Presencial; BRL 180 000 - 240 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Private Pension

Dental Plan

Meal Allowance

Health Insurance

Home Office Allowance

TotalPass

SESC Partnership

Certification Reimbursement

English Courses

University Partnerships

Pharmacy Discounts

Relaxed Environment

Continuous Learning

Resumo da oferta

dti digital is seeking a data engineer to design and operate high‑performance data Lakehouse solutions. You will build and maintain automated ingestion pipelines using Databricks Delta Live Tables, ensure data quality, and govern data assets with Unity Catalog.

The role requires advanced Python/SQL skills and production experience with Terraform, AWS S3, and CI/CD, collaborating with global teams to drive continuous improvements across development and delivery cycles.

Qualificações

• Hands-on expertise with Databricks including PySpark, Spark SQL, Delta Lake, Delta Live Tables (DLT), and Unity Catalog.
• Advanced Python and SQL skills with strong software development practices, including clean code and testing.
• Experience building file ingestion pipelines for Parquet/Delta using AWS S3 and orchestrating data workflows in production.
• Hands-on experience with Terraform and CI/CD processes, including Git and automated deployments.
• Experience implementing Data Quality rules and managing access control through RBAC and data security policies.
• Advanced English proficiency to work with global platform teams and produce technical documentation.

Responsabilidades

• Automation & Infrastructure: Provision and maintain data workspaces across Dev, Acc, Prod using IaC with Terraform and CI/CD pipelines.
• Pipeline Engineering (DLT): Develop and optimize automated ingestion and transformation pipelines using Databricks Delta Live Tables for Bronze and Silver layers with Unity Catalog governance.
• Data Quality: Implement and monitor automated DLT data quality validations and production alerts.
• Unstructured Source Automation: Design ingestion workflows for various data sources and transform into auditable Lakehouse pipelines.
• Governance & Documentation: Catalog data assets, map data lineage, and produce detailed documentation to prevent technical debt.

Conhecimentos

Databricks

DLT (Delta Live Tables)

Python

SQL

PySpark

Delta Lake

Unity Catalog

AWS S3

Terraform

CI/CD

Data Quality

Formação académica

BSc in Computer Science or related

Ferramentas

Spark SQL

Git

Descrição da oferta de emprego

• Is passionate about solving problems, driving innovation, and developing high-performance data solutions;
• Has strong logical thinking skills and is eager to learn and grow alongside dti digital;
• Is focused, analytical, and proactive when dealing with day-to-day challenges and Lakehouse governance;
• Enjoys working as part of a team and communicating effectively with global teams to collaborate throughout development and delivery cycles;
• Is creative and capable of investigating root causes and driving continuous improvements to existing pipelines and architectures;
• And, most importantly, identifies with our culture!

WE ARE LOOKING FOR SOMEONE WHO:

• Is passionate about solving problems, driving innovation, and developing high-performance data solutions;
• Has strong logical thinking skills and is eager to learn and grow alongside dti digital;
• Is focused, analytical, and proactive when dealing with day-to-day challenges and Lakehouse governance;
• Enjoys working as part of a team and communicating effectively with global teams to collaborate throughout development and delivery cycles;
• Is creative and capable of investigating root causes and driving continuous improvements to existing pipelines and architectures;
• And, most importantly, identifies with our culture!

RESPONSIBILITIES AND DUTIES

• Automation & Infrastructure: Provision and maintain data workspaces across multiple environments (Dev, Acc, and Prod), applying Infrastructure as Code (IaC) principles using Terraform and CI/CD pipelines.
• Pipeline Engineering (DLT): Develop and optimize automated ingestion and transformation pipelines using Databricks Delta Live Tables (DLT) for the Bronze and Silver layers, ensuring federated governance through Unity Catalog.
• Data Quality: Implement and monitor automated, native DLT data quality validations (Expectations) and production alerts.
• Unstructured Source Automation: Design and automate ingestion workflows for manual and recurring data sources (files, spreadsheets, and periodic reports), transforming them into auditable Lakehouse pipelines.
• Governance & Documentation: Catalog data assets, map data lineage, and produce detailed technical documentation for tables and business rules in Unity Catalog, preventing the accumulation of technical debt.

REQUIREMENTS AND QUALIFICATIONS

• Hands-on expertise and solid experience with Databricks, including PySpark, Spark SQL, Delta Lake, Delta Live Tables (DLT), and Unity Catalog.
• Advanced Python and SQL skills, with a strong foundation in software development best practices, including clean code, code reuse, and testing.
• Experience building file ingestion pipelines for columnar formats (Parquet, Delta) using AWS S3, as well as orchestrating data workflows in production environments.
• Hands-on experience with Terraform and CI/CD processes, including Git and automated deployments across environments.
• Experience implementing Data Quality rules and managing access control through RBAC and data security policies.
• Advanced English proficiency to work directly with global platform teams, participate in daily ceremonies, and produce technical documentation.

NICE-TO-HAVE EXPERIENCE / KNOWLEDGE

• Knowledge of enterprise ERP table structures, particularly SAP FI/CO, and related extraction tools (e.g., Theobald Xtract Universal).
• Hands-on experience with Data Vault 2.0 architecture and data modeling, including Hubs, Links, and Satellites.
• Experience with modern data connectors or integration through REST APIs (e.g., dltHub and custom Python connectors).
• Familiarity with and daily use of AI tools for accelerating software development and engineering (e.g., Claude Code, GitHub Copilot, Databricks Assistant/Genie).
• Active Databricks Data Engineer certification (Associate or Professional).

BENEFITS OF WORKING AT DTI

• Private Pension Plan: For every Brazilian real contributed by the employee, up to 6% of their salary, dti matches the same amount;
• Dental Plan;
• Meal and/or Food Allowance: R$890/month;
• Health Insurance: No employee or dependent contribution;
• Home Office Allowance: R$120/month;
• TotalPass: Discounts on gyms, studios, online classes, and more;
• SESC Partnership: Discounts on accommodations and leisure activities;
• Certification Reimbursement;
• English Course Discounts: Partnerships with Open English, Cultura Inglesa, CNA, and EHeadset;
• USP/ESALQ Partnership: Discounts on graduate and specialization programs;
• PUC-RS Partnership: Discounts on undergraduate, graduate, specialization, language, and certification programs;
• FIAP Partnership: Discounts on undergraduate and graduate programs;
• FIA Partnership: Discounts on undergraduate and graduate programs;
• ESPM Partnership: Discounts on graduate and short-term courses;
• Pet Health Insurance;
• Discounts at Drogasil and Droga Raia Pharmacies;
• A relaxed and flexible work environment, with continuous feedback;
• Continuous learning opportunities, including Tech Shots, Mentoring, Conversation Circles, and Knowledge Transfers.

DOES THIS SOUND LIKE YOU?

If your resume is selected for the first stage of our recruitment process, we will contact you. So, please keep an eye on your email — including your spam folder — and your phone.

If your resume is not selected during the screening stage, we will not be able to provide individual feedback. However, your profile will remain in our talent pool, and you are welcome to apply for other opportunities with us in the future. ;)