Senior Analytics Engineer (Databricks, Spark)

exadelinc; São Paulo; Presencial; BRL 300 000 - 420 000; Tempo integral

Resumo da oferta

Exadel Inc. seeks a data engineering professional to join a client project in Brazil, focused on building scalable data pipelines with Databricks, Spark, and modern data warehousing practices.

The role emphasizes production-grade pipelines, dimensional modeling, and scalable performance, with emphasis on Delta Lake, SCDs, and governance. You will own end-to-end data flows and collaborate across teams in a fast-growing AI-driven environment.

Qualificações

• Professional experience in Data Engineering or Analytics Engineering.
• Proven track record building production-grade data pipelines.
• Advanced SQL and data modeling skills (Fact/Dimension tables, SCD).

Responsabilidades

• Design, develop, and maintain scalable data pipelines integrating diverse data sources.
• Transform and model structured and semi-structured data for analytics.
• Build analytics-ready datasets and metrics as a single source of truth.
• Ensure data quality, testing, monitoring, and observability in production.
• Collaborate with data engineers, software engineers, analytics, and product teams.

Conhecimentos

Data engineering

Production pipelines

SQL

Data modeling

Problem solving

Ferramentas

Databricks

Apache Spark

Unity Catalog

Git

CI/CD

Descrição da oferta de emprego

Why Join Exadel

We're an AI-first global tech company with 25+ years of engineering leadership, 2,000+ team members, and 500+ active projects powering Fortune 500 clients, including HBO, Microsoft, Google, and Starbucks.

From AI platforms to digital transformation, we partner with enterprise leaders to build what's next.

What powers it all? Our people are ambitious, collaborative, and constantly evolving.

About the Client

A leading global digital banking platform and one of the largest neobanks in Latin America, headquartered in Brazil. The company provides a mobile-first suite of financial services-including credit cards, savings accounts, and investments-designed to eliminate bureaucracy and high fees. By leveraging advanced data analytics and a customer-centric digital infrastructure, the organization delivers transparent and accessible financial solutions to millions of users across Brazil, Mexico, and Colombia.

What You'll Do

• Design, develop, and maintain scalable, production-grade data pipelines integrating SaaS platforms, APIs, batch processes, and enterprise data sources.
• Transform, integrate, standardize, and model structured and semi-structured data from multiple heterogeneous systems.
• Develop and optimize data processing solutions using Databricks and Apache Spark for large-scale distributed data processing.
• Design and implement reliable dimensional data models, including Fact, Dimension, and Event tables, following modern Data Engineering and Data Warehouse best practices.
• Build trusted, analytics-ready datasets and business metrics that serve as a Single Source of Truth for analytical and business use cases.
• Develop incremental, idempotent, scalable, and maintainable data pipelines.
• Optimize data processing workloads, queries, and pipeline performance with a focus on scalability and efficiency.
• Implement automated data quality, validation, testing, monitoring, and observability practices.
• Troubleshoot production issues, investigate root causes, and implement sustainable solutions to improve reliability and operational excellence.
• Validate data reconciliation between legacy and modern cloud platforms throughout migration initiatives.
• Collaborate closely with Data Engineers, Software Engineers, Analytics teams, Product teams, and other cross-functional stakeholders.
• Take end-to-end ownership of technical solutions, from design and development through deployment, production support, and continuous improvement.
• Contribute to CI/CD, code reviews, automated testing, and modern software engineering practices.
• Leverage AI-assisted development tools to accelerate coding, testing, troubleshooting, documentation, and other engineering activities while applying sound technical judgment.
• Identify opportunities to improve automation, platform reliability, engineering productivity, and data quality.
• Produce technical documentation and facilitate knowledge sharing and knowledge transfer across engineering teams.

What You Bring

• Professional experience in Data Engineering, Analytics Engineering, or a related field.
• Proven experience building and operating production-grade data pipelines.
• Strong hands-on experience with Databricks, including Workflows, Jobs, and Unity Catalog.
• Advanced experience with Apache Spark, using PySpark and/or Scala.
• Experience processing and optimizing large-scale datasets, including datasets containing hundreds of millions of records.
• Advanced SQL skills, including complex queries, data transformations, and query optimization.
• Strong understanding of data modeling and analytical data structures, including Fact and Dimension tables, SCD, and data grain.
• Strong knowledge of Delta Lake and/or Apache Iceberg.
• Experience building incremental and idempotent data pipelines.
• Experience implementing data quality, validation, testing, monitoring, and observability practices.
• Strong troubleshooting and production support experience.
• Experience with Git, CI/CD pipelines, automated testing, code reviews, and modern software engineering practices.
• Strong analytical and problem-solving skills.
• Good communication skills and the ability to clearly communicate technical concepts and solutions.

Nice to have

• Experience supporting cloud migration and data modernization initiatives, particularly migrations from GCP to AWS.
• Experience with AWS services, including S3, Glue, and Lake Formation.
• Familiarity with Google Cloud Platform (GCP) or other major cloud platforms.
• Experience with modern Lakehouse architectures and Semantic Layer concepts.
• Experience integrating REST APIs and processing JSON data.
• Knowledge of Data Governance principles and practices.
• Experience with Infrastructure as Code (IaC) tools such as Terraform.
• Experience with data observability and monitoring tools.
• Experience working with sensitive HR/People data and familiarity with LGPD, GDPR, or similar data privacy regulations.
• Familiarity with modern AI-assisted development tools and workflows