Teach Lead Data Engineer

Jobgether; Brasil; Presencial; BRL 280 000 - 520 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Career growth opportunities

Collaborative environment

Global data/tech exposure

Resumo da oferta

Jobgether is seeking a Tech Lead Data Engineer based in Brazil to steer a petabyte-scale AWS data platform. You will design scalable pipelines using Spark, Python, and AWS-native services while mentoring engineers and shaping long-term architecture.

You’ll collaborate with product and operations, adopt modern technologies such as vector databases and AI/LLM techniques, and drive platform evolution in an agile environment.

Qualificações

• Proven experience building large-scale data platforms in cloud environments.
• Strong backgrounds in Java/Scala and Spark for petabyte-scale pipelines.
• Experience with AWS data services and modern data transformations.

Responsabilidades

• Lead design and evolution of a petabyte-scale AWS data platform.
• Mentor engineers, conduct code reviews, and uphold engineering standards.
• Build scalable data pipelines with Spark, Python, and AWS-native tools.
• Collaborate on architecture and long-term platform strategy.

Conhecimentos

Java

Scala

Kotlin

Python

Distributed systems

Mentoring

AWS

Data engineering

SQL/ETL

Git

Formação académica

Bachelor's degree in Computer Science or related field

Ferramentas

Spark

Airflow

DBT

Iceberg

pgvector

Redis vector

Descrição da oferta de emprego

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Tech Lead Data Engineer based in Brazil.

As a Tech Lead Data Engineer, you will lead the evolution of a petabyte-scale data platform running on AWS, combining advanced software engineering with large-scale data engineering. You will design and build highly scalable, resilient, and high-performance data pipelines and services that support data-driven products and analytics. The role combines hands‑on technical work with architectural leadership, mentoring, and long‑term platform strategy. You will collaborate closely with product, engineering, architecture, and operations teams to solve complex technical challenges and ensure reliable platform evolution. You will also help introduce modern technologies, including vector databases and AI/LLM techniques, to improve data access and engineering efficiency. This is an opportunity to work on sophisticated distributed systems within an agile, collaborative, and innovation‑focused environment.

Accountabilities:

• Design, implement, and continuously evolve a petabyte-scale AWS data platform using advanced software engineering, distributed systems principles, and cloud‑native practices.
• Build and optimize highly scalable data pipelines using JVM‑based languages such as Java, Scala, or Kotlin, together with Apache Spark, Python, and AWS‑native services.
• Ensure data processing solutions are efficient, resilient, production‑ready, and capable of supporting analytics and downstream products at scale.
• Act as a principal‑level technical leader by mentoring engineers, conducting high‑quality code reviews, and promoting engineering standards and best practices.
• Contribute to architectural and technical design decisions in partnership with architects, product owners, and engineering leadership.
• Help define the long‑term evolution of the data platform while ensuring technical decisions remain aligned with broader business and product objectives.
• Plan and deliver complex technical initiatives within agile value‑stream teams, ensuring predictable execution, strong collaboration, and high‑quality outcomes.
• Ensure operational readiness through appropriate coding standards, testing practices, monitoring strategies, release procedures, and production deployment processes.
• Monitor and improve platform stability, scalability, performance, reliability, and operational resilience.
• Investigate and adopt modern technologies and approaches, including GraphQL integrations, vector databases, and AI/LLM‑based techniques where they can enhance platform capabilities.
• Collaborate across multiple engineering teams and repositories to maintain consistency, technical quality, and effective delivery.
• Contribute to continuous improvement initiatives that raise engineering maturity and strengthen the organization’s data platform capabilities.

Requirements:

• Expert‑level experience in software and data engineering, preferably within large‑scale or highly complex data platforms.
• Deep experience building petabyte‑scale systems using Java‑based languages such as Scala and Apache Spark.
• Strong expertise in the AWS data ecosystem, particularly services and technologies such as Glue, S3, Athena, Managed Airflow, and Iceberg.
• Advanced understanding of distributed systems, parallel processing, and highly parallelized workloads.
• Strong knowledge of query optimization, data partitioning, and efficient data storage patterns.
• Extensive AWS cloud experience; knowledge of Azure or GCP is an advantage.
• Proven ability to influence long‑term architecture, platform design, and technical strategy.
• Strong technical leadership, mentoring, code review, and engineering quality skills.
• Hands‑on experience working in agile environments and collaborating effectively across multiple engineering teams.
• Strong version control and multi‑repository collaboration skills using tools such as Git, GitHub, and Bitbucket.
• Bachelor’s degree in Computer Science, Engineering, or a related technical field, or equivalent professional experience.
• Advanced English proficiency, with the ability to communicate effectively in international technical environments.
• Availability to travel to São Carlos, São Paulo, when required.
• Experience with DBT or other modern data transformation frameworks is desirable.
• Knowledge of concurrent and parallel programming is a plus.
• Familiarity with Python, Angular, and TypeScript is desirable.
• Experience with vector databases such as pgvector or Redis vector fields is an advantage.
• Practical experience applying AI and LLM techniques to data platforms, data access, or software quality is desirable.
• Strong communication, collaboration, problem‑solving, and mentoring skills, with the ability to navigate complex technical challenges and influence stakeholders.

Benefits:

• Opportunity to work on a petabyte‑scale AWS data platform and complex distributed data systems.
• Exposure to advanced cloud‑native technologies and modern data engineering practices.
• Opportunity to work with emerging technologies such as vector databases and AI/LLM‑based solutions.
• Strong focus on technical development, engineering quality, mentorship, and knowledge sharing.
• Collaboration with multidisciplinary teams across product, engineering, architecture, and operations.
• Opportunity to contribute to large‑scale data-driven products and long‑term platform evolution.
• Inclusive working environment that values diversity, different perspectives, and professional development.
• Access to employee affinity initiatives supporting inclusion and communities across different backgrounds.
• Opportunity to contribute to technically challenging projects within a global data and technology environment.