Senior Data Engineer

Jobtailor; São Paulo; Presencial; BRL 120 000 - 180 000; Tempo integral

Resumo da oferta

Jobtailor in São Paulo is seeking a Data Engineer to design and maintain scalable ETL pipelines using Python, Spark, and Databricks on AWS. You will ensure high-quality, low-latency data delivery for analytics across the company.

You will collaborate with Analytics, Product, and Engineering teams, implement data lake architectures, and continuously improve data processes and platform reliability in a high-volume fintech-like environment.

Qualificações

• Strong experience as a Data Engineer, developing and maintaining production data pipelines.
• Experience with AWS, Python, Databricks, and batch and streaming ETL processes.
• Knowledge of relational and non-relational databases, and familiarity with tools such as Kafka, Kibana, and Elasticsearch.
• Good communication skills to collaborate with different areas and translate business needs into technical solutions.
• English for reading and understanding technical documentation.
• Experience at technology companies or high-scale fintechs.
• Experience with Data Lakes and modern data architectures.
• Experience working with large-volume data environments.
• Able to document solutions and share knowledge with the team.

Responsabilidades

• Develop and evolve ETL pipelines for data ingestion, transformation, and delivery.
• Build integrations between different data sources, ensuring data quality, performance, and standardization.
• Develop solutions using Python and Spark, working on AWS Cloud on the Databricks platform.
• Monitor and improve production pipelines, ensuring reliability and process efficiency.
• Collaborate with Analytics, Product, and Engineering teams to meet new data requirements.
• Ensure data is available with high quality and low latency for consumption across the company.
• Contribute to improvements in architecture, best practices, and the continuous evolution of data engineering.

Conhecimentos

Data Engineering

ETL pipelines

AWS

Python

Databricks

Streaming ETL

Data Lakes

SQL/Databases

Communication

Documentation

Collaboration

Technical English

Kafka

Kibana

Elasticsearch

Ferramentas

AWS

Python

Databricks

Kafka

Kibana

Elasticsearch

Descrição da oferta de emprego

Responsibilities

• Develop and evolve ETL pipelines for data ingestion, transformation, and delivery.
• Build integrations between different data sources, ensuring data quality, performance, and standardization.
• Develop solutions using Python and Spark, working on AWS Cloud on the Databricks platform.
• Monitor and improve production pipelines, ensuring reliability and process efficiency.
• Collaborate with Analytics, Product, and Engineering teams to meet new data requirements.
• Ensure data is available with high quality and low latency for consumption across the company.
• Contribute to improvements in architecture, best practices, and the continuous evolution of data engineering.

Requirements

• Strong experience as a Data Engineer, developing and maintaining production data pipelines.
• Experience with AWS, Python, Databricks, and batch and streaming ETL processes.
• Knowledge of relational and non-relational databases, and familiarity with tools such as Kafka, Kibana, and Elasticsearch.
• Good communication skills to collaborate with different areas and translate business needs into technical solutions.
• English for reading and understanding technical documentation.
• Experience at technology companies or high-scale fintechs.
• Experience with Data Lakes and modern data architectures.
• Experience working with large-volume data environments.
• Able to document solutions and share knowledge with the team.