Senior Data Engineer, Spark

Cadmus Soluções em TI; São Paulo; On-site; BRL 223,200 - 446,400; Full time

Job summary

Jobtailor in São Paulo seeks a data engineer to develop and optimize high-performance data pipelines and design scalable data processing solutions.

You will contribute to the data architecture, implement ETL/ELT across multiple sources, and work with Apache Spark and PySpark. The role emphasizes data quality, reliability, and collaboration with cross-functional teams.

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems or related fields.
• Advanced English for oral and written communication.
• Excellent communication and collaboration skills across technical and business teams.
• Minimum of 7 years of professional experience in Technology.
• At least 5 years of hands-on experience as a Data Engineer using Apache Spark.

Responsibilities

• Develop and optimize high-performance data pipelines.
• Design scalable solutions for processing large volumes of data.
• Contribute to the definition and evolution of the data architecture.
• Implement ETL processes and integration across multiple data sources.
• Work with distributed systems and event-driven architectures.
• Ensure data quality, performance, and reliability.
• Integrate data platforms with applications and APIs.
• Participate in technical discussions and support the evolution of engineering best practices.

Skills

Advanced English

Data Engineering

ETL/ELT

Apache Spark

PySpark

Distributed data processing

Data modeling

Integration

Apache Kafka

Real-time processing

REST APIs

Cloud Computing

SQL Databases

NoSQL databases

Distributed Systems

Education

Bachelors in CS/Engineering/IS

Job description

Responsibilities

• Develop and optimize high-performance data pipelines.
• Design scalable solutions for processing large volumes of data.
• Contribute to the definition and evolution of the data architecture.
• Implement ETL processes and integration across multiple data sources.
• Work with distributed systems and event-driven architectures.
• Ensure data quality, performance, and reliability.
• Integrate data platforms with applications and APIs.
• Participate in technical discussions and support the evolution of engineering best practices.

Requirements

• Bachelor's degree in Computer Science, Engineering, Information Systems or related fields.
• Advanced English for oral and written communication.
• Excellent communication and collaboration skills across technical and business teams.
• Minimum of 7 years of professional experience in Technology.
• At least 5 years of hands-on experience as a Data Engineer using Apache Spark.
• Mandatory Technical Knowledge
• Data Engineering
• ETL/ELT
• Apache Spark
• PySpark
• Distributed data processing
• Data modeling and integration
• Messaging and Streaming
• Apache Kafka
• Real-time event processing
• Databases
• Relational databases (SQL)
• NoSQL databases
• Architecture and Platform
• System Architecture
• Distributed Systems
• REST APIs
• Cloud Computing
• Application integration