Data Engineer SR - Apache Flink

Encora Inc.; Jaboatão dos Guararapes; Presencial; BRL 80.000 - 120.000; Tempo integral

Resumo da oferta

A leading company is seeking a Data Engineer to design and maintain scalable data pipelines using Apache Flink and Kafka. The role involves optimizing data infrastructure on Google Cloud Platform and ensuring data quality. Candidates should have experience in stream processing and strong SQL skills.

Qualificações

• Experience as a Data Engineer.
• Strong experience with Apache Flink and Kafka.
• Proficient in SQL and at least one programming language.

Responsabilidades

• Design and optimize data pipelines using Apache Flink.
• Implement and maintain Kafka Connect connectors.
• Collaborate with teams to understand data requirements.

Conhecimentos

Apache Flink

Kafka

SQL

Docker

Kubernetes

Data Modeling

Git

Java

Python

Ferramentas

Google Cloud Platform

BigQuery

Apache Airflow

Tableau

PowerBI

Prometheus

Grafana

Descrição da oferta de emprego

As a Data Engineer, you will work with stream processing technologies, particularly Apache Flink and Kafka Connect. In this role, you'll design, implement, and maintain scalable data pipelines that process real-time and batch data, enabling critical business insights and data-driven decision making.

Responsibilities and Duties

1. Design, develop, and optimize data pipelines using Apache Flink for stream and batch processing;
2. Implement and maintain Kafka Connect connectors for seamless data integration;
3. Build and maintain data infrastructure on Google Cloud Platform (GCP);
4. Design and optimize BigQuery tables, views, and stored procedures;
5. Collaborate with product managers and analysts to understand data requirements;
6. Ensure data quality, reliability, and proper governance;
7. Troubleshoot and resolve data pipeline issues;
8. Document data flows, transformations, and architecture.

Minimum Qualifications

• Experience as a Data Engineer;
• Strong experience with Apache Flink, including knowledge of the DataStream and Table APIs;
• Experience with Kafka and Kafka Connect for data integration;
• Experience with containerization (Docker) and orchestration (Kubernetes);
• Experience in Cloud Infrastructure platforms such as AWS, GCP;
• Solid SQL skills with specific experience in BigQuery SQL dialect;
• Experience with data modeling and schema design;
• Proficient in at least one programming language (Java or Python);
• Experience with version control systems (Git) and CI/CD pipelines.

Additional Skills

• Knowledge and hands-on experience with Kafka Infrastructure (i.e., managing and configuring Kafka components);
• Experience with other stream processing frameworks (Spark Streaming, Kafka Streams);
• Knowledge of data governance and security best practices;
• Proficiency with Google Cloud Platform services, particularly Dataflow, Pub/Sub, BigQuery;
• Experience with Apache Airflow;
• Experience with BI tools such as Tableau or PowerBI;
• Familiarity with monitoring tools like Prometheus, Grafana;
• Certifications such as Google Cloud Professional Data Engineer.

About Encora

Encora is the preferred digital engineering and modernization partner of some of the world’s leading enterprises and digital native companies. With over 9,000 experts in 47+ offices and innovation labs worldwide, Encora’s technology practices include Product Engineering & Development, Cloud Services, Quality Engineering, DevSecOps, Data & Analytics, Digital Experience, Cybersecurity, and AI & LLM Engineering.

At Encora, we hire professionals based solely on their skills and qualifications, and do not discriminate based on age, disability, religion, gender, sexual orientation, socioeconomic status, or nationality.