Data Lake Architect

Bebeedataleader; Manaus; Presencial; BRL 160.000 - 200.000; Tempo integral

Resumo da oferta

A leading data solutions company located in Brazil is seeking a proficient Data Engineer to design and implement AWS Serverless DataLake architecture. The successful candidate will develop and optimize data ingestion pipelines and ensure data quality and transformation using serverless technologies. Responsibilities also include mentoring junior staff and collaborating with cross-functional teams to deliver timely, high-quality data solutions. This role offers an exciting opportunity to influence the future of data processing in a dynamic environment.

Qualificações

• Experience in designing AWS Serverless architectures.
• Proficiency in data ingestion and integration processes.
• Strong background in data quality and transformation using serverless technologies.

Responsabilidades

• Implement and optimize DataLake architecture for data efficiency.
• Mentor junior engineers and ensure adherence to best practices.
• Collaborate with teams to translate business requirements into data solutions.

Conhecimentos

AWS Lambda

Data ingestion pipelines

Data processing workflows

Cloud technologies

Ferramentas

AWS Glue

DataLake technologies

Descrição da oferta de emprego

About the RoleAs a key member of our team, you will play a pivotal role in designing and implementing an AWS Serverless DataLake architecture.

Your primary objective will be to efficiently handle large volumes of data and support various data processing workflows.You will develop data ingestion pipelines and integration processes, ensuring the smooth and reliable transfer of data from diverse sources into the DataLake.

To ensure data quality and consistency, you will implement transformation and enrichment processes using AWS Lambda, Glue, or similar serverless technologies.Collaborating closely with data scientists and analysts, you will design appropriate data models and schemas in the DataLake.

This involves understanding their data requirements and optimizing storage and retrieval mechanisms to provide high-performance access to the data.You will be responsible for monitoring and troubleshooting the DataLake infrastructure, identifying and resolving performance bottlenecks, data processing errors, and other issues.

Additionally, you will continuously evaluate new AWS services and technologies to enhance the DataLake architecture, improve data processing efficiency, and drive innovation.As a seasoned professional, you will mentor and provide technical guidance to junior data engineers, fostering their growth and ensuring adherence to best practices.

You will also collaborate with cross-functional teams to understand business requirements, prioritize tasks, and deliver high-quality solutions within defined timelines.