High-Performance Data Engineer

Bebeedataengineering; Porto Alegre; Presencial; BRL 440.000 - 551.000; Tempo integral

Resumo da oferta

A leading data engineering company in Porto Alegre is looking for a Senior Data Engineer to design and maintain high-performance data processing pipelines using Python, Pandas, and Polars. Applicants should have experience in developing RESTful APIs with FastAPI and a solid understanding of data modeling principles. The role offers a competitive salary, benefits, and opportunities for professional growth within a dynamic team environment.

Serviços

Competitive salary

Benefits package

Professional development opportunities

Qualificações

• Strong expertise in Python for data processing.
• Experience in building backend services or APIs.
• Proficiency with GitHub and CI/CD pipelines.

Responsabilidades

• Design and maintain high-performance data processing pipelines.
• Develop RESTful APIs using FastAPI.
• Handle normalized datasets and generate dynamic Excel reports.

Conhecimentos

Python

Pandas

Polars

FastAPI

Data modeling

GitHub

CI/CD pipelines

Infrastructure-as-Code

Ferramentas

Bicep

Terraform

AWS CDK

Descrição da oferta de emprego

Senior Data Engineer Position

This role involves designing, building, and maintaining high-performance data processing pipelines utilizing Python libraries such as Pandas and Polars. Develop RESTful APIs using frameworks like FastAPI or similar technologies. Consume and process normalized Parquet files from multiple upstream sources to generate dynamic Excel reports.

Key Responsibilities

Strong expertise in Python for data processing with a focus on Pandas, along with proficiency in Polars and openpyxl. Experience in building backend services or APIs using frameworks like FastAPI. Solid understanding of data modeling principles, particularly the Star Schema, and handling normalized datasets. Familiarity with enterprise messaging patterns and data integration from various sources, including API-based and file-based systems. Proficiency in working with GitHub and CI/CD pipelines, specifically GitHub Actions or comparable tools. Infrastructure-as-Code experience with Bicep or equivalent tools, such as Terraform or AWS CDK.

About This Opportunity

We are seeking a skilled Senior Data Engineer to join our team.

What We Offer

A competitive salary and benefits package. The opportunity to work with a talented team of engineers and contribute to the development of innovative data processing solutions. A dynamic and supportive work environment that fosters growth and professional development.