Sr Python Data Engineer

Softensity Inc; Manaus; Presencial; BRL 318.000 - 425.000; Tempo integral

Resumo da oferta

A technology solutions company in Brazil is looking for a Senior Python Data Engineer. In this role, you will design and build high-performance data processing pipelines and develop RESTful APIs using FastAPI. The ideal candidate must have strong proficiency in Python and experience with data modeling principles. You will also collaborate on automation and contribute to CI/CD pipelines. This position welcomes applications from all qualified candidates without regard to gender or identity.

Qualificações

• Strong proficiency in Python for data processing.
• Experience in building backend services or APIs.
• Solid understanding of data modeling principles.
• Familiarity with enterprise messaging patterns.
• Experience working with CI/CD pipelines.
• Infrastructure-as-Code experience.

Responsabilidades

• Design and maintain data processing pipelines.
• Develop and expose RESTful APIs.
• Consume and process normalized Parquet files.
• Collaborate on infrastructure-as-code automation.
• Contribute to CI/CD pipelines.

Conhecimentos

Proficiency in Python

Experience with Pandas

Experience with API frameworks

Data modeling principles

Familiarity with messaging patterns

Experience with GitHub Actions

Infrastructure-as-Code tools

Ferramentas

FastAPI

Bicep

GitHub Actions

Descrição da oferta de emprego

Senior Python Data Engineer

About the Project

Design, build, and maintain high-performance data processing pipelines using Python libraries (Pandas, Polars).

Develop and expose RESTful APIs using FastAPI or similar frameworks.

Consume and process normalized Parquet files from multiple upstream sources to generate dynamic Excel reports.

Contribute to a spec-driven development workflow (using GitHub Copilot, Claude, etc.) to scaffold and generate API / data pipeline code.

Optimize report generation logic for speed and scalability, currently targeting sub-20 second response times.

Integrate with messaging and storage mechanisms (e.g., Service Bus, Storage Accounts).

Collaboration & Automation

Collaborate on infrastructure-as-code automation using Bicep (or similar IaC tools).

Participate in design discussions for future migration to Snowflake and / or a data lake architecture.

Contribute to CI / CD pipelines using GitHub Actions.

Required Skills and Experience

• Strong proficiency in Python for data processing (must have expertise in Pandas, nice to have : Polars, openpyxl).
• Experience building backend services or APIs using frameworks like FastAPI.
• Solid understanding of data modeling principles (Star Schema) and handling normalized datasets.
• Familiarity with enterprise messaging patterns and data integration from various sources (API-based and file-based).
• Experience working with GitHub and CI / CD pipelines (GitHub Actions or similar).
• Infrastructure-as-Code experience with Bicep or comparable tools (Terraform, AWS CDK).
• Comfort with spec-driven development and leveraging AI tools like GitHub Copilot for scaffolding.

Equal Opportunity Employer

Softensity is an equal-opportunity employer.

All qualified applicants are considered without regard to gender, identity, or personal background.