Senior Data Engineer

AgileEngine; Brasil; Híbrido; BRL 622 000 - 933 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Professional growth

Competitive compensation

Flextime

Resumo da oferta

AgileEngine is seeking a Senior Data Engineer to architect, build, and scale a modern data platform in Brazil. You will design production-grade ETL/ELT pipelines, optimize Snowflake schemas, and enable enterprise analytics with robust DataOps practices.

Responsibilities include designing scalable data pipelines, integrating REST APIs and event streams, and implementing data quality and observability. You will champion CI/CD, Git, Docker, and IaC while delivering clean, well-tested Python code.

Qualificações

• 4+ years of experience as a Data Engineer.
• Strong Python fundamentals with modular, tested code.
• Advanced SQL and data modeling in Snowflake.
• Experience with ETL/ELT pipelines and data warehousing.
• Orchestration with Airflow, Prefect, or Dagster.
• REST API integration and event-driven ingestion.
• Data quality, lineage, and observability practices.
• DevOps/DataOps: Git, Docker, CI/CD, IaC basics.

Responsabilidades

• Design, build, and maintain reliable ETL/ELT workflows.
• Model Snowflake schemas for analytics and performance.
• Ingest data from REST APIs and event streams.
• Maintain orchestration pipelines with Airflow/Prefect/Dagster.
• Implement data quality checks and lineage tracking.
• Apply CI/CD, Git, Docker, and IaC patterns.
• Write clean, well-tested Python code.

Conhecimentos

Python

SQL

Snowflake

Data Modeling

Airflow/Prefect/Dagster

ETL/ELT

REST APIs

CI/CD

Git

Docker

IaC

Data Quality

Data Observability

Ferramentas

dbt

Great Expectations

Apache Airflow

Prefect

Dagster

Descrição da oferta de emprego

We are looking for a Senior Data Engineerto architect, build, and scale a modern data platform — designing production-grade ETL/ELT pipelines, optimizing Snowflake data warehouse schemas, and establishing robust DataOps practices. You will orchestrate workflows using Airflow, Prefect, or Dagster, implement data quality and lineage frameworks, integrate third‑party REST APIs and event‑driven sources, and apply software engineering standards including CI/CD, Docker, and automated testing to data repositories. The role prioritizes clean, well-tested Python code and a deep commitment to data reliability and accessibility.

What you will do

• Data Pipeline Development: Design, build, and maintain reliable, scalable ETL/ELT workflows that process batch and streaming data from diverse sources.
• Data Warehousing & Modeling: Design efficient, production-ready schemas (normalized and denormalized) in Snowflake to optimize query performance and enable enterprise analytics.
• API & Event Integration: Connect and ingest data from third‑party REST APIs, event‑driven streams, and batch sources into core data storage platforms.
• Orchestration: Maintain and expand workflow orchestration pipelines using modern tools (Airflow, Prefect, or Dagster).
• Data Quality & Observability: Implement automated testing, validation, lineage tracking, and proactive alerting frameworks to guarantee data accuracy and system uptime.
• DataOps & Engineering Standards: Drive CI/CD best practices, maintain code bases using Git and Docker, and adopt basic Infrastructure‑as‑Code (IaC) patterns.
• Code Excellence: Apply modern software engineering standards—including design patterns, automated unit/integration testing, and clear documentation—to data repositories.

Must haves

• 4+ years of experience as a Data Engineer.
• Core Python Fundamentals: Demonstrable expertise writing modular, maintainable, and well‑tested Python code (OOP/functional patterns, package management, standard testing frameworks).
• Advanced SQL & Modeling: Deep knowledge of complex SQL queries, query optimization, database design principles, and normalization/denormalization patterns.
• Data Warehousing: Solid, hands‑on experience building, managing, and optimizing data architectures within Snowflake.
• Workflow Orchestration: Production experience using workflow orchestration engines like Apache Airflow, Prefect, or Dagster.
• Integrations & Ingestion: Hands‑on experience working with REST APIs, event‑driven architectures, and both batch and streaming pipelines.
• Data Quality & Lineage: Experience building automated data quality checks, data lineage, and alerting mechanisms (e.g., using tools like dbt test, Great Expectations, or similar).
• DevOps / DataOps Practices: Strong skills in version control (Git), containerization (Docker), CI/CD automation, and familiarity with Infrastructure‑as‑Code basics.

Nice to haves

• Experience with dbt (data build tool) for data transformations.
• Familiarity with major cloud providers (AWS, GCP, or Azure).
• Exposure to message streaming tech like Apache Kafka or AWS Kinesis.

Perks and Benefits

• Professional growth

Accelerate your professional journey with mentorship, TechTalks, and personalized growth roadmaps

• Competitive compensation

We match your ever‑growing skills, talent, and contributions with competitive USD‑based compensation and budgets for education, fitness, and team activities

• A selection of exciting projects

Join projects with modern solutions development and top‑tier clients that include Fortune 500 enterprises and leading product brands

• Flextime

Tailor your schedule for an optimal work‑life balance, by having the options of working from home and going to the office – whatever makes you the happiest and most productive.