Senior Data Engineer

Avenue Code; Brasil; Presencial; BRL 180 000 - 300 000; Tempo integral

Resumo da oferta

Avenue Code is seeking a Senior Data & AI Platform Engineer to design, build, and operate scalable cloud-native data platforms on AWS. You will develop robust data architectures, ingestion and processing pipelines, and AI-ready platforms that enable analytics, ML, and Generative AI initiatives across the organization.

The role demands hands-on experience with data lakes, lakehouse architectures, streaming platforms, and AI/ML infrastructure, while ensuring security, governance, observability,

Qualificações

• Experience designing and operating enterprise-scale data platforms on AWS.
• Strong knowledge of data lakes, lakehouse architectures, and data modeling.
• Proficiency with streaming and event-driven architectures.
• Experience with Terraform and/or AWS CloudFormation.

Responsabilidades

• Design and implement scalable data platforms on AWS to support analytics, ML, and Generative AI workloads.
• Build and maintain batch and real-time data ingestion, transformation, and processing pipelines.
• Develop and optimize ETL/ELT workflows using cloud-native AWS services.
• Design and manage Data Lakes and Lakehouse architectures for high availability and performance.
• Implement data integration across systems using APIs, event-driven and streaming tech.
• Develop orchestration and workflow automation for complex data pipelines.
• Enable AI/ML workloads by building data foundations for training and inference.
• Implement monitoring, logging, alerting, and observability for platform reliability.
• Establish data governance, security access control, lineage, and compliance standards.
• Drive Infrastructure as Code and CI/CD best practices.
• Collaborate with data scientists, AI engineers, software engineers, and stakeholders.
• Continuously optimize cloud resources and data workloads with FinOps principles.

Conhecimentos

AWS data platforms

S3 & AWS services

ETL/ELT pipelines

Data lakes/lakehouse

Streaming architectures

Terraform / IaC

Cloud security / IAM

CI/CD for data

Monitoring & observability

Cost optimization / FinOps

Descrição da oferta de emprego

Avenue Code is the leading software consultancy focused on delivering end-to-end development solutions for digital transformation across every vertical. We’re privately held, profitable, and have been on a solid growth trajectory since day one. We care deeply about our clients, our partners, and our people. We prefer the word ‘partner’ over ‘vendor’, and our investment in professional relationships is a reflection of that philosophy. We pride ourselves on our technical acumen, our collaborative problem-solving ability, and the warm professionalism of our teams.

Avenue Code has been believing in and promoting plurality actions for over 10 years, understanding that recognizing differences and fostering a safe environment, employment opportunities, representation, and support are the best ways to promote an increasingly equitable culture.

Senior Data & AI Platform Engineer

This professional will be responsible for designing, building, and operating scalable cloud‑native data platforms on AWS. They will develop robust data architectures, ingestion and processing pipelines, and AI‑ready platforms that support analytics, machine learning, and Generative AI initiatives across the organization. The ideal candidate combines strong expertise in modern data engineering with hands‑on experience building data lakes, lakehouse architectures, streaming platforms, and AI/ML infrastructure while ensuring security, governance, reliability, observability, and cost optimization.

Responsibilities

• Design and implement scalable data platforms and architectures on AWS to support analytics, machine learning, and Generative AI workloads.
• Build and maintain batch and real‑time data ingestion, transformation, and processing pipelines.
• Develop and optimize ETL/ELT workflows using cloud‑native AWS services.
• Design and manage Data Lakes and Lakehouse architectures, ensuring high availability, scalability, and performance.
• Implement data integration solutions across internal and external systems using APIs, event‑driven architectures, and streaming technologies.
• Develop orchestration and workflow automation solutions for complex data processing pipelines.
• Enable AI and ML workloads by building and maintaining data foundations for model training, inference, and Generative AI applications.
• Implement monitoring, logging, alerting, and observability practices to ensure platform reliability and operational excellence.
• Establish and maintain data governance, security access control, lineage, and compliance standards.
• Drive infrastructure automation through Infrastructure as Code (IaC) and CI/CD best practices.
• Partner with Data Scientists, AI Engineers, Software Engineers, and business stakeholders to deliver scalable data solutions.
• Continuously optimize cloud resources and data workloads, applying FinOps principles to improve efficiency and reduce costs.

Required Qualifications

• Strong experience designing and operating enterprise‑scale data platforms on AWS.
• Hands‑on experience with Amazon S3, AWS Glue, AWS Lambda, Amazon EMR, Amazon Athena, Amazon Redshift, Amazon RDS, Amazon DynamoDB, Amazon Kinesis, Amazon MSK (Kafka), Amazon EventBridge, AWS Step Functions, Amazon API Gateway, Amazon ECS and/or Amazon EKS.
• Experience building ETL/ELT pipelines and modern data architectures.
• Strong knowledge of Data Lakes, Lakehouse architectures, and data modeling concepts.
• Experience with streaming and event‑driven architectures.
• Experience implementing infrastructure automation using Terraform and/or AWS CloudFormation.
• Strong understanding of cloud security, IAM, secrets management, and governance frameworks.
• Experience implementing monitoring and observability solutions using Amazon CloudWatch and related tools.
• Solid understanding of CI/CD practices for data and platform engineering environments.

Avenue Code reinforces its commitment to privacy and to all the principles guaranteed by the most accurate global data protection laws, such as GDPR, LGPD, CCPA and CPRA. The candidate data shared with Avenue Code will be kept confidential and will not be transmitted to disinterested third parties, nor will it be used for purposes other than the application for open positions. As a consultancy company, Avenue Code may share your information with its clients and other companies from the AI/R Group to which Avenue Code’s consultants are allocated to perform its services.