Senior Data Engineer — Scale & Optimize Data Pipelines

Epam Systems Inc; Buenos Aires; On-site; ARS 92,724,000 - 139,086,000; Full time

Job summary

Endava is seeking a Data Engineer to join our data team. You will design, build, and maintain scalable ETL/ELT pipelines, optimize data models, and contribute to architectural decisions across cloud data platforms.

You will work with Airflow, DBT, SQL, AWS, Snowflake, and CI/CD practices; mentor colleagues and aid production deployments while documenting runbooks and improving system reliability.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines for data ingestion, transformation, validation, and delivery.
• Develop reusable, modular, and configuration-driven data processing solutions.
• Contribute to technical architecture and evaluate solutions based on scalability, performance, reliability, and maintainability.
• Design and optimize data models, SQL queries, stored procedures, and data processing workflows.
• Design and maintain Apache Airflow workflows, including scheduling, dependencies, retries, monitoring, and error handling.
• Develop automated data validation and quality checks using DBT and SQL.
• Implement appropriate monitoring, alerting, and observability for critical data pipelines.
• Troubleshoot and resolve complex data processing and quality issues.
• Develop data solutions using AWS and cloud-based data warehouse technologies such as Snowflake.
• Work with cloud services for data storage, processing, compute, messaging, and serverless workloads.
• Support data platforms across development, QA, staging, and production environments.
• Optimize data workloads for performance, scalability, reliability, and cost.
• Develop and maintain CI/CD pipelines for data engineering applications and infrastructure.
• Apply Infrastructure-as-Code practices using Terraform or equivalent technologies.
• Support production deployments, incident resolution, root cause analysis, and continuous improvement.
• Maintain technical documentation, operational runbooks, and troubleshooting guides.
• Take ownership of complex engineering initiatives from design through production.
• Conduct code reviews and promote engineering best practices.
• Mentor junior and mid-level engineers and provide technical guidance.
• Collaborate with architecture, DevOps, product, and other technology teams.

Job description

Endava is seeking a Data Engineer to join our data team. You will design, build, and maintain scalable ETL/ELT pipelines, optimize data models, and contribute to architectural decisions across cloud data platforms.

You will work with Airflow, DBT, SQL, AWS, Snowflake, and CI/CD practices; mentor colleagues and aid production deployments while documenting runbooks and improving system reliability.