Senior Data Engineer (Airflow)

Capgemini; Mexico; Hybrid; PHP 4,299,754 - 5,528,255; Full time

Benefits offered by this job

Private Health Insurance

Paid Time Off

Flexible work arrangements

Training & Development opportunities

Job summary

Capgemini in Pampanga, Mexico, is seeking a Data Platform Specialist to build and maintain automated workflows using Apache Airflow. The role includes designing optimized data pipelines and collaborating across teams. Ideal candidates will have 4-6 years of experience, a Bachelor's degree in IT or related fields, and must be fluent in English. The position offers competitive pay, flexible work arrangements, and additional benefits like private health insurance and paid time off.

Qualifications

• 4-6 years of experience in a similar role.
• Fluency in English is mandatory.
• Availability for CST or PST time zones.

Responsibilities

• Build and maintain automated data workflows using Apache Airflow.
• Implement end-to-end data pipeline projects.
• Document workflows and solutions for team knowledge sharing.

Skills

Airflow

Python

DynamoDB

Amazon RDS

Snowflake

Jenkins

Education

Bachelor's degree in IT, Information systems, or Computer Science

Tools

Apache Airflow

CI/CD with Jenkins

Job description

Our Client is one of the United States’ largest insurers, providing a wide range of insurance and financial services products with gross written premiums well over US$25 Billion (P&C). They proudly serve more than 10 million U.S. households with more than 19 million individual policies across all 50 states through the efforts of over 48,000 exclusive and independent agents and nearly 18,500 employees. Finally, our Client is part of one the largest Insurance Groups in the world.

What to expect on your journey with us

• A solid and innovative company with a strong market presence
• A dynamic, diverse, and multicultural work environment
• Leaders with deep market knowledge and strategic vision
• Continuous learning and development

Team Function

The new data platforms team will be our centralized shared services team supporting all data platforms such as Snowflake and Astronomer. They will be responsible for the strategy and implementation of these platforms as well as best practices for the business units to follow. In this case, the position is focused on Astronomer/Airflow.

Key Responsibilities

• Airflow-Specific Responsibilities: Build and maintain automated data workflows and orchestrations using Apache Airflow. Build automation processes using Copilot to generate Airflow DAGs.
• Implement at least two major end-to-end data pipeline projects using Airflow.
• Design and optimize complex DAGs for scalability, maintainability, and reliability.
• Create reusable, parameterized, and modular Airflow components (operators, sensors, hooks) to streamline workflow development.
• Ensure effective monitoring, alerting, and logging of Airflow DAGs for quick issue resolution.
• Document workflows, solutions, and processes for team knowledge sharing and training.
• Mentor and support other team members in Airflow usage and adoption.
• Explain best practices, identify pros and cons, and communicate technical decisions to team members.
• Develop reusable frameworks, leveraging reusable concepts for efficiency and scalability.
• Implement and utilize reusable ecosystem components, including Python & Apache Airflow, DynamoDB, Amazon RDS.
• Develop reusable frameworks to enforce data governance and data quality standards.
• CI/CD: CI/CD pipeline development using re-usable frameworks and Jenkins.

Requirements

• Between 4-6 years of experience in a similar role
• Bachelor's degree in IT, Information systems, Computer Science or a related field
• Insurance Experience (Desirable)
• Fluency in English
• Availability to work according to CST or PST time zones.

Technical Skills

• Airflow (MUST) / Astronomer (PLUS) - Advanced (5 Years)
• Python - Advanced (4-6 Years) (MUST)
• Snowflake – Intermediate (PLUS)
• DBT - Entry Level (PLUS)
• AWS Glue - Entry Level (PLUS)
• DynamoDB - Intermediate
• Amazon RDS - Intermediate
• Jenkins - Intermediate

Other Critical Skills

• Work Independently
• Strategic Thinking
• Guide Others
• Documentation
• Communicate Technical Decisions

Please note that resumes must be submitted in English.

This position comes with a competitive compensation and benefits package.

• A competitive salary and performance-based bonuses.
• Flexible work arrangements (remote and/or office-based).
• You will also enjoy a dynamic and inclusive work culture within a globally renowned group.
• Private Health Insurance.
• Paid Time Off.
• Training & Development opportunities in partnership with renowned companies.