Senior Data Software Engineer/ Airflow, Snowflake, AI, dbt

EPAM Systems Inc; United States; Remote; USD 130,000 - 180,000; Full time

Job summary

EPAM Systems is seeking a Senior Data Software Engineer to implement monitoring, incident management, and operational capabilities for an automated Data Platform Monitoring solution. You will design and maintain monitoring rules, incident automation, and operator health views to ensure reliability and observability of data products and pipelines.

Responsibilities include building monitoring for data freshness, volume, and run outcomes; integrating with Jira; and enabling Snowflake connections,

Qualifications

• 3+ years of experience in data engineering roles.
• Expertise in Apache Airflow, Snowflake, and dbt.
• Proficiency in building and maintaining data pipelines, transformation logic, and operational data flows.
• Background in implementing monitoring, observability, and alerting for data platforms.
• Skills in incident automation and integration with tools such as Jira.
• Familiarity with applying AI capabilities within data platform or monitoring solutions.
• Understanding of pipeline health metrics, including freshness, volume, and run outcomes.
• Capability to troubleshoot complex data platform issues and drive continuous improvement.
• English proficiency at an Upper-Intermediate level (B2) or higher.

Responsibilities

• Implement monitoring rules for data pipeline health, freshness, volume, run outcomes.
• Develop data processing and transformation logic for monitoring events.
• Build automated Jira incidents from detected monitoring events.
• Ensure incidents contain component, region, DAG/job, run, timestamp, detecting check, error reference, and severity.
• Support incident recurrence handling and stakeholder notification.
• Develop and maintain operator workspace health view for monitoring data products.
• Enable integration of monitoring data with Snowflake and other components of the solution.
• Participate in testing monitoring rules and incident automation against real and historical failures.
• Troubleshoot issues, improve monitoring logic, and support solution handover.

Skills

Data engineering

Monitoring & observability

Incident automation

Jira integration

AI capabilities in data platforms

Tools

Apache Airflow

Snowflake

dbt

Job description

We are seeking a Senior Data Software Engineer to implement monitoring, incident management, and operational capabilities for an automated Data Platform Monitoring solution. In this role, you will design and maintain robust monitoring rules, incident automation, and operator health views to ensure the reliability and observability of data products and pipelines.

Responsibilities

• Implement monitoring rules for data pipeline health, freshness, volume, run outcomes, and other agreed monitoring objectives
• Develop data processing and transformation logic for monitoring events and operational data
• Build automated Jira incident creation based on detected monitoring events
• Ensure incidents contain the required information, including component, region, DAG/job, run, timestamp, detecting check, error reference, and severity
• Support incident recurrence handling, stakeholder notification, and monitoring of incident status
• Develop and maintain the operator workspace/health view for monitoring data products and pipeline status
• Enable integration of monitoring data with Snowflake and other components of the solution
• Participate in testing monitoring rules and incident automation against real and historical failures
• Troubleshoot issues, improve monitoring logic, and support solution handover

Requirements

• 3+ years of experience in data engineering roles
• Expertise in Apache Airflow, Snowflake, and dbt
• Proficiency in building and maintaining data pipelines, transformation logic, and operational data flows
• Background in implementing monitoring, observability, and alerting for data platforms
• Skills in incident automation and integration with tools such as Jira
• Familiarity with applying AI capabilities within data platform or monitoring solutions
• Understanding of pipeline health metrics, including freshness, volume, and run outcomes
• Capability to troubleshoot complex data platform issues and drive continuous improvement
• English proficiency at an Upper-Intermediate level (B2) or higher