data engineer data integration

EPAM

United States

Remote

USD 120,000 - 180,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

EPAM ищет опытного специалиста по Data Engineering для разработки и оптимизации ETL/ELT пайплайнов, включая пакетные и near‑real-time решения. Ваша задача — проектировать архитектуру, управлять оркестрациями и интеграцией данных, обеспечивая качество и документацию.

Требуется продвинутый опыт в облачных инструментах интеграции, Airflow, SQL/Python/PySpark, а также работа с хранилищами Snowflake/Azure Synapse. Вакансия предполагает удалённую работу в нескольких локациях.

Qualifications

  • 5+ лет опыта в data engineering, warehousing и интеграции данных от начала до конца.
  • Знание облачных инструментов интеграции: Azure Data Factory, AWS Glue или GCP Dataflow.
  • Опыт с Apache Airflow; знакомство с CDC/потоковыми технологиями как Kafka, Debezium.
  • Кодинг на SQL, Python, PySpark/Apache Spark на продакшн.
  • Опыт работы с Snowflake, PostgreSQL, MS SQL Server или Azure Synapse.
  • Понимание OLAP/OLTP, схем Star и Snowflake, Delta Lake.
  • Git и CI/CD для data products.
  • Умение объяснять сложные идеи бизнес-заказчикам и разработчикам.
  • Уровень английского не ниже Upper-Intermediate.
  • Дополнительные навыки: dbt, Docker/Kubernetes, AWS Lambda или Azure Functions.

Responsibilities

  • Проектирование, сборка и оптимизация пакетных и ближневременных ETL/ELT пайплайнов.
  • Разработка и управление оркестрациями рабочих процессов и автоматизацией загрузки данных.
  • Проектирование и реализация слоёв хранилища данных и озера данных с разбиением, индексированием и SCD Type 2.
  • Установка и контроль качества данных и фреймворков валидации.
  • Определение требований к данным, анализ ограничений и архитектурные решения Source-to-Target mappings.
  • Поддержка чистого, модульного репозитория кода.
  • Лидерство в код-ревью и соблюдение инженерных стандартов.
  • Настройка CI/CD пайплайнов с Docker контейнерами.
  • Доставка технических спецификаций, документации по метаданным и словарей данных.

Skills

Data engineering
Data warehousing
End-to-end data integration
SQL
Python
PySpark / Apache Spark
OLAP/OLTP
Star/Snowflake schemas
Delta Lake/Lakehouse
Git
CI/CD pipelines
English communication
Airflow
dbt (nice to have)
ClickHouse
PostgreSQL
MS SQL Server
Azure Synapse
Delta Lake

Tools

Azure Data Factory
AWS Glue
GCP Dataflow
Apache Airflow
Kafka
Debezium
Snowflake
ClickHouse
PostgreSQL
MS SQL Server
Azure Synapse
Docker
Kubernetes
AWS Lambda
Azure Functions
RESTful APIs

Job description

Описание:

EPAM develops enterprise software products, open source solutions, and technology accelerators.

Задачи:
  • Design, build, and optimize scalable batch and near-real-time ETL/ELT pipelines;
  • Develop and manage complex workflow orchestrations and automate ingestion routines;
  • Design and implement data warehouse and lakehouse layers with partitioning, indexing, and SCD Type 2 patterns;
  • Establish data quality checks and validation frameworks;
  • Define data requirements, analyze technical constraints, design Source-to-Target Mappings, and make architectural decisions;
  • Maintain a clean, modular code repository;
  • Lead code reviews and enforce engineering standards;
  • Configure CI/CD pipelines with Docker containers;
  • Deliver technical specifications, metadata lineage documentation, architectural diagrams, and data dictionaries.
Требования:
  • 5+ Years of hands-on experience in data engineering, data warehousing, database design, and end-to-end data integration;
  • Advanced knowledge of cloud integration tools such as Azure Data Factory, AWS Glue, or GCP Dataflow;
  • Proficiency with Apache Airflow and exposure to CDC or real-time streaming tools such as Kafka and Debezium;
  • Production-level coding skills in SQL, Python, and PySpark / Apache Spark;
  • Experience with high-performance databases and cloud-native systems such as Snowflake, ClickHouse, PostgreSQL, MS SQL Server, or Azure Synapse;
  • Master-level understanding of OLAP, OLTP, Star and Snowflake schemas, Delta Lake/Lakehouse patterns, and data staging processes;
  • Hands-on experience with Git and automated CI/CD deployment pipelines for data products;
  • Ability to explain complex technical ideas clearly to business stakeholders and developers;
  • Fluency in English at Upper-Intermediate level or higher;
  • Nice to have: deep knowledge of dbt and schema validation practices, Docker or Kubernetes experience, serverless ingestion services using AWS Lambda or Azure Functions and RESTful APIs.
Условия:

Remote work is available in Georgia and four other locations.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

data engineer in data integration
data engineer in data integration

EPAM • United States

On-site
USD 110,000 - 180,000
data engineer for enterprise data products
data engineer for enterprise data products

EPAM • United States

Remote
USD 120,000 - 180,000
Remote Data Engineer - ELT/ETL, Airflow & Snowflake
Remote Data Engineer - ELT/ETL, Airflow & Snowflake

EPAM • United States

Remote
USD 120,000 - 160,000
Data engineer
Data engineer

AgileEngine, LLC. • United States

Hybrid
USD 120,000 - 160,000
Mentorship program
Tech Talks
Growth roadmap
+1
data analyst in data transformation
data analyst in data transformation

EPAM • United States

On-site
USD 120,000 - 180,000
Data engineer
Data engineer

DATAI SERVICES MCHJ • United States

Remote
USD 90,000 - 130,000
Senior Data Integration Engineer (Databricks/Snowflake)
Senior Data Integration Engineer (Databricks/Snowflake)

EPAM Systems • United States

Remote
USD 120,000 - 180,000
Data Engineer
Data Engineer

Sigma Software • United States

Remote
USD 120,000 - 180,000
Data engineer
Data engineer

Amex Development • United States

Remote
USD 150,000 - 210,000
Data engineer
Data engineer

Neotalent Conclusion • United States

Remote
USD 140,000 - 190,000
Remote work option
Continuous training and certifications
Internal mobility program