Data software engineer distributed systems

EPAM; United States; Remote; USD 140,000 - 190,000; Full time

Benefits offered by this job

Certifications (GCP, Azure, AWS)

Free English classes

Global relocation opportunities

Well-being programs

Job summary

EPAM ищет опытного Lead Data Software Engineer для разработки и поддержки крупных распределённых конвейеров данных на базе Apache Spark, Kafka и Airflow. Ваша задача — трансформировать бизнес-требования в эффективные технические решения и обеспечивать качество на протяжении всего SDLC.

Кандидат должен иметь сильный фокус на производительность, масштабируемость и сотрудничество с跨-функциональными командами. Работа предполагает гибридную/удалённую форму с взаимодействием с глобальными клиентами.

Qualifications

• 5+ лет опыта в Data Software Engineering с фокусом на крупномасштабные распределённые системы.
• Опыт руководства командой как технического лидера и выступление в роли технического контакта для клиентов.
• Владение Python и SQL на продвинутом уровне, глубокий опыт с Spark, Kafka и Airflow.
• Опыт работы в облаке (AWS, Azure или GCP) и настройке SDLC/Agile процессов.

Responsibilities

• Проектирование и внедрение надёжных конвейеров данных и ingestion-решений с масштабируемой архитектурой.
• Руководство командой, менторство и организация работ над сроками.
• Ведение технической коммуникации с заказчиками и согласование дорожной карты.
• Code reviews и обеспечение соблюдения стандартов и quality gates.
• Разработка и поддержка документации по проектам.

Skills

Apache Spark

Python

SQL

Kafka

Airflow

Cloud platforms

Team Lead

SDLC / Agile

English

Customer communication

Tools

Databricks

Snowflake

Job description

Описание:

We're looking for a hands-on, experienced Lead Data Software Engineer who has a deep understanding of Apache Spark and large-scale distributed computing. This role covers the end-to-end lifecycle of our data infrastructure, from designing and implementing ingestion solutions to ensuring the long-term reliability and scalability of our data pipelines. The ideal candidate is a collaborative professional who can work with cross-functional teams, translate complex business requirements into high-quality technical solutions, and maintain a rigorous focus on engineering standards and performance.

Задачи:

Design and implementation of robust data ingestion solutions and pipelines using Cloud Native and Big Data technologies, including architecting key system components and selecting the appropriate tech stack to ensure long-term scalability, performance, and alignment with the broader technical roadmap Team leadership through daily guidance and mentorship so the team delivers high-quality, scalable solutions on schedule, along with serving as the primary technical point of contact for customers, translating business visions into actionable technical roadmaps and managing expectations around delivery, feasibility, and progress Full technical stack management, including configuration management, monitoring, debugging, and performance tuning of data solutions Development and maintenance of scalable data pipelines for efficient data processing and analysis Partnership with cross-functional teams and data architects to identify business requirements and translate them into technical specifications Code reviews and development testing leadership and participation to ensure adherence to standards, quality gates, and SDLC best practices Creation and maintenance of detailed technical documentation for all data engineering projects to support transparency and knowledge sharing Timely and efficient troubleshooting and resolution of data-related issues, ensuring all solutions remain high-quality, reliable, and scalable

Требования:

5+ years of experience in Data Software Engineering with a focus on large-scale distributed systems A demonstrated track record of designing modular system components and selecting technology stacks (such as storage formats and processing engines) that ensure long-term performance, scalability, and alignment with the broader technical roadmap Experience as a Team Lead providing technical guidance to ensure high-quality delivery, while acting as a technical voice for customers to bridge the gap between business needs and engineering solutions Proficiency in Python and SQL Extensive hands-on experience with Apache Spark, Kafka, and Airflow Proficiency with major cloud providers (AWS, Azure, or GCP) Practical experience in Apache Spark job performance tuning Strong understanding of the SDLC, quality gates, and Agile methodologies Excellent communication skills with English proficiency at a B2+ level Motivation, independence, and the ability to handle several projects simultaneously with a focus on efficiency Будет плюсом: Knowledge of Databricks or Snowflake (nice to have)

Условия:

Delivering innovative solutions to industry leaders, making a global impact Enjoyable working environment, whether it is the vibrant office or the comfort of your own home Opportunity to work abroad for up to two months per year Relocation opportunities within our offices in 55+ countries Corporate and social events Leadership development, career advising, soft skills and well-being programs Certifications, including GCP, Azure and AWS Unlimited access to EPAM's internal learning database Free English classes with certified teachers Discounts in local language schools, including offline courses for the Uzbek language Monetary bonuses for engaging in the referral program Medical & family care package Four trust days per year (sick leave without a medical certificate) Discounts for fitness clubs, dance schools and sports programs Benefits package (sports activities, a variety of stores and services)