Lead Data Software Engineer

EPAM Systems, Inc.; United States; Remote; USD 140,000 - 190,000; Full time

Benefits offered by this job

Healthcare benefits

Paid time off

Job summary

EPAM Systems, Inc. is seeking a Lead Data Software Engineer to drive delivery and shape data architecture across critical projects. You will lead teams, collaborate with stakeholders, and ensure scalable ETL/ELT pipelines using Spark, Kafka, and NoSQL.

You will guide technical direction, write user stories, coordinate across disciplines, and establish a delivery governance framework while maintaining quality and reliability in a fast-paced environment.

Qualifications

• 5+ years of data software engineering experience with leadership.
• Proficient in Python for data engineering and Spark-based processing.
• Strong SQL and data modeling skills, familiar with streaming data.
• Experience with Databricks, Kafka, and NoSQL data stores.

Responsibilities

• Develop and maintain ETL/ELT pipelines and large-scale data flows.
• Lead delivery, coordinate with stakeholders, and drive technical direction.
• Ensure scalable, reliable data solutions across projects.
• Write user stories and acceptance criteria in agile processes.

Skills

Python

SQL

Apache Spark

Databricks

Scala

Java

PySpark

Kafka

NoSQL

Tools

Spring Boot

ReactJS

NoSQL databases

Kafka

Job description

We are looking for a Lead Data Software Engineer to join our company. This position brings together hands-on technical work and leadership responsibility, offering the chance to shape data architecture while driving delivery on critical projects. You will work closely with stakeholders and engineering teams to make sure solutions stay reliable and scalable as demands evolve. If you enjoy pairing hands-on technical involvement with accountability for delivery success, this role could be a strong fit for you.ResponsibilitiesDevelop and preserve the codebase supporting ETL and ELT pipelines, together with large batch, micro-batch, and streaming systemsSet up the infrastructure required to pull, transform, and load data efficiently from a variety of sources using ADF, Spark, Kafka, or comparable technologiesIdentify, plan, and roll out internal process improvements, such as automating manual tasks, improving data delivery, and modifying infrastructure to increase scalabilityArchitect and release data services solutions using technologies such as Spring Boot, ReactJS, NoSQL, and other related UI and API toolsMaintain governance across delivery management and production processes, aligned with the chosen delivery modelAct as the key contact for delivery-related matters with senior leadership and stakeholders, covering escalations, upsells, and ramp-downsProvide technical direction throughout the delivery process, keeping the architecture solid and forward-thinking while ensuring implementation meets quality expectationsWrite user stories and their corresponding acceptance criteria as part of an agile/scrum processCoordinate work across multiple disciplines and stakeholders to keep delivery progressingEnsure project delivery aligns with customer processes and methodologies, applying strong knowledge of different delivery models with an agile-focused approachEstablish a continuous delivery risk management framework that enables proactive decision-making throughout the delivery lifecycleMonitor and enhance delivery productivity over timeServe as a technical resource for Data Engineers, helping ensure their work meets quality and timeliness standardsOversee production support and deployment responsibilitiesRequirementsA minimum of 5 years of experience in a similar positionAt least one year of experience leading and managing teamsSolid command of core Python for data engineering purposesProficient in SQL for querying and processing dataHands-on experience with Apache Spark for large-scale data processingFamiliarity with Anthropic Claude CodeExperience using Databricks for data engineering and analytics tasksExcellent English proficiency (B2 level or higher)Nice to haveExperience with Apache Kafka for managing streaming dataWorking knowledge of JavaHands-on experience with PySparkFamiliarity with ScalaExperience with Spark StreamingWe offerInternational projects with top brandsWork with global teams of highly skilled, diverse peersHealthcare benefitsEmployee financial programsPaid time off and sick leaveUpskilling, reskilling and certification coursesUnlimited access to the LinkedIn Learning library and 22,000+ coursesGlobal career opportunitiesVolunteer and community involvement opportunitiesEPAM Employee GroupsAward-winning culture recognized by Glassdoor, Newsweek and LinkedIn