Lead Data Software Engineer

EPAM Systems; Chile; Remote; CLP 68,695,000 - 107,949,000; Full time

Benefits offered by this job

Healthcare benefits

Paid time off & sick leave

Upskilling & certification courses

LinkedIn Learning access

Global career opportunities

Volunteer and community involvement

Job summary

EPAM Systems Inc is seeking a Lead Data Software Engineer to design and implement resilient data pipelines that enable informed business decisions. You will work with cross-functional teams to translate data needs into robust solutions while maintaining high quality and test coverage.

You will lead development efforts, oversee ingestion pipelines sourced from diverse origins, and contribute to code reviews, ensuring data quality and reliability across platforms in an agile setup.

Qualifications

• At least 5 years of relevant practical experience in the field.

Responsibilities

• Build and structure scalable data pipelines across systems.

Skills

PySpark

Spark SQL

Data orchestration

Unit testing (pytest)

Git

Leadership

Agile

English (B2+)

Tools

Delta Lake

Unity Catalog

Auto Loader

JDBC/CDC

DB2

SQL Server

Databricks

Lakeflow

Job description

Our team is expanding, and we're in search of a Lead Data Software Engineer to join us. This position is well-suited for an individual who enjoys engineering resilient data systems that inform key business decisions. You'll work alongside a group committed to producing clean, well-validated code and maintaining disciplined engineering practices. For someone drawn to solving data problems at scale, this role presents considerable opportunity for growth and lasting impactResponsibilitiesBuild and structure scalable data pipelines that ensure reliable data flow across a variety of systemsDevelop and keep up ingestion pipelines that source data from multiple, varied origin systemsWrite clean, high-quality, well-tested code rooted in sound software engineering practicesWork together with cross-departmental teams to understand data needs and convert them into viable technical solutionsObserve data pipeline performance closely, addressing potential issues early to maintain consistent system reliabilityContribute to code review processes to help sustain quality expectations and share insights across the teamInvestigate and correct data quality concerns to maintain dataset accuracy and consistencyPlay a part in continuously refining data architecture methods and engineering standardsCompile documentation covering technical processes and solutions to facilitate knowledge sharing and support new hire onboardingWork inside an agile structure, engaging in sprint planning and other routine team gatheringsRequirementsAt least 5 years of relevant practical experience in the fieldA minimum of one year spent in roles leading and managing development teamsDemonstrated skill with PySpark and Spark SQL for developing and managing large-scale data workflowsHands-on background constructing and supervising Lakeflow pipelines and Jobs for data orchestration purposesProficient with Auto Loader and JDBC/CDC ingestion strategies, including experience with sources like DB2 and SQL ServerStrong familiarity with Delta Lake and Unity Catalog for managing data storage and governance requirementsTrack record maintaining data quality and carrying out source reconciliation to ensure accurate, trustworthy dataBackground developing unit tests using pytest to validate code reliability and correctnessWorking knowledge of Git and Asset Bundles to support version control and deployment managementUpper-intermediate English language skills or higher (B2)Nice to haveExperience participating in DataStage migration effortsBackground extracting data from Dynamics 365/Dataverse systemsDatabricks certification heldWe offerInternational projects with top brandsWork with global teams of highly skilled, diverse peersHealthcare benefitsEmployee financial programsPaid time off and sick leaveUpskilling, reskilling and certification coursesUnlimited access to the LinkedIn Learning library and 22,000+ coursesGlobal career opportunitiesVolunteer and community involvement opportunitiesEPAM Employee GroupsAward-winning culture recognized by Glassdoor, Newsweek and LinkedInEPAM is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, age, sexual orientation, gender identity or expression, disability, protected veteran status, or any other characteristic protected by applicable law.