Senior Data Software Engineer

EPAM Systems Inc; Colombia; Remote; COP 90,000,000 - 150,000,000; Full time

Benefits offered by this job

Healthcare benefits

Paid time off

Learning & certification

Job summary

EPAM Systems Inc. is seeking a Senior Data Software Engineer to design scalable data pipelines and dependable data flows that power critical business decisions. You will collaborate with a team focused on clean, thoroughly tested code and disciplined engineering practices.

The role offers opportunities to work on large-scale data projects, using PySpark, Delta Lake, and Databricks, while contributing to data governance and quality across multiple platforms.

Qualifications

• 3+ years of hands-on experience with PySpark and Spark SQL.
• Experience building Lakeflow pipelines and data orchestration.
• Experience with Delta Lake and Unity Catalog for storage and governance.
• Experience with JDBC/CDC ingestion (DB2, SQL Server).
• Experience writing unit tests with pytest.
• Git and deployment workflows.

Responsibilities

• Architect and construct scalable data pipelines across platforms.
• Build and sustain data ingestion workflows from multiple sources.
• Produce clean, well-tested code grounded in solid software engineering.
• Collaborate with cross-functional teams to translate data needs into technical solutions.
• Monitor data pipeline performance and resolve issues to ensure dependability.
• Participate in code reviews to maintain quality and share knowledge.
• Diagnose and fix data quality problems to ensure accuracy.
• Document technical workflows and solutions for onboarding and knowledge sharing.
• Engage in agile sprint planning and related team activities.

Skills

PySpark

Spark SQL

Data pipelines

Unit testing

Git

English (B2+)

Tools

Auto Loader

JDBC/CDC

Delta Lake

Unity Catalog

Databricks

DataStage

Job description

We're on the lookout for a Senior Data Software Engineer to come aboard our team. This position is a great match for someone who takes pleasure in architecting durable data solutions that power important business choices. You'll collaborate closely with a group that prioritizes clean, thoroughly tested code and disciplined engineering approaches. For anyone energized by tackling large-scale data problems, this role provides ample opportunity for both growth and meaningful contributionResponsibilitiesArchitect and construct scalable data pipelines that enable dependable data flow across multiple platformsBuild and sustain data ingestion workflows pulling from a range of different source systemsProduce clean, high-performing, thoroughly tested code grounded in solid software engineering principlesPartner with teams across different functions to understand data needs and shape them into working technical solutionsKeep watch over data pipeline performance, stepping in ahead of time to resolve issues and preserve system dependabilityJoin in on code review sessions to help maintain quality benchmarks and spread knowledge throughout the teamDiagnose and fix data quality problems, safeguarding accuracy and consistency across all datasetsHelp drive continuous enhancement of data architecture practices and engineering standardsCreate documentation covering technical workflows and solutions to aid knowledge sharing and new team member onboardingOperate within an agile framework, taking part in sprint planning sessions and other recurring team activitiesRequirementsAt least 3 years of relevant hands-on professional backgroundSolid practical expertise with PySpark and Spark SQL for constructing and processing large-scale data workflowsBackground building and overseeing Lakeflow pipelines and Jobs to support data orchestrationSkilled in using Auto Loader along with JDBC/CDC ingestion approaches, working with sources like DB2 and SQL ServerStrong background with Delta Lake and Unity Catalog for handling data storage and governance needsExperience maintaining data quality standards and performing source reconciliation to ensure dependable, accurate datasetsBackground writing unit tests through pytest to confirm code correctness and reliabilityFamiliarity with Git and Asset Bundles for managing version control and deployment workflowsUpper-intermediate English language skills or higher (B2)Nice to haveBackground working on DataStage migration initiativesExperience extracting data from Dynamics 365/DataverseHolding a Databricks certificationWe offerInternational projects with top brandsWork with global teams of highly skilled, diverse peersHealthcare benefitsEmployee financial programsPaid time off and sick leaveUpskilling, reskilling and certification coursesUnlimited access to the LinkedIn Learning library and 22,000+ coursesGlobal career opportunitiesVolunteer and community involvement opportunitiesEPAM Employee GroupsAward-winning culture recognized by Glassdoor, Newsweek and LinkedInEPAM is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, age, sexual orientation, gender identity or expression, disability, protected veteran status, or any other characteristic protected by applicable law.