Senior Data Engineer

EPAM Systems Inc

Colombia

On-site

COP 80,000,000 - 120,000,000

Full time

13 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Healthcare benefits
Paid time off
Upskilling & certs
LinkedIn Learning access
Global career opportunities

Job summary

EPAM Systems Inc is seeking a Senior Data Engineer to design and deliver scalable, cloud-native data solutions on Azure Databricks. You will manage large-scale data pipelines, ensure data quality, and collaborate with cross-functional teams to drive analytics and BI initiatives.

The role emphasizes Python, Spark, SQL optimization, and governance practices, with a focus on maintainability, efficiency, and best-practice engineering in a global team.

Qualifications

  • 3+ years of experience in Data Engineering or Software Engineering.
  • Expertise in Azure Databricks, Apache Spark, and PySpark.
  • Proficiency in Spark SQL and building solutions on Azure cloud platforms.
  • Advanced Python with strong SQL development and optimization experience.
  • Background in Unity Catalog and enterprise data governance practices.
  • Familiarity with Git workflows and CI/CD pipelines.
  • Knowledge of Databricks Workflows and Azure Data Factory.
  • Strong analytical, troubleshooting, and problem-solving abilities.
  • Excellent communication and stakeholder management.
  • English proficiency at B2 level or higher.

Responsibilities

  • Design, build, and sustain scalable data ingestion, transformation, and delivery pipelines.
  • Deliver cloud-native data solutions using Azure and Databricks technologies.
  • Create and refine ETL/ELT processes with PySpark, Spark SQL, and Delta Lake.
  • Apply metadata-driven ingestion frameworks to boost scalability and maintainability.
  • Build batch and incremental data processing pipelines.
  • Set up data quality controls, monitoring, and error handling mechanisms.
  • Collaborate with stakeholders, architects, and engineers to capture requirements.
  • Address production issues and drive improvements in platform reliability.
  • Tune Databricks workloads for efficiency, scalability, and cost management.
  • Uphold engineering best practices including code reviews and automated testing.
  • Contribute to data governance, security, and Unity Catalog initiatives.
  • Guide team members and shape engineering standards.

Skills

Python programming
SQL development
Git workflows
CI/CD pipelines
Data governance
Azure cloud
Databricks Workflows
Agile delivery

Tools

Azure Databricks
Apache Spark
PySpark
Spark SQL

Job description

EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

Senior Data Engineer

We are seeking a Senior Data Engineer with proven expertise in Azure Data Platform technologies, Databricks, and PySpark to strengthen our growing data engineering organization. In this position, you will craft and deliver scalable, cloud-native data solutions that drive enterprise analytics, reporting, and business intelligence efforts. You will manage large-scale datasets, develop reusable ingestion frameworks, refine data pipelines, and play a key role in evolving modern data platforms built on Azure and Databricks. The right candidate pairs strong hands-on engineering capabilities with a background in designing robust, metadata-driven data solutions, along with a genuine drive to deliver high-quality, maintainable data products.

Responsibilities
  • Design, build, and sustain scalable data ingestion, transformation, and delivery pipelines
  • Deliver cloud-native data solutions using Azure and Databricks technologies
  • Create and refine ETL/ELT processes with PySpark, Spark SQL, and Delta Lake
  • Apply metadata-driven and configuration-based ingestion frameworks to boost scalability and maintainability
  • Build and sustain batch and incremental data processing pipelines
  • Set up data quality controls, validation frameworks, monitoring, logging, and error handling mechanisms
  • Work closely with business stakeholders, architects, and engineering teams to capture requirements and deliver data solutions
  • Address production issues and push forward continuous improvements in platform reliability and performance
  • Track and fine-tune Databricks workloads for efficiency, scalability, and cost management
  • Uphold engineering best practices including code reviews, automated testing, CI/CD, and infrastructure automation
  • Contribute to data governance, security, and access management initiatives using Unity Catalog and Azure services
  • Guide team members and shape engineering standards and best practices
Requirements
  • 3+ years of experience in Data Engineering or Software Engineering
  • Expertise in Azure Databricks, Apache Spark, and PySpark
  • Proficiency in Spark SQL and a track record of building solutions on Azure cloud platforms
  • Advanced Python programming skills paired with strong SQL development and optimization experience
  • Background in Unity Catalog and enterprise data governance practices
  • Familiarity with Git-based source control workflows
  • Experience designing and maintaining CI/CD pipelines
  • Knowledge of orchestration tools such as Databricks Workflows and Azure Data Factory (ADF)
  • Strong analytical, troubleshooting, and problem-solving skills
  • Excellent communication and stakeholder management abilities
  • English proficiency at B2 level or higher
Nice to have
  • Knowledge of Terraform and Infrastructure as Code (IaC)
  • Experience with Databricks Asset Bundles
  • Skills in building reusable data engineering frameworks and accelerators
  • Expertise in advanced Spark performance tuning and optimization
  • Familiarity with enterprise-scale lakehouse architectures
  • Exposure to data security, compliance, and governance initiatives
  • Experience working within Agile delivery teams
We offer
  • International projects with top brands
  • Work with global teams of highly skilled, diverse peers
  • Healthcare benefits
  • Employee financial programs
  • Paid time off and sick leave
  • Upskilling, reskilling and certification courses
  • Unlimited access to the LinkedIn Learning library and 22,000+ courses
  • Global career opportunities
  • Volunteer and community involvement opportunities
  • EPAM Employee Groups
  • Award-winning culture recognized by Glassdoor, Newsweek and LinkedIn

EPAM is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, age, sexual orientation, gender identity or expression, disability, protected veteran status, or any other characteristic protected by applicable law.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead Data Engineer (Databricks & Spark)
Lead Data Engineer (Databricks & Spark)

EPAM Systems Inc • Colombia

On-site
COP 120,000,000 - 180,000,000
Healthcare benefits
Employee learning programs
Paid time off
Senior Data Software Engineer
Senior Data Software Engineer

EPAM Systems • Colombia

Remote
COP 90,000,000 - 150,000,000
Healthcare benefits
Paid time off
Learning & certification
Lead Data Software Engineer
Lead Data Software Engineer

EPAM Systems Inc • Colombia

On-site
COP 120,000,000 - 180,000,000
Healthcare benefits
Upskilling and certification courses
Global career opportunities
+2
Senior Data Software Engineer
Senior Data Software Engineer

EPAM Systems Inc • Colombia

Remote
COP 120,000,000 - 180,000,000
Healthcare benefits
Professional development
Paid time off
+4
Senior Data Integration Engineer
Senior Data Integration Engineer

EPAM Systems • Colombia

Remote
COP 120,000,000 - 240,000,000
Healthcare benefits
Paid time off
Upskilling and certification courses
+2
Senior Data Engineer – Azure Databricks & PySpark
Senior Data Engineer – Azure Databricks & PySpark

EPAM Systems Inc • Colombia

On-site
COP 80,000,000 - 120,000,000
Healthcare benefits
Paid time off
Upskilling & certs
+2
Lead Data Software Engineer (Python+GCP)
Lead Data Software Engineer (Python+GCP)

EPAM Systems Inc • Colombia

On-site
COP 180,000,000 - 260,000,000
Learning platforms
Health coverage
Stock options
+3
Senior Data Integration Engineer (AI, Python, Databricks)
Senior Data Integration Engineer (AI, Python, Databricks)

EPAM Systems • Colombia

Remote
COP 120,000,000 - 180,000,000
Senior Data Scientist
Senior Data Scientist

EPAM Systems Inc • Colombia

Remote
COP 120,000,000 - 180,000,000
Healthcare benefits
Paid time off
Upskilling and certification courses
+5
Lead Data Quality Engineer
Lead Data Quality Engineer

EPAM Systems Inc • Colombia

On-site
COP 180,000,000 - 210,000,000
Learning platforms
Health coverage
Vision benefits
+6