Senior Data Engineer – Azure Databricks & PySpark

EPAM Systems Inc

Colombia

On-site

COP 80,000,000 - 120,000,000

Full time

13 hours ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Healthcare benefits
Paid time off
Upskilling & certs
LinkedIn Learning access
Global career opportunities

Job summary

EPAM Systems Inc is seeking a Senior Data Engineer to design and deliver scalable, cloud-native data solutions on Azure Databricks. You will manage large-scale data pipelines, ensure data quality, and collaborate with cross-functional teams to drive analytics and BI initiatives.

The role emphasizes Python, Spark, SQL optimization, and governance practices, with a focus on maintainability, efficiency, and best-practice engineering in a global team.

Qualifications

  • 3+ years of experience in Data Engineering or Software Engineering.
  • Expertise in Azure Databricks, Apache Spark, and PySpark.
  • Proficiency in Spark SQL and building solutions on Azure cloud platforms.
  • Advanced Python with strong SQL development and optimization experience.
  • Background in Unity Catalog and enterprise data governance practices.
  • Familiarity with Git workflows and CI/CD pipelines.
  • Knowledge of Databricks Workflows and Azure Data Factory.
  • Strong analytical, troubleshooting, and problem-solving abilities.
  • Excellent communication and stakeholder management.
  • English proficiency at B2 level or higher.

Responsibilities

  • Design, build, and sustain scalable data ingestion, transformation, and delivery pipelines.
  • Deliver cloud-native data solutions using Azure and Databricks technologies.
  • Create and refine ETL/ELT processes with PySpark, Spark SQL, and Delta Lake.
  • Apply metadata-driven ingestion frameworks to boost scalability and maintainability.
  • Build batch and incremental data processing pipelines.
  • Set up data quality controls, monitoring, and error handling mechanisms.
  • Collaborate with stakeholders, architects, and engineers to capture requirements.
  • Address production issues and drive improvements in platform reliability.
  • Tune Databricks workloads for efficiency, scalability, and cost management.
  • Uphold engineering best practices including code reviews and automated testing.
  • Contribute to data governance, security, and Unity Catalog initiatives.
  • Guide team members and shape engineering standards.

Skills

Python programming
SQL development
Git workflows
CI/CD pipelines
Data governance
Azure cloud
Databricks Workflows
Agile delivery

Tools

Azure Databricks
Apache Spark
PySpark
Spark SQL

Job description

EPAM Systems Inc is seeking a Senior Data Engineer to design and deliver scalable, cloud-native data solutions on Azure Databricks. You will manage large-scale data pipelines, ensure data quality, and collaborate with cross-functional teams to drive analytics and BI initiatives.

The role emphasizes Python, Spark, SQL optimization, and governance practices, with a focus on maintainability, efficiency, and best-practice engineering in a global team.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer
Senior Data Engineer

EPAM Systems Inc • Colombia

On-site
COP 80,000,000 - 120,000,000
Healthcare benefits
Paid time off
Upskilling & certs
+2
Senior Data Software Engineer: Architect Scalable Pipelines
Senior Data Software Engineer: Architect Scalable Pipelines

EPAM Systems • Colombia

Remote
COP 90,000,000 - 150,000,000
Healthcare benefits
Paid time off
Learning & certification
Lead Databricks & Spark Data Engineer - Platform Architect
Lead Databricks & Spark Data Engineer - Platform Architect

EPAM Systems Inc • Colombia

On-site
COP 120,000,000 - 180,000,000
Healthcare benefits
Employee learning programs
Paid time off
Senior Data Engineer - Azure Databricks Pipelines
Senior Data Engineer - Azure Databricks Pipelines

Applaudo • Medellín

On-site
COP 100,000,000 - 140,000,000
Azure Data Engineer - Databricks, PySpark & TB-Scale ETL
Azure Data Engineer - Databricks, PySpark & TB-Scale ETL

Pyramid Consulting, Inc • Colombia

On-site
COP 226,218,753 - 301,625,004
Lead Data Engineer (Databricks & Spark)
Lead Data Engineer (Databricks & Spark)

EPAM Systems Inc • Colombia

On-site
COP 120,000,000 - 180,000,000
Healthcare benefits
Employee learning programs
Paid time off
Azure Data Engineer
Azure Data Engineer

Pyramid Consulting, Inc • Colombia

On-site
COP 226,218,753 - 301,625,004
Senior Data Integration Engineer: AI-Driven Cloud Pipelines
Senior Data Integration Engineer: AI-Driven Cloud Pipelines

EPAM Systems Inc • Colombia

Remote
COP 120,000,000 - 180,000,000
Healthcare benefits
Paid time off and sick leave
Upskilling, reskilling and certif​ ica
+3
Senior Data Integration Engineer — AI‑Driven Pipelines
Senior Data Integration Engineer — AI‑Driven Pipelines

EPAM Systems Inc • Colombia

Remote
COP 120,000,000 - 180,000,000
Lead Data Integration Engineer (Azure & BI)
Lead Data Integration Engineer (Azure & BI)

EPAM Systems Inc • Colombia

Remote
COP 120,000,000 - 240,000,000
Healthcare benefits
Paid time off
Upskilling and certification courses
+2