Lead Data Engineer (Databricks & Spark)

EPAM Systems, Inc.; Brasil; Teletrabalho; BRL 180 000 - 280 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Healthcare benefits

Employee financial programs

Paid time off

Resumo da oferta

EPAM Systems, Inc. is seeking a Lead Data Engineer to design scalable ingestion, transformation, and governed data products on Databricks. You will blend hands-on delivery with technical leadership to enable self-service analytics and AI-assisted engineering.

You will lead end-to-end data pipelines, establish best practices, and mentor engineers across teams, delivering trusted datasets and semantic models for reporting and analytics at scale.

Qualificações

• 5+ years of data engineering experience with production-grade platforms.
• Strong Databricks experience in enterprise production environments.
• End-to-end delivery experience across ingestion, transformation, semantic layers, and reporting.
• Advanced Apache Spark and PySpark expertise for scalable pipeline development.
• Advanced SQL skills for transformation, modeling, and performance tuning.

Responsabilidades

• Design scalable data ingestion and transformation frameworks on Databricks.
• Define and enforce Databricks best practices for medallion layers and operational excellence.
• Build batch and streaming pipelines using Apache Spark and PySpark.
• Deliver end-to-end data products from ingestion through governed consumption layers.
• Develop trusted analytical datasets and semantic models for reporting and self-service analytics.

Conhecimentos

Databricks

Apache Spark

PySpark

SQL

Leadership

Ferramentas

Scala

Descrição da oferta de emprego

We are building a next-generation analytics platform and need a Lead Data Engineer to shape scalable ingestion, transformation, and governed data products on Databricks. You will combine hands-on delivery with technical leadership and architecture to enable self-service analytics and AI-assisted engineering.ResponsibilitiesDesign scalable data ingestion and transformation frameworks on DatabricksDefine and enforce Databricks best practices for medallion layers and operational excellenceBuild batch and streaming pipelines using Apache Spark and PySparkDeliver end-to-end data products from ingestion through governed consumption layersDevelop trusted analytical datasets and semantic models for reporting and self-service analyticsImplement data quality, lineage, monitoring, and observability standardsLead proof-of-concepts and evaluate emerging capabilities in the Databricks ecosystemMentor engineers and drive reusable frameworks adopted across teamsRequirements5+ years of data engineering experience with production-grade platformsStrong Databricks experience in enterprise production environmentsLead-level technical leadership skills to drive standards and mentor engineersEnd-to-end delivery experience across ingestion, transformation, semantic layers, and reportingAdvanced Apache Spark and PySpark expertise for scalable pipeline developmentAdvanced SQL skills for transformation, modeling, and performance tuningUpper-Intermediate English proficiency (B2) for leading technical discussionsNice to haveData architecture experience for enterprise lakehouse patternsData solution architecture skills to define scalable platform designsScala proficiency for Spark development and optimizationWe offerInternational projects with top brandsWork with global teams of highly skilled, diverse peersHealthcare benefitsEmployee financial programsPaid time off and sick leaveUpskilling, reskilling and certification coursesUnlimited access to the LinkedIn Learning library and 22,000+ coursesGlobal career opportunitiesVolunteer and community involvement opportunitiesEPAM Employee GroupsAward-winning culture recognized by Glassdoor, Newsweek and LinkedInEPAM is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, age, sexual orientation, gender identity or expression, disability, protected veteran status, or any other characteristic protected by applicable law.