Senior Databricks Engineer

Concentrix; United States; Teletrabalho; USD 150 000 - 230 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Remote / Work at Home

Resumo da oferta

Concentrix is seeking a Senior Databricks Engineer in the United States to own end-to-end data pipelines and lakehouse models using Databricks on Azure, AWS, or GCP, delivering scalable ingestion, transformation, and analytics capabilities.

You will guide other data engineers, establish standards, and partner with stakeholders to ensure production-ready data solutions that balance performance, governance, and cost.

Qualificações

• 10+ years of professional experience in data engineering, data and analytics, or data warehousing.
• 5+ years of hands-on experience with Databricks in a data engineering capacity, including production workloads.
• 5+ years of experience with Python, PySpark, Spark SQL, and data manipulation of large datasets.
• 3+ years of data engineering experience on cloud services such as Azure, AWS, or GCP.

Responsabilidades

• Design, develop, and maintain robust batch and streaming data pipelines.
• Build and manage lakehouse data models and ensure data quality and governance.
• Lead design reviews, establish engineering standards, and guide junior data engineers.
• Collaborate with cross-functional teams to meet business, performance, and analytics requirements.
• Optimize performance and cost across Delta tables, Spark workloads, and cloud services.

Conhecimentos

Data engineering

Databricks

Python

PySpark

SQL

Ferramentas

Databricks

Azure

AWS

GCP

Descrição da oferta de emprego

Job Title: Senior Databricks Engineer Job Description We're Concentrix . The intelligent transformation partner. Solution-focused. Tech-powered. Intelligence-fueled. The global technology and services leader that powers the world's best brands, today and into the future. We're solution-focused, tech-powered, intelligence-fueled. With unique data and insights, deep industry expertise, and advanced technology solutions, we're the intelligent transformation partner that powers a world that works, helping companies become refreshingly simple to work, interact, and transact with. We shape new game-changing careers in over 70 countries, attracting the best talent. The Concentrix Technical Products and Services team is the driving force behind Concentrix 's transformation, data, and technology services. We integrate world-class digital engineering, creativity, and a deep understanding of human behavior to find and unlock value through tech-powered and intelligence-fueled experiences. We combine human-centered design, powerful data, and strong tech to accelerate transformation at scale. You will be surrounded by the best in the world providing market leading technology and insights to modernize and simplify the customer experience. Within our professional services team, you will deliver strategic consulting, design, advisory services, market research, and contact center analytics that deliver insights to improve outcomes and value for our clients. Hence achieving our vision. Our gamechangers around the world have devoted their careers to ensuring every relationship is exceptional. And we're proud to be recognized with awards such as "World's Best Workplaces," "Best Companies for Career Growth," and "Best Company Culture," year after year. Join us and be part of this journey towards greater opportunities and brighter futures.

Senior Databricks Engineer - United States | Remote / Work at Home

Position Summary

The Senior Data Engineer - Databricks will own the end-to-end data lifecycle, from analysis and solution design through implementation, orchestration, governance, and optimization. This role will build modern data solutions on the Databricks Data Intelligence Platform across Microsoft Azure, AWS, or GCP, using Databricks, Python, PySpark, and SQL to deliver scalable ingestion, transformation, modeling, and pipeline orchestration. As a senior engineer, you will lead solution design, establish engineering standards, and guide other data engineers in building reliable, governed, and production-ready data pipelines that support analytics, reporting, and AI/ML use cases.

Responsibilities

• Design, develop, and maintain robust batch and streaming data pipelines for extracting, transforming, and loading data from multiple sources into the Databricks lakehouse on Azure.
• Automate and streamline data ingestion using Auto Loader, Lakeflow Connect, and Azure services such as Azure Data Factory and Event Hubs.
• Build and manage pipelines with Lakeflow Spark Declarative Pipelines, formerly Delta Live Tables, including data quality expectations and change data capture.
• Perform complex data manipulation and transformation using PySpark and Spark SQL to prepare data for advanced analytics, reporting, and AI/ML use cases.
• Design and implement workflow orchestration and scheduling using Lakeflow Jobs, formerly Databricks Workflows, to support reliable and efficient data operations.
• Create scalable lakehouse data models using medallion architecture - bronze, silver, and gold - and dimensional modeling, including SCD Type 1 and Type 2.
• Implement data governance and security in Unity Catalog, including access controls, lineage, row filters, and column masks for sensitive data.
• Optimize performance and cost across Delta tables, Spark workloads, serverless and classic compute, and SQL warehouses.
• Establish CI/CD and testing practices for data pipelines using Git folders, Declarative Automation Bundles, formerly Databricks Asset Bundles, and Azure DevOps or GitHub Actions.
• Monitor pipeline health and data quality using system tables and data quality monitoring; lead root-cause analysis and resolution of production issues.
• Lead design and code reviews, define engineering standards, and mentor data engineers across the team.
• Collaborate with cross-functional teams, including business stakeholders and IT, to ensure data solutions meet business, performance, and project requirements.

Qualifications

Qualifications 10+ years of professional experience in data engineering, data and analytics, or data warehousing. 5+ years of hands-on experience working with Databricks in a data engineering capacity, including production workloads. 5+ years of experience with Python, including manipulating large datasets using PySpark DataFrames, Spark SQL, and/or the pandas API on Spark. 3+ years of data engineering experience on Microsoft Azure, including services such as ADLS Gen2, Azure Data Factory, Key Vault, and Microsoft Entra ID. Strong experience designing and developi