Sr. Databricks Engineer

Concentrix Corporation; Town of Texas (WI); Hybrid; USD 106,000 - 140,000; Full time

Benefits offered by this job

Medical, dental, vision insurance

401(k) retirement plan

Paid time off and holidays

Paid learning days

Job summary

Concentrix Corporation is seeking a Sr. Databricks Engineer to design and implement end-to-end data solutions on the Databricks platform.

You will drive data ingestion, transformation, and model delivery while guiding a team of data engineers, ensuring governance, performance, and cost optimization across cloud environments. The role requires deep expertise in Python, SQL, Delta Lake, and streaming architectures, with focus on enterprise-grade data solutions and CI/CD practices.

Qualifications

• We require 10+ years in data engineering, analytics, or warehousing.
• Minimum 5+ years working with Databricks in production workloads.
• 5+ years Python with PySpark, Spark SQL, or pandas API on Spark.
• 3+ years in Azure data services (ADLS Gen2, Data Factory, Key Vault).
• Strong SQL, data modeling, and governance skills using Unity Catalog.

Responsibilities

• Design, develop, and maintain batch and streaming pipelines on Databricks lakehouse.
• Lead engineering standards for pipelines and mentor junior engineers.
• orchestrate data workflows with Lakeflow/Databricks Workflows and CI/CD.
• Ensure data governance, security, and lineage in Unity Catalog.
• Optimize Delta tables, Spark workloads, and cost across compute options.
• Collaborate with business and IT to meet performance goals.

Skills

Python

SQL

Delta Lake

Spark

Unity Catalog

Structured Streaming

Auto Loader

Event Hubs / Kafka

CI/CD

Databricks

Education

Bachelor’s degree in Computer Science or related field

Tools

Databricks

Azure Data Factory

Terraform

Power BI

Job description

## Sr. Databricks EngineerApply: USA, TX, Work-at-Home: Full time: Posted Yesterday: End Date: October 16, 2026 (13 days left to apply): R1768838Job Title:Sr. Databricks EngineerJob DescriptionWe're Concentrix. The intelligent transformation partner. Solution-focused. Tech-powered. Intelligence-fueled. The global technology and services leader that powers the world’s best brands, today and into the future. We’re solution-focused, tech-powered, intelligence-fueled. With unique data and insights, deep industry expertise, and advanced technology solutions, we’re the intelligent transformation partner that powers a world that works, helping companies become refreshingly simple to work, interact, and transact with. We shape new game-changing careers in over 70 countries, attracting the best talent. The Concentrix Technical Products and Services team is the driving force behind Concentrix’s transformation, data, and technology services. We integrate world-class digital engineering, creativity, and a deep understanding of human behavior to find and unlock value through tech-powered and intelligence-fueled experiences. We combine human-centered design, powerful data, and strong tech to accelerate transformation at scale. You will be surrounded by the best in the world providing market leading technology and insights to modernize and simplify the customer experience. Within our professional services team, you will deliver strategic consulting, design, advisory services, market research, and contact center analytics that deliver insights to improve outcomes and value for our clients. Hence achieving our vision. Our game-changers around the world have devoted their careers to ensuring every relationship is exceptional. And we’re proud to be recognized with awards such as \"World's Best Workplaces,\" “Best Companies for Career Growth,” and “Best Company Culture,” year after year. Join us and be part of this journey towards greater opportunities and brighter futures. **Title: Sr. Databricks Engineer****Location: Remote in USA** Working hours in EST**Required certification: Databricks Data Engineer Professional** **Role Summary** The Senior Data Engineer is responsible for the end-to-end data lifecycle, including the analysis, design, and implementation of data solutions on the Databricks Data Intelligence Platform running on Microsoft Azure, AWS or GCP. The role leverages Databricks, Python, PySpark, and SQL to deliver data ingestion, transformation, modeling, and pipeline orchestration. As the senior engineer, this role leads solution design, sets engineering standards for how pipelines are built, governed, and deployed, and guides other data engineers. **Responsibilities** * Design, develop, and maintain robust batch and streaming data pipelines for extracting, transforming, and loading (ETL/ELT) data from multiple sources into the Databricks lakehouse on Azure.* Automate and streamline data ingestion using Auto Loader, Lakeflow Connect, and Azure services such as Azure Data Factory and Event Hubs.* Build and manage pipelines with Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables), including data quality expectations and change data capture.* Perform complex data manipulation and transformation using PySpark and Spark SQL, ensuring data is prepared for advanced analytics, reporting, and AI/ML use cases.* Design and implement orchestration and scheduling of data workflows with Lakeflow Jobs (formerly Databricks Workflows) to ensure reliability and efficiency.* Design and implement scalable data models within the lakehouse using medallion architecture (bronze/silver/gold) and dimensional modeling, including SCD Type 1 and Type 2.* Implement data governance and security in Unity Catalog, including access controls, lineage, and row filters and column masks for sensitive data.* Optimize performance and cost across Delta tables, Spark workloads, serverless and classic compute, and SQL warehouses.* Establish CI/CD and testing practices for data pipelines using Git folders, Declarative Automation Bundles (formerly Databricks Asset Bundles), and Azure DevOps or GitHub Actions.* Monitor pipeline health and data quality using system tables and data quality monitoring; lead root-cause analysis and resolution of production issues.* Lead design and code reviews, define engineering standards, and mentor data engineers on the team.* Collaborate with cross-functional teams, including business stakeholders and IT, to ensure data solutions meet business and performance requirements aligned to project goals. **Required Qualifications** **Experience** * 10+ years of professional experience in data engineering, data & analytics, or data warehousing.* 5+ years of hands-on experience working with Databricks in a data engineering capacity, including production workloads.* 5+ years of experience with Python, including manipulating large datasets using PySpark DataFrames, Spark SQL, and/or the pandas API on Spark.* 3+ years of data engineering experience on Microsoft Azure (e.g., ADLS Gen2, Azure Data Factory, Key Vault, Microsoft Entra ID). **Skills and Knowledge** * Expert proficiency in Python and SQL for data engineering, data modeling, transformation, and querying.* Deep knowledge of Delta Lake and Spark performance tuning, including liquid clustering, OPTIMIZE/VACUUM, predictive optimization, partitioning, data skew, and Photon.* Hands-on experience with Unity Catalog for data governance, security, and lineage.* Experience implementing streaming data ingestion in Databricks using Structured Streaming, Auto Loader, and Event Hubs or Kafka.* Strong understanding of testing and deployment strategies for data pipelines, including unit testing, CI/CD, and productionizing ETL pipelines and Databricks SQL queries.* Experience enabling BI consumption through Databricks SQL warehouses and tools such as Power BI. **Education** * Bachelor’s degree in computer science, Information Systems, Data Engineering, or a closely related field, or equivalent professional experience. **Preferred Qualifications** * Active Databricks Certified Data Engineer Professional certification.* Active Databricks Certified Data Engineer Associate certification.* Experience with healthcare data (e.g., claims, EHR/clinical data, HL7/FHIR) and HIPAA/PHI handling.* Infrastructure as code experience with Terraform (Databricks and Azure providers).* Experience preparing data for machine learning and generative AI use cases.* Familiarity with Delta Sharing and Lakehouse Federation. The base salary for this position is $106,087 to $140,000 plus incentives that align with individual and company performance. Actual salaries will vary based on work location, qualifications, skills, education, experience, and competencies. Benefits available to eligible employees in this role include medical, dental, and vision insurance, comprehensive employee assistance program, 401(k) retirement plan, paid time off and holidays and paid learning days.