Senior Databricks Engineers

EXL Talent Acquisition Team; Austin (TX); À distance; USD 120 000 - 180 000; Plein temps

Résumé du poste

Databricks is seeking an experienced Databricks Data Engineer to join a remote, USA-based healthcare analytics team. The role focuses on designing and implementing scalable data platforms on Databricks and cloud platforms, delivering analytics, reporting, and operational decisions.

The candidate should have hands-on Databricks experience, strong PySpark/Python/SQL skills, and knowledge of data governance and modern data architecture.

Qualifications

• Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field.
• Minimum 5 years of experience in data engineering, data architecture, or related technical roles.
• Hands-on experience with Databricks in enterprise environments.
• Strong knowledge of PySpark, Spark SQL, Python, and SQL.
• Experience designing and implementing Lakehouse architectures.
• Experience building scalable ETL/ELT data pipelines.
• Strong understanding of data modeling, data warehousing, and analytics platforms.
• Experience working with structured and unstructured data.
• Strong problem-solving and communication skills.
• Ability to work independently in a remote environment.
• Experience implementing data governance and security frameworks.

Responsabilités

• Design and implement scalable data solutions using Databricks Lakehouse architecture.
• Develop and optimize batch and real-time data pipelines for large-scale data processing.
• Lead technical solution design for data integration, transformation, and analytics workloads.
• Collaborate with business stakeholders, data engineers, analysts, and product teams to translate requirements into technical solutions.
• Implement data governance, security, and data quality best practices using Unity Catalog and related technologies.
• Optimize performance, reliability, and cost efficiency of Databricks workloads.
• Support data warehouse modernization and cloud migration initiatives.
• Create architecture documentation, technical standards, and design guidelines.
• Troubleshoot complex data platform issues and provide technical recommendations.
• Stay current with Databricks and cloud platform capabilities and drive adoption of best practices.

Connaissances

Databricks

PySpark

Spark SQL

Python

SQL

Lakehouse architectures

ETL/ELT pipelines

Data modeling

Data warehousing

Analytics platforms

Formation

Bachelor's degree in Computer Science/Information Systems/Engineering/related field

Outils

Databricks

Unity Catalog

Azure

AWS

GCP

Description du poste

Databricks Data engineer (Remote, USA)

We are seeking an experienced Databricks Solution Architect to join our growing healthcare analytics team. This is a full-time, remote position based in the United States. The ideal candidate will have strong hands-on experience designing and implementing scalable data solutions on Databricks and cloud platforms, with a deep understanding of healthcare data and modern data architecture.

This role is an individual contributor position responsible for solution design, technical leadership, and delivering high-quality data platforms that support analytics, reporting, and operational decision-making.

Key Responsibilities

• Design and implement scalable data solutions using Databricks Lakehouse architecture.


• Develop and optimize batch and real-time data pipelines for large-scale data processing.


• Lead technical solution design for data integration, transformation, and analytics workloads.


• Collaborate with business stakeholders, data engineers, analysts, and product teams to translate requirements into technical solutions.


• Implement data governance, security, and data quality best practices using Unity Catalog and related technologies.


• Optimize performance, reliability, and cost efficiency of Databricks workloads.


• Support data warehouse modernization and cloud migration initiatives.


• Create architecture documentation, technical standards, and design guidelines.


• Troubleshoot complex data platform issues and provide technical recommendations.


• Stay current with Databricks and cloud platform capabilities and drive adoption of best practices.

Required Qualifications

• Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field.


• Minimum 5 years of experience in data engineering, data architecture, or related technical roles.


• Hands-on experience with Databricks in enterprise environments.


• Strong knowledge of PySpark, Spark SQL, Python, and SQL.


• Experience designing and implementing Lakehouse architectures.


• Experience building scalable ETL/ELT data pipelines.


• Strong understanding of data modeling, data warehousing, and analytics platforms.


• Experience working with structured and unstructured data.


• Strong problem-solving and communication skills.


• Ability to work independently in a remote environment.


• Experience implementing data governance and security frameworks.

Preferred Qualifications

• Experience in the healthcare industry, including claims, clinical, provider, member, or revenue cycle data.


• Experience with Azure, AWS, or GCP cloud platforms.


• Exposure to machine learning and AI/ML workloads on Databricks.

Certifications

Required:

• Databricks Certified Data Engineer Associate


• Databricks Certified Data Engineer Professional

Preferred:

• Cloud platform certifications (Azure, AWS, or GCP)