Data Engineer

Tixy; United States; Presencial; USD 140.000 - 190.000; Jornada completa

Descripción de la vacante

Tixy is seeking a Senior Databricks Engineer with healthcare domain experience to design, develop, and optimize scalable data pipelines on Databricks and cloud platforms. You will work with healthcare data such as claims, member and provider data, implementing data quality and governance while collaborating with data architects and stakeholders.

Ideal candidates have hands-on Databricks, Spark, PySpark, and SQL skills, plus Delta Lake expertise and knowledge of HIPAA privacy requirements.

Formación

• 5+ years of data engineering experience in a healthcare setting.
• Hands-on Databricks, Spark, PySpark, and SQL expertise.
• Experience with Delta Lake and Lakehouse concepts.
• Knowledge of HIPAA and data privacy requirements.
• Cloud platform experience across Azure/AWS/GCP is preferred.

Responsabilidades

• Design, develop and maintain scalable data pipelines on Databricks.
• Build and optimize ETL/ELT pipelines for healthcare data.
• Develop data solutions using Delta Lake and Lakehouse architecture.
• Work with healthcare datasets such as claims and member data.
• Implement data quality, monitoring, and security controls.
• Collaborate with data scientists, architects, and stakeholders.
• Participate in CI/CD and Databricks governance practices.

Conocimientos

Databricks

Apache Spark

PySpark

SQL

Delta Lake

Cloud platforms

HIPAA knowledge

Healthcare domain experience

Educación

Bachelor's degree

Herramientas

Databricks Workflows

Databricks Jobs

Databricks notebooks

Azure Data Lake Storage

Azure Synapse

Terraform / IaC

Descripción del empleo

We are looking for a Senior Databricks Engineer with strong healthcare domain experience to design, develop, and optimize scalable data engineering solutions using Databricks and modern cloud technologies. The ideal candidate will have hands-on experience with Databricks, Apache Spark, PySpark, SQL, and cloud data platforms, along with a solid understanding of healthcare data and workflows.

AI Skills are mandatory like Claude, Databricks Genie and MLflow

Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks, Apache Spark, and PySpark.
• Build and optimize ETL/ELT pipelines for large and complex healthcare datasets.
• Develop data solutions using Delta Lake, Databricks SQL, and Lakehouse architecture.
• Work with healthcare data such as claims, member, provider, clinical, pharmacy, and patient data.
• Integrate data from multiple sources including APIs, databases, files, and cloud storage.
• Develop and optimize complex SQL queries for data transformation and analysis.
• Implement data quality, validation, monitoring, and error-handling processes.
• Work with cloud platforms such as Azure/AWS/GCP, preferably Azure Databricks.
• Collaborate with data architects, analysts, data scientists, and business stakeholders.
• Implement security and governance practices for sensitive healthcare data.
• Troubleshoot performance issues and optimize Spark jobs, Databricks workflows, and data pipelines.
• Participate in code reviews, testing, deployment, and production support.

Required Skills

• 5+ years of experience in Data Engineering.
• Strong hands-on experience with Databricks and Apache Spark.
• Strong programming skills in Python/PySpark.
• Advanced SQL skills.
• Experience with Delta Lake and Databricks Lakehouse architecture.
• Strong experience building ETL/ELT data pipelines.
• Experience with cloud platforms such as Azure, AWS, or GCP.
• Experience with Databricks Workflows, Jobs, notebooks, and clusters.
• Strong understanding of data modeling, data warehousing, and distributed data processing.
• Healthcare domain experience is required.
• Experience working with healthcare datasets such as claims, eligibility, provider, member, clinical, or pharmacy data.
• Understanding of healthcare data privacy and security requirements, including HIPAA.

Preferred Skills

• Experience with Azure Data Lake Storage (ADLS), Azure Data Factory, and Azure Synapse.
• Experience with FHIR, HL7, or other healthcare data standards.
• Experience with Snowflake or other cloud data warehouses.
• Experience with CI/CD and DevOps practices.
• Knowledge of Unity Catalog and Databricks governance.
• Experience with Terraform or infrastructure-as-code.
• Databricks certification is a plus.

Education

Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field preferred.

Key Qualifications

• Databricks + PySpark + SQL + ETL/ELT + Cloud + Healthcare Domain Experience + HIPAA