Senior Databricks Data Engineer

Bitcruit Solutions; EE. UU.; A distancia; USD 120.000 - 180.000; Jornada completa

Descripción de la vacante

Bitcruit Solutions is seeking a Senior Databricks Data Engineer with deep pharma/scientific data experience to design and optimize Databricks pipelines for clinical, laboratory, and assay data.

The ideal candidate will work directly with scientists and business stakeholders to translate data requirements into scalable solutions, with strong emphasis on data modeling, integration and quality. This is a 6-month remote contract for US-based professionals, starting within 2–3 weeks.

Formación

• Hands-on Databricks engineering experience in pharma/scientific data environments.
• Strong Python, PySpark and SQL coding skills.
• Proven ability to build and maintain data ingestion and transformation pipelines.

Responsabilidades

• Design, develop and optimize data pipelines, transformations, views and data solutions within Databricks.
• Engage with scientists and stakeholders to gather and document requirements.
• Model data for complex pharmaceutical datasets including laboratory, assay and clinical data.
• Establish relationships between compounds, assays, experimental results and identifiers.
• Integrate data from multiple scientific systems, lab sources and external vendors.
• Handle corrected or revised scientific data and maintain data quality through reprocessing pipelines.
• Translate requirements into scalable Databricks solutions and explain design decisions to stakeholders.
• Troubleshoot data quality, integration and modeling issues.

Conocimientos

Databricks

Python

PySpark

SQL

Data pipelines

Data modeling

Data integration

Data quality

Descripción del empleo

Senior Databricks Data Engineer - Pharma / Scientific Data

Location: 100% Remote - US-based candidates only (Onshore)

Working Hours: 100% Pacific Time (PT) business hours

Start Date: Within 2-3 weeks

Duration: 6 months, with strong possibility of extension

Experience Level: Senior

Role Overview

We are seeking a Senior Databricks Data Engineer with strong pharmaceutical/scientific data experience, particularly working with clinical, laboratory and assay data. Client is NOT looking for pre-clinical data experience.

The ideal candidate combines hands-on Databricks engineering expertise with strong business analysis, data modeling and stakeholder engagement skills. This person must be comfortable working directly with scientists, understanding their data requirements and translating them into scalable technical solutions.

Pharma/scientific domain experience is critical. Candidates with strong Databricks skills but limited exposure to scientific data will not be a good fit.

Key Responsibilities

• Design, develop and optimize data pipelines, transformations, views and data solutions within Databricks.
• Work directly with scientists and business stakeholders to gather, clarify and document requirements.
• Design data models and mappings for complex scientific datasets, particularly laboratory, assay and clinical data.
• Establish relationships between scientific entities such as compounds, assays, experimental results and associated identifiers.
• Integrate data from multiple scientific systems, laboratory sources and external vendors.
• Develop processes for handling corrected or revised scientific data, including pipeline reprocessing and maintaining data accuracy.
• Translate scientific and functional requirements into robust, scalable Databricks solutions.
• Demonstrate technical approaches, explain design decisions and guide stakeholders toward appropriate solutions.
• Troubleshoot data quality, integration and modeling issues.

Required Skills & Experience

• Extensive hands-on experience with Databricks.
• Strong Python, PySpark and SQL skills.
• Experience building and maintaining data ingestion and transformation pipelines.
• Strong data modeling, mapping and integration experience.
• Experience managing pipeline reruns, incremental processing, corrections and data quality.

Pharmaceutical / Scientific Data Expertise - Critical

• Proven experience working with pharmaceutical, biotechnology or life sciences data.
• Hands-on experience with laboratory and assay datasets.
• Understanding of scientific data structures, experimental results, compounds and associated identifiers.
• Experience integrating, transforming and modeling scientific data for analytics.
• Familiarity with clinical data is valuable.

Business Analysis & Stakeholder Engagement

• Experience gathering technical and functional requirements directly from scientists or scientific business users.
• Ability to ask relevant questions to uncover business needs, data relationships and quality issues.
• Strong analytical and problem-solving capabilities.
• Excellent verbal and written communication skills.
• Ability to explain technical solutions, trade-offs and recommendations to nontechnical stakeholders.
• Capable of working independently and taking ownership of requirements through implementation.

Preferred Qualifications

• Experience working in pharmaceutical R&D, drug discovery or scientific research environments.
• Familiarity with laboratory data sources and scientific data workflows.
• Experience developing data solutions supporting scientific analysis and decision-making.