Data Engineer W/ Databricks + GCP - Austin

Biorce; Austin (TX); On-site; USD 120,000 - 190,000; Full time

Benefits offered by this job

MacBook provided

Hybrid work model

Job summary

Biorce is seeking a Data Engineer to join its Austin, TX office to build scalable data pipelines and drive AI data engineering on a Databricks Lakehouse platform running on Google Cloud.

You will collaborate with data scientists and DevOps to design robust data flows, ensure quality, security, and cost efficiency in a fast-paced environment.

Qualifications

• 3+ years of professional experience in Data Engineering or related roles.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines on Databricks (Spark/PySpark, Databricks Workflows, Delta Live Tables / declarative pipelines, Auto Loader), integrating with GCP services such as Dataflow, Pub/Sub, and BigQuery where appropriate.
• Model data in a Delta Lake medallion architecture (bronze - silver - gold) and build transformations in dbt on Databricks SQL.
• Build and orchestrate complex data ingestion workflows from diverse clinical, research, and third-party sources into the Lakehouse.
• Collaborate with data scientists to enable seamless model training, feature generation, and inference data flows (including integration with MLflow).
• Ensure data quality, integrity, and lineage across all systems through rigorous validation, testing, and monitoring, leveraging Unity Catalog for governance and lineage.
• Develop and optimize SQL, Python, and PySpark transformations to ensure high performance and maintainability.
• Manage data storage, partitioning, clustering, and lifecycle strategies (Delta optimization, liquid clustering) for efficiency and cost control.
• Ensure compliance with SOC 2, ISO 27001, HIPAA, GDPR, and clinical data governance standards in all data operations, using Unity Catalog for access control and auditability.
• Continuously improve internal frameworks for ingestion, metadata management, and data documentation.
• Contribute to cross-functional discussions to shape the evolution of Biorce's Databricks-based data and AI architecture.

Job description

About the company

Biorce is a pioneering Healthtech company dedicated to revolutionizing drug development through the power of AI. We are passionate about accelerating medical advancements and improving patient outcomes.

Our team comprises seasoned clinical research professionals, data scientists, and AI experts, working collaboratively to bridge the gap between cutting-edge technology and real-world clinical needs.

With an unwavering commitment to revolutionize healthcare, we envision a world where all patients benefit from accelerated and cost-effective access to treatments. Biorce is poised to redefine the landscape of healthcare, shaping a future where innovation and accessibility converge for the betterment of humanity.

About the role

Following our successful expansion into the U.S. and continued growth across Europe, we are seeking a Data Engineer to help drive our AI and data engineering from our Austin TX (USA) office. Reporting directly to our Data / AI leadership, this person will play a critical role in driving the development of scalable, reliable, and efficient data pipelines across our cloud data platform.

This is an exciting opportunity to build and optimize the data backbone of Biorce's next-generation platform on a Databricks Lakehouse running on Google Cloud (GCP), combining Databricks-native tooling such as Delta Lake, Unity Catalog, Spark/PySpark, and Databricks Workflows with GCP services like BigQuery, Cloud Storage, and Pub/Sub, and dbt as the transformation layer, all in a high-impact, fast-iterating environment.

Who We're Looking For

We are looking for a skilled Data Engineer to join our growing AI and data team. Someone who can work closely with data scientists, AI engineers, and DevOps to design and operationalize robust data flows that fuel advanced analytics, machine learning, and regulatory-grade insights. This person should be able to shape the evolution of Biorce's data and AI architecture, with the Databricks Lakehouse at its core while ensuring scalable, reliable, compliant, and cost-efficient data operations.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines on Databricks (Spark/PySpark, Databricks Workflows, Delta Live Tables / declarative pipelines, Auto Loader), integrating with GCP services such as Dataflow, Pub/Sub, and BigQuery where appropriate.
• Model data in a Delta Lake medallion architecture (bronze - silver - gold) and build transformations in dbt on Databricks SQL.
• Build and orchestrate complex data ingestion workflows from diverse clinical, research, and third-party sources into the Lakehouse.
• Collaborate with data scientists to enable seamless model training, feature generation, and inference data flows (including integration with MLflow).
• Ensure data quality, integrity, and lineage across all systems through rigorous validation, testing, and monitoring, leveraging Unity Catalog for governance and lineage.
• Develop and optimize SQL, Python, and PySpark transformations to ensure high performance and maintainability.
• Manage data storage, partitioning, clustering, and lifecycle strategies (Delta optimization, liquid clustering) for efficiency and cost control.
• Ensure compliance with SOC 2, ISO 27001, HIPAA, GDPR, and clinical data governance standards in all data operations, using Unity Catalog for access control and auditability.
• Continuously improve internal frameworks for ingestion, metadata management, and data documentation.
• Contribute to cross-functional discussions to shape the evolution of Biorce's Databricks-based data and AI architecture.

Must-Haves

• 3+ years of professional experience in Data Engineering or related roles.

Hands-on experience with the Databricks Lakehouse platform: Delta Lake, Spark/PySpark, Databricks Workflows, and Databricks SQL (Unity Catalog a strong plus).

• Working knowledge of the GCP data ecosystem: BigQuery, Cloud Storage, Pub/Sub, and related services (Dataflow, Composer, Cloud Functions).

Strong proficiency in SQL and Python for data transformation and automation.

• Experience designing batch and streaming data pipelines with scalable and fault-tolerant architectures.
• Familiarity with data modeling, schema design, and Lakehouse/warehouse optimization (medallion architecture, Delta tables).
• Understanding of API-based ingestion, data normalization, and pipeline monitoring.
• Exposure to version-controlled, modular pipeline development, such as Terraform, Databricks Asset Bundles, or GitOps.
• Experience working collaboratively with data scientists and MLOps teams.
• Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field.

Nice-to-Haves

• Production experience with dbt (dbt-Databricks adapter) for complex transformations.
• Familiarity with Delta Live Tables, Auto Loader, Photon, or Databricks performance tuning.
• Experience with clinical, biomedical, or healthcare datasets.
• Familiarity with MLflow, Mosaic AI, Vertex AI, or ML metadata tracking.
• Understanding of data governance and cataloging (Unity Catalog, Data Catalog, Looker, or similar).
• Knowledge of Apache Beam or advanced Spark internals.
• Exposure to infrastructure-as-code (Terraform) and containerized workflows (Kubernetes or Docker).
• Experience implementing data validation frameworks such as Great Expectations, dbt tests, or TFX Data Validation.
• Strong focus on reliability, observability, and continuous improvement of data systems.

Why Join Us?

• A dynamic work environment with an international team, where collaboration and diversity thrive.
• Work alongside top talent, united by a shared purpose and committed to making a real impact.
• Comprehensive private health coverage to ensure your physical and mental well-being.
• Hybrid work model offering flexibility to balance your professional and personal life.
• Company events to celebrate achievements and enjoy time together.
• Get equipped with a MacBook to enhance your productivity and work experience.
• Our office is pet-friendly! You'll likely be greeted by a few wagging tails upon arrival.

--

By submitting this application, I agree that my personal data will be collected, processed, and retained by the company solely for the purposes of managing and assessing my candidacy.