Data Engineer W/ Databricks + GCP - Austin

Biorce; Austin (TX); Hybrid; USD 120,000 - 180,000; Full time

Benefits offered by this job

Hybrid work model

MacBook provided

Comprehensive health coverage

Pet-friendly office

Job summary

Biorce, an Austin-based Healthtech, is seeking a Data Engineer to build scalable data pipelines on a Databricks Lakehouse running on Google Cloud. You will collaborate with data scientists and AI engineers to operationalize data for analytics and ML.

The role emphasizes Delta Lake modeling, dbt transformations, governance via Unity Catalog, and cost-efficient, compliant data operations. Hybrid work in Austin with growth in AI platforms.

Qualifications

• 3+ years of professional experience in Data Engineering or related roles.
• Hands-on experience with the Databricks Lakehouse platform (Delta Lake, Spark/PySpark, Databricks Workflows, Unity Catalog).
• Strong SQL and Python for data transformation and automation.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines on Databricks.
• Model data in Delta Lake architecture and build dbt transformations.
• Ingest data from diverse sources into the Lakehouse.
• Collaborate with data scientists to enable model training and inference data flows.
• Ensure data quality, lineage, and governance across systems.
• Optimize SQL/Python/PySpark transforms for performance.

Skills

SQL

Python

Databricks

GCP

Spark/PySpark

Education

Bachelor's degree in CS/Engineering

Master's degree in CS/Engineering

Tools

dbt

Terraform

Kubernetes

Airflow

Job description

Biorce is a pioneering Healthtech company dedicated to revolutionizing drug development through the power of AI. We are passionate about accelerating medical advancements and improving patient outcomes.

Our team comprises seasoned clinical research professionals, data scientists, and AI experts, working collaboratively to bridge the gap between cutting-edge technology and real-world clinical needs.

With an unwavering commitment to revolutionize healthcare, we envision a world where all patients benefit from accelerated and cost-effective access to treatments. Biorce is poised to redefine the landscape of healthcare, shaping a future where innovation and accessibility converge for the betterment of humanity.

About the company

Biorce is a pioneering Healthtech company dedicated to revolutionizing drug development through the power of AI. We are passionate about accelerating medical advancements and improving patient outcomes.

Our team comprises seasoned clinical research professionals, data scientists, and AI experts, working collaboratively to bridge the gap between cutting-edge technology and real-world clinical needs.

With an unwavering commitment to revolutionize healthcare, we envision a world where all patients benefit from accelerated and cost-effective access to treatments. Biorce is poised to redefine the landscape of healthcare, shaping a future where innovation and accessibility converge for the betterment of humanity.

About the role

Following our successful expansion into the U.S. and continued growth across Europe, we are seeking a Data Engineer to help drive our AI and data engineering from our Austin TX (USA) office. Reporting directly to our Data / AI leadership, this person will play a critical role in driving the development of scalable, reliable, and efficient data pipelines across our cloud data platform.

This is an exciting opportunity to build and optimize the data backbone of Biorce's next-generation platform on a Databricks Lakehouse running on Google Cloud (GCP), combining Databricks-native tooling such as Delta Lake, Unity Catalog, Spark/PySpark, and Databricks Workflows with GCP services like BigQuery, Cloud Storage, and Pub/Sub, and dbt as the transformation layer, all in a high-impact, fast-iterating environment.

Who We're Looking For

We are looking for a skilled Data Engineer to join our growing AI and data team. Someone who can work closely with data scientists, AI engineers, and DevOps to design and operationalize robust data flows that fuel advanced analytics, machine learning, and regulatory-grade insights. This person should be able to shape the evolution of Biorce's data and AI architecture, with the Databricks Lakehouse at its core while ensuring scalable, reliable, compliant, and cost-efficient data operations.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines on Databricks (Spark/PySpark, Databricks Workflows, Delta Live Tables / declarative pipelines, Auto Loader), integrating with GCP services such as Dataflow, Pub/Sub, and BigQuery where appropriate.
• Model data in a Delta Lake medallion architecture (bronze → silver → gold) and build transformations in dbt on Databricks SQL.
• Build and orchestrate complex data ingestion workflows from diverse clinical, research, and third-party sources into the Lakehouse.
• Collaborate with data scientists to enable seamless model training, feature generation, and inference data flows (including integration with MLflow).
• Ensure data quality, integrity, and lineage across all systems through rigorous validation, testing, and monitoring, leveraging Unity Catalog for governance and lineage.
• Develop and optimize SQL, Python, and PySpark transformations to ensure high performance and maintainability.
• Manage data storage, partitioning, clustering, and lifecycle strategies (Delta optimization, liquid clustering) for efficiency and cost control.
• Ensure compliance with SOC 2, ISO 27001, HIPAA, GDPR, and clinical data governance standards in all data operations, using Unity Catalog for access control and auditability.
• Continuously improve internal frameworks for ingestion, metadata management, and data documentation.
• Contribute to cross-functional discussions to shape the evolution of Biorce's Databricks-based data and AI architecture.

Must-Haves

• 3+ years of professional experience in Data Engineering or related roles.

Hands-on experience with the Databricks Lakehouse platform: Delta Lake, Spark/PySpark, Databricks Workflows, and Databricks SQL (Unity Catalog a strong plus).

• Working knowledge of the GCP data ecosystem: BigQuery, Cloud Storage, Pub/Sub, and related services (Dataflow, Composer, Cloud Functions).

Strong proficiency in SQL and Python for data transformation and automation.

• Experience designing batch and streaming data pipelines with scalable and fault-tolerant architectures.
• Familiarity with data modeling, schema design, and Lakehouse/warehouse optimization (medallion architecture, Delta tables).
• Understanding of API-based ingestion, data normalization, and pipeline monitoring.
• Exposure to version-controlled, modular pipeline development, such as Terraform, Databricks Asset Bundles, or GitOps.
• Experience working collaboratively with data scientists and MLOps teams.
• Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field.

Nice-to-Haves

• Production experience with dbt (dbt-Databricks adapter) for complex transformations.
• Familiarity with Delta Live Tables, Auto Loader, Photon, or Databricks performance tuning.
• Experience with clinical, biomedical, or healthcare datasets.
• Familiarity with MLflow, Mosaic AI, Vertex AI, or ML metadata tracking.
• Understanding of data governance and cataloging (Unity Catalog, Data Catalog, Looker, or similar).
• Knowledge of Apache Beam or advanced Spark internals.
• Exposure to infrastructure-as-code (Terraform) and containerized workflows (Kubernetes or Docker).
• Experience implementing data validation frameworks such as Great Expectations, dbt tests, or TFX Data Validation.
• Strong focus on reliability, observability, and continuous improvement of data systems.

Why Join Us?

• A dynamic work environment with an international team, where collaboration and diversity thrive.
• Work alongside top talent, united by a shared purpose and committed to making a real impact.
• Comprehensive private health coverage to ensure your physical and mental well-being.
• Hybrid work model offering flexibility to balance your professional and personal life.
• Company events to celebrate achievements and enjoy time together.
• Get equipped with a MacBook to enhance your productivity and work experience.
• Our office is pet-friendly! You'll likely be greeted by a few wagging tails upon arrival.

--

By submitting this application, I agree that my personal data will be collected, processed, and retained by the company solely for the purposes of managing and assessing my candidacy.