Databricks Data Engineer for Research Pipelines (Cambridge)

TechData Service Company, LLC; Cambridge (MA); On-site; USD 96,000 - 131,000; Full time

Benefits offered by this job

Dental insurance

Health insurance

Job summary

TechData Service Company, LLC in Cambridge, MA seeks an IT Data Engineer for an onsite contract role to build scalable Databricks-based data platforms and automated pipelines. You will collaborate with research and data teams to ensure reproducible, governed workflows and ready-for-analytics data.

Key focus areas include ETL/ELT design, data quality, and data lakehouse modernization using Delta Lake, Databricks Jobs, and workflow automation with Nextflow.

Qualifications

• Bachelor's degree in a technical field or related discipline.
• Minimum 2+ years of data engineering or analytics platform experience.
• Proficient in Python and SQL with cloud and data lake/lakehouse exposure.

Responsibilities

• Design, develop, and maintain scalable data pipelines with Databricks and Spark.
• Build ETL/ELT processes for diverse data sources; ensure data quality and governance.
• Develop Nextflow pipelines for bioinformatics and scientific workloads.
• Collaborate with data scientists and researchers to support analytics initiatives.

Skills

Databricks

Apache Spark

Nextflow

Seqera Platform

Docker

Kubernetes

AWS

S3

Terraform

CloudFormation

Python

SQL

GitHub Actions

GitHub

CI/CD

Data governance

ETL/ELT

Education

Bachelor's degree

Tools

Databricks

Nextflow

Docker

Kubernetes

Terraform

CloudFormation

Job description

TechData Service Company, LLC in Cambridge, MA seeks an IT Data Engineer for an onsite contract role to build scalable Databricks-based data platforms and automated pipelines. You will collaborate with research and data teams to ensure reproducible, governed workflows and ready-for-analytics data.

Key focus areas include ETL/ELT design, data quality, and data lakehouse modernization using Delta Lake, Databricks Jobs, and workflow automation with Nextflow.