IT Data Engineer – BB4352

TechData Service Company, LLC; Cambridge (MA); On-site; USD 96,000 - 131,000; Full time

Benefits offered by this job

Dental insurance

Health insurance

Job summary

TechData Service Company, LLC in Cambridge, MA seeks an IT Data Engineer for an onsite contract role to build scalable Databricks-based data platforms and automated pipelines. You will collaborate with research and data teams to ensure reproducible, governed workflows and ready-for-analytics data.

Key focus areas include ETL/ELT design, data quality, and data lakehouse modernization using Delta Lake, Databricks Jobs, and workflow automation with Nextflow.

Qualifications

• Bachelor's degree in a technical field or related discipline.
• Minimum 2+ years of data engineering or analytics platform experience.
• Proficient in Python and SQL with cloud and data lake/lakehouse exposure.

Responsibilities

• Design, develop, and maintain scalable data pipelines with Databricks and Spark.
• Build ETL/ELT processes for diverse data sources; ensure data quality and governance.
• Develop Nextflow pipelines for bioinformatics and scientific workloads.
• Collaborate with data scientists and researchers to support analytics initiatives.

Skills

Databricks

Apache Spark

Nextflow

Seqera Platform

Docker

Kubernetes

AWS

S3

Terraform

CloudFormation

Python

SQL

GitHub Actions

GitHub

CI/CD

Data governance

ETL/ELT

Education

Bachelor's degree

Tools

Databricks

Nextflow

Docker

Kubernetes

Terraform

CloudFormation

Job description

TechData Service Company, LLC is looking for an IT Data Engineer for an onsite contract role in Cambridge, MA. This position supports Discovery R&D and data science initiatives by building Databricks-based, scalable data platforms and automated scientific pipelines. You will work close to research and data teams to help ensure data workflows are reproducible, governed, and ready for downstream analytics.

How you’ll make an impact

• Design, develop, and maintain scalable data pipelines using Databricks, Apache Spark, and cloud-native technologies.
• Build and optimize ETL/ELT processes across structured, semi-structured, and unstructured datasets.
• Create data ingestion frameworks for research, laboratory, clinical, and external scientific datasets.
• Implement data quality, validation, monitoring, and governance practices.
• Support data lakehouse architecture and participate in data platform modernization efforts.
• Develop and maintain Databricks notebooks, workflows, Delta Live Tables, and Databricks Jobs.
• Apply Medallion Architecture (Bronze, Silver, Gold) to manage the data lifecycle.
• Manage Delta Lake environments, optimizing performance, scalability, and cost.
• Integrate Databricks with cloud-native services and enterprise applications.
• Deploy, configure, and support the Seqera Platform (formerly Nextflow Tower).
• Develop and maintain Nextflow pipelines for bioinformatics, genomics, imaging, AI/ML, and scientific computing workloads.
• Integrate Seqera workflows with AWS infrastructure and compute environments.
• Support containerized workflows using Docker and Kubernetes.
• Design and implement cloud-based data solutions on AWS, including S3 and data lifecycle policies.
• Build Infrastructure-as-Code solutions using Terraform or CloudFormation.
• Implement security controls and access management aligned with enterprise IT standards.
• Partner with data scientists, researchers bioinformaticians, and stakeholders to clarify data requirements and support workflow automation.
• Troubleshoot pipeline failures, performance issues, and workflow bottlenecks; contribute to platform roadmaps and continuous improvement.
• SOPs, and knowledge articles.

What you bring

• Bachelor’s degree in Computer Science, Information Technology, Data Engineering, Bioinformatics, or a related technical field.
• 5+ years of experience in data engineering, cloud engineering, or analytics platform development.
• 3+ years hands‑on experience with Databricks and Apache Spark.
• 2+ years experience with Seqera Platform (Nextflow Tower) and Nextflow workflows.
• Experience supporting scientific research, life sciences, pharmaceutical, biotech, or healthcare environments preferred.
• Strong familiarity with: Databricks Lakehouse Platform, PySpark and Spark SQL, Delta Lake and Delta Live Tables, Databricks Workflows, and Unity Catalog.
• Experience with SQL and Python, plus bioinformatics workflow automation.
• Hands‑on work with Docker and container orchestration via Kubernetes, including high‑performance computing environments.
• AWS required: S3, IAM, EC2, VPC, Lambda.
• Infrastructure-as-Code with Terraform or CloudFormation, plus cloud monitoring and logging tools.
• Practical experience across data lake and lakehouse architectures, ETL/ELT frameworks, data modeling, data cataloging, and governance.
• Experience with API integrations, data quality frameworks, and development practices using GitHub or GitLab.
• Knowledge of CI/CD pipelines and tools such as Jenkins or GitHub Actions, along with Agile and DevOps methodologies.

Compensation and role details

• Location: Cambridge, MA (onsite)
• Job type: contract
• Pay: USD 70 - 95 per hour
• Experience: minimum 2 years

Benefits

• Dental insurance
• Health insurance

Application questions

• Where are you currently located?
• What hourly rate are you seeking on a W2?