Senior Data Engineer, Azure Databricks & Data Pipelines

Jobtailor; Massachusetts; On-site; USD 140,000 - 190,000; Full time

Job summary

Jobtailor is seeking a Senior Big Data Engineer in Massachusetts to design and implement end-to-end data pipelines on Azure and Databricks. You will own ETL processes, data ingestion, and warehousing concepts while collaborating in an agile team and following best practices.

The role requires 10+ years in big data, 5+ years in Databricks, and strong cloud experience including Azure, Kubernetes, and Docker. Snowflake exposure is a plus.

Qualifications

• 10+ years of overall Big Data data pipeline experience.
• 5+ years of hands-on Databricks experience.
• 5+ years of cloud-based development experience, including Azure Services, Azure DevOps, Kubernetes, and Docker.
• Bachelor’s Degree in computer or IT-related subject.
• Experience with Snowflake is a plus.

Responsibilities

• Design, build, and test end-to-end data pipelines using Azure, Databricks, and cloud infrastructure.
• Develop ETL processes for data ingestion, transformation, and loading into data lakes.
• Work on Databricks and data warehousing concepts.
• Follow best practices and coding standards.
• Collaborate in an Agile project team.
• Manage full software development lifecycle.

Skills

Databricks

Apache Spark

Scala

SQL

Data Pipelines

Data Quality

Git

Shell Scripting

Unix

Azure

Kubernetes

Docker

Cosmos DB

Snowflake

Delta Lake

Azure DevOps

Harness

TOAD

Education

Bachelor’s Degree in Computer or IT-related Subject

Tools

Databricks

Cosmos DB

Delta Lake

Azure DevOps

Kubernetes

Docker

Snowflake

TOAD

Git

Harness

ADO

Job description

Jobtailor is seeking a Senior Big Data Engineer in Massachusetts to design and implement end-to-end data pipelines on Azure and Databricks. You will own ETL processes, data ingestion, and warehousing concepts while collaborating in an agile team and following best practices.

The role requires 10+ years in big data, 5+ years in Databricks, and strong cloud experience including Azure, Kubernetes, and Docker. Snowflake exposure is a plus.