Senior Data Eng: Lakehouse & Real-Time Data (Databricks)

Tech Mirrors; Boston (MA); Hybrid; USD 100,000 - 130,000; Full time

Job summary

Tech Mirrors is seeking a highly experienced Senior Data Engineer / Data Architect with deep expertise in Databricks, Snowflake, and Azure cloud data platforms. The ideal candidate will implement scalable data pipelines, Lakehouse architectures, and real-time data processing solutions, especially in Life Sciences or Healthcare sectors.

The role involves designing data engineering pipelines, optimizing Spark workloads, and managing CI/CD processes. Knowledge of cloud-native data practices is essential, as well as collaboration with cross-functional teams to support analytics and reporting needs.

Qualifications

• Extensive experience designing and implementing scalable data pipelines.
• Strong proficiency in Spark (PySpark) and Delta Lake.
• Background in regulated domains such as Life Sciences or Healthcare.

Responsibilities

• Design and implement end-to-end data engineering pipelines.
• Develop scalable ETL/ELT pipelines using PySpark and Spark SQL.
• Optimize Spark workloads for improved performance.

Skills

Databricks

Snowflake

Azure

PySpark

Spark SQL

Tools

Talend

Azure DevOps

Jenkins

Git

Job description

Tech Mirrors is seeking a highly experienced Senior Data Engineer / Data Architect with deep expertise in Databricks, Snowflake, and Azure cloud data platforms. The ideal candidate will implement scalable data pipelines, Lakehouse architectures, and real-time data processing solutions, especially in Life Sciences or Healthcare sectors.

The role involves designing data engineering pipelines, optimizing Spark workloads, and managing CI/CD processes. Knowledge of cloud-native data practices is essential, as well as collaboration with cross-functional teams to support analytics and reporting needs.