Senior Data Engineer

Talentify; San Diego (CA); On-site; USD 150,000 - 210,000; Full time

Job summary

Talentify seeks a hands-on Technical Lead to steer the Microsoft Fabric-based data platform. You will architect and operate the data lake-house, mentor engineers, and partner with R&D, Manufacturing, and Supply Chain to deliver scalable data solutions.

You will drive CI/CD, enforce governance, and guide external partners while building semantic layers for Power BI and analytics. Join a leadership team shaping enterprise data strategy.

Qualifications

• Bachelor’s degree in CS, data engineering, IS, or related field.
• 7–10 years of experience in data engineering, data platforms, or analytics infrastructure.
• 2+ years of technical leadership experience with data/analytics engineers.
• Hands-on experience with Microsoft Fabric; Databricks or Azure Synapse preferred.
• Proficiency in PySpark, Python, T-SQL, ETL/ELT, and modern data engineering practices.
• Experience with medallion architecture at enterprise scale.
• Experience with DevOps, CI/CD, Azure DevOps/GitHub, and infrastructure-as-code.
• Knowledge of data governance and Microsoft Purview.
• Experience managing technical delivery from external vendors.

Responsibilities

• Lead the architecture and engineering of the Microsoft Fabric data lake-house, including OneLake, medallion architecture, Spark/SQL, and CI/CD.
• Provide hands-on technical leadership, mentoring, code reviews, and architectural guidance to data and analytics engineers.
• Design and build scalable data pipelines integrating 50+ enterprise source systems.
• Partner with business stakeholders across R&D, Commercial, Manufacturing, Supply Chain, and Corporate functions to translate business needs into data solutions.
• Manage and oversee external implementation partners, ensuring engineering quality and adherence to standards.
• Establish platform support models, SLAs, monitoring, data quality, incident management, and operational processes.
• Define and track platform KPIs such as pipeline reliability, data freshness, query performance, onboarding velocity, and incident resolution.
• Develop the semantic layer, including Power BI datasets, gold-layer views, and self-service analytics.
• Implement data governance, security, classification, RBAC, sensitivity labeling, and audit requirements.
• Establish engineering standards for development, testing, branching, documentation, and deployment.

Skills

Data engineering experience

Technical leadership

Medallion architecture

PySpark

Python

T-SQL

ETL/ELT

Microsoft Fabric

Databricks

Azure Synapse

DevOps

CI/CD

Azure DevOps/GitHub

IaC

Microsoft Purview

Vendor management

Power BI

Education

Bachelor’s degree in Computer Science, Data Engineering, Information Systems, or related field

Tools

Microsoft Fabric

Databricks

Azure Synapse

Power BI

Job description

We are seeking a hands‑on Technical Lead to lead the development and operation of its Microsoft Fabric‑based Platform. This role combines data platform architecture, hands‑on engineering, technical leadership, and business partnership.

Key Responsibilities

• Lead the architecture and engineering of the Microsoft Fabric data lake‑house, including OneLake, medallion architecture, Spark/SQL, and CI/CD.
• Provide hands‑on technical leadership, mentoring, code reviews, and architectural guidance to data and analytics engineers.
• Design and build scalable data pipelines integrating 50+ enterprise source systems.
• Partner with business stakeholders across R&D, Commercial, Manufacturing, Supply Chain, and Corporate functions to translate business needs into data solutions.
• Manage and technically oversee external implementation partners, ensuring engineering quality and adherence to standards.
• Establish platform support models, SLAs, monitoring, data quality, incident management, and operational processes.
• Define and track platform KPIs such as pipeline reliability, data freshness, query performance, onboarding velocity, and incident resolution.
• Develop the semantic layer, including Power BI datasets, gold‑layer views, and self‑service analytics.
• Implement data governance, security, classification, RBAC, sensitivity labeling, and audit requirements.
• Establish engineering standards for development, testing, branching, documentation, and deployment.

Required Qualifications

• Bachelor’s degree in Computer Science, Data Engineering, Information Systems, or related field.
• 7–10 years of experience in data engineering, data platforms, or analytics infrastructure.
• 2+ years of technical leadership experience with data/analytics engineers.
• Strong hands‑on experience with Microsoft Fabric; Databricks or Azure Synapse experience may be considered.
• Strong proficiency in PySpark, Python, T‑SQL, ETL/ELT, and modern data engineering practices.
• Experience with medallion/layered data architecture at enterprise scale.
• Experience with DevOps, CI/CD, Azure DevOps/GitHub, and infrastructure‑as‑code.
• Knowledge of data governance and quality frameworks such as Microsoft Purview.
• Experience managing technical delivery from external vendors.

Preferred

• Biopharma, life sciences, or other regulated‑industry experience, including GxP and validation requirements.
• Experience with Veeva, LIMS, ELN/Benchling, CTMS, EDC, MasterControl, or HRIS integrations.
• Experience with AI/ML workloads on a lake‑house.
• Microsoft Fabric Data Engineering certification.