Senior Data Engineer - Azure & MS Fabric, Hybrid SF

KPI Partners; San Francisco (CA); Hybrid; USD 150,000 - 190,000; Full time

Job summary

KPI Partners in San Francisco is seeking a Senior/Lead Data Engineer for a 12-month contract, hybrid work with three onsite days. You will design and build data pipelines and warehousing on Azure native services, PySpark, and Databricks as part of multi-year transformation projects.

Join a high-impact team focused on scalable data platforms, mentoring peers, and delivering robust solutions that improve data reliability, performance, and business insights.

Qualifications

• 10+ years overall IT experience.
• 8+ years data engineering focusing on large-scale solutions.
• 4+ years Azure stack experience (Azure, ADF, Databricks, Functions).
• 1-2 years Microsoft Fabric.
• Proven SQL expertise for large datasets.
• Strong ETL and data warehousing fundamentals.
• Self-motivated and able to thrive in a fast-paced environment.

Responsibilities

• Data Engineering : Design, development, and implementation of data pipelines and solutions using PySpark, SQL, and related technologies.
• Collaboration: Work closely with cross-functional teams to understand business requirements and translate them into robust data solutions.
• Data Warehousing: Design and implement data warehousing solutions, ensuring scalability, performance, and reliability.
• Continuous Learning: Stay up to date with modern technologies and trends in data engineering and apply them to improve our data platform.
• Mentorship: Provide guidance and mentorship to junior data engineers, ensuring best practices in coding, design, and development.

Skills

SQL

PySpark

Azure

Databricks

Microsoft Fabric

ADF

Job description

KPI Partners in San Francisco is seeking a Senior/Lead Data Engineer for a 12-month contract, hybrid work with three onsite days. You will design and build data pipelines and warehousing on Azure native services, PySpark, and Databricks as part of multi-year transformation projects.

Join a high-impact team focused on scalable data platforms, mentoring peers, and delivering robust solutions that improve data reliability, performance, and business insights.