Hybrid Microsoft Fabric Data Engineer Lead (PySpark)

Sogeti; Phoenix (AZ); Hybrid; USD 120,000 - 160,000; Full time

Job summary

Sogeti in Phoenix, AZ is seeking a Microsoft Fabric Data Engineer to design and optimize data pipelines and lakehouse architectures. The role requires hands-on PySpark, Spark SQL, and Delta Lake work, with optimization of large-scale analytics.

Candidates must reside in the Phoenix area and commit to in-office work 4 days a week (Mon-Thurs) with Fri remote. The position involves collaborating with data architects, analysts, and reporting teams to deliver scalable solutions.

Qualifications

• Hands-on with Microsoft Fabric Data Engineering and Delta Lake optimization.
• Proficient in PySpark, Spark SQL, and Python for large-scale pipelines.
• Familiar with OneLake and Lakehouse architectures.
• Understanding of Medallion Architecture (Bronze/Silver/Gold).
• Experience with Data Factory Pipelines & Dataflows Gen2.
• Strong performance tuning and capacity management skills.

Responsibilities

• Design and develop data pipelines using Microsoft Fabric, PySpark, and Spark SQL.
• Build Bronze, Silver, and Gold lakehouse architectures.
• Optimize Delta Lake tables for large-scale analytics.
• Develop incremental load, CDC, and upsert/merge patterns with PySpark.
• Tune Spark workloads to improve performance and reduce costs.
• Analyze Spark execution plans, shuffles, and memory usage.
• Optimize Fabric capacities and monitor workload performance.
• Collaborate with data architects, analysts, and reporting teams.

Skills

Microsoft Fabric

PySpark

Spark SQL

Python

OneLake

Lakehouse

Medallion Architecture

Data Factory

Performance Tuning

Tools

Notebooks

Delta Lake

Job description

Sogeti in Phoenix, AZ is seeking a Microsoft Fabric Data Engineer to design and optimize data pipelines and lakehouse architectures. The role requires hands-on PySpark, Spark SQL, and Delta Lake work, with optimization of large-scale analytics.

Candidates must reside in the Phoenix area and commit to in-office work 4 days a week (Mon-Thurs) with Fri remote. The position involves collaborating with data architects, analysts, and reporting teams to deliver scalable solutions.