Data Engineer — Silver Lakehouse Pipelines (Contract)

PLATO; United States; Remote; USD 140,000 - 190,000; Full time

Job summary

PLATO is seeking a contract role in the United States to build and operate Silver-layer data conversion pipelines in a medallion Lakehouse, turning raw Bronze data into cleansed datasets. This role focuses on dependable scheduling, data integrity and performance using Microsoft Fabric and Azure data services.

The contract runs from December 2026 to March 2027 with a strong likelihood of extension. You will configure pipelines, ensure retry mechanisms, implement logging, monitoring and unit

Qualifications

• At least five years of experience with Microsoft Fabric including Data Factory and Lakehouse.
• At least five years of experience using PySpark and Delta Lake for data transformations.
• Experience building and scheduling pipelines with Fabric Data Factory or Azure Data Factory.
• Exposure to Microsoft Purview for data cataloguing and lineage.
• Experience with Azure DevOps, Git and CI/CD pipelines.

Responsibilities

• Develop Silver-layer transformations with Fabric, Azure Data Factory, Spark and Delta Lake.
• Configure pipeline schedules and dependencies, including retry and failure handling.
• Use reusable PySpark libraries and metadata-driven configuration.
• Apply logging, monitoring, error handling and unit-testing practices.
• Maintain data integrity through reconciliation, checkpoints, replay and recovery.
• Tune Spark workloads for runtime and cost efficiency.

Skills

Data engineering

Cloud data platforms

CI/CD practices

Tools

Microsoft Fabric

Azure Data Factory

Spark

Delta Lake

Azure Storage

PySpark

Spark SQL

Purview

Azure DevOps

Git

Job description

PLATO is seeking a contract role in the United States to build and operate Silver-layer data conversion pipelines in a medallion Lakehouse, turning raw Bronze data into cleansed datasets. This role focuses on dependable scheduling, data integrity and performance using Microsoft Fabric and Azure data services.

The contract runs from December 2026 to March 2027 with a strong likelihood of extension. You will configure pipelines, ensure retry mechanisms, implement logging, monitoring and unit