Cloud Data Engineer - PySpark

Piper Companies; United States; Remote; USD 110,000 - 140,000; Full time

Benefits offered by this job

Medical Insurance

Dental Insurance

Vision Insurance

401(k)

Paid Time Off

Paid Holidays

Job summary

Zachary Piper Solutions is seeking a Data Cloud Engineer to support a long-term Department of Defense modernization program at Fort Meade, MD. The Data Cloud Engineer will design, build, and maintain scalable cloud-based data solutions supporting enterprise modernization and mission-critical analytics within secure DoD environments.

You will work with Azure Synapse, Databricks, AWS Glue; PySpark; implement data lakes/warehouses; ensure data quality, governance, and compliance with RMF, STIG, and

Qualifications

• 10+ years of experience designing and supporting production-grade cloud data pipelines.
• Strong experience with Azure Synapse, Databricks, Azure Data Factory, and/or AWS data services.
• Advanced experience with PySpark, Spark SQL, Python, and SQL.
• Experience with ETL/ELT, CDC, data lakes/warehouses, data quality, and governance.
• Experience implementing CI/CD for cloud data environments.
• Bachelor's degree in a related technical field.
• Active Secret clearance required.

Responsibilities

• Design and maintain scalable ETL/ELT data pipelines using Azure Synapse, Databricks, AWS Glue, and related cloud technologies.
• Develop large-scale data transformations using PySpark, Spark SQL, Python, and SQL.
• Build data ingestion frameworks for structured and unstructured data, including relational databases, CSV, JSON, and REST APIs.
• Implement full and incremental data loads, CDC, and rerunnable pipeline architectures.
• Design and support cloud-based data lakes, data warehouses, and analytics-ready datasets.
• Implement data quality and governance controls including schema validation, RBAC, lineage, cataloging, and credential management.
• Monitor and troubleshoot data pipelines for performance, failures, latency, logging, and operational reliability.
• Implement CI/CD pipelines supporting automated deployments and environment promotion.
• Partner with cloud and cybersecurity teams to maintain compliance with RMF, STIG, FedRAMP, and DoD security requirements.
• Support cloud data migration, integration, and modernization initiatives across Azure and AWS environments.

Skills

ETL/ELT pipelines

PySpark

Python

SQL

CI/CD

RBAC

Data governance

Cloud data pipelines

Education

Bachelor's degree in a related technical field

Tools

Azure Synapse

Databricks

Azure Data Factory

AWS Glue

Job description

Zachary Piper Solutions is seeking a Data Cloud Engineer to support a long-term Department of Defense modernization program at Fort Meade, MD. The Data Cloud Engineer will design, build, and maintain scalable cloud-based data solutions supporting enterprise modernization and mission-critical analytics within secure DoD environments.

Responsibilities for the Data Cloud Engineer Include:

• Design and maintain scalable ETL/ELT data pipelines using Azure Synapse, Databricks, AWS Glue, and related cloud technologies.
• Develop large-scale data transformations using PySpark, Spark SQL, Python, and SQL.
• Build data ingestion frameworks for structured and unstructured data, including relational databases, CSV, JSON, and REST APIs.
• Implement full and incremental data loads, Change Data Capture (CDC), and rerunnable pipeline architectures.
• Design and support cloud-based data lakes, data warehouses, and analytics-ready datasets.
• Implement data quality and governance controls including schema validation, RBAC, lineage, cataloging, and credential management.
• Monitor and troubleshoot data pipelines for performance, failures, latency, logging, and operational reliability.
• Implement CI/CD pipelines supporting automated deployments and environment promotion.
• Partner with cloud and cybersecurity teams to maintain compliance with RMF, STIG, FedRAMP, and DoD security requirements.
• Support cloud data migration, integration, and modernization initiatives across Azure and AWS environments.

Qualifications for the Data Cloud Engineer Include:

• 10+ years of experience designing and supporting production-grade cloud data pipelines.
• Strong experience with Azure Synapse, Databricks, Azure Data Factory, and/or AWS data services.
• Advanced experience with PySpark, Spark SQL, Python, and SQL.
• Experience with ETL/ELT, CDC, data lakes/warehouses, data quality, and governance.
• Experience implementing CI/CD and automated deployments for cloud data environments.
• Bachelor's degree in a related technical field.
• Active Secret clearance required.

Compensation for the Data Cloud Engineer Includes:

• Salary Range: $110,000 - $140,000 depending on experience.
• Comprehensive Benefits: Medical, Dental, Vision, 401(k), PTO, and Paid Holidays.

Keywords:

#LI-CR2 #LI-REMOTE

Data Cloud Engineer, Cloud Data Engineer, Senior Data Engineer, Data Engineer, Azure Data Engineer, AWS Data Engineer, Big Data Engineer, Data Platform Engineer, Azure, AWS, Azure Synapse, Synapse Analytics, Databricks, Azure Databricks, Azure Data Factory, ADF, AWS Glue, Redshift, Amazon Redshift, S3, PySpark, Apache Spark, Spark SQL, Python, SQL, ETL, ELT, Data Pipeline, Data Pipelines, Data Engineering, Data Lake, Data Warehouse, Data Lakehouse, Cloud Data Platform, Big Data, Data Integration, Data Migration, REST API, JSON, CDC, Change Data Capture, Incremental Loads, Data Quality, Data Governance, Schema Validation, Schema Drift, RBAC, Data Lineage, Data Catalog, CI/CD, DevOps, Oracle, RMF, STIG, FedRAMP, DoD, DISA, Fort Meade, Secret Clearance