Cloud Data Engineer - PySpark - 176062

Piper Companies; United States; Remote; USD 110,000 - 140,000; Full time

Benefits offered by this job

Medical Insurance

401(k)

PTO & Holidays

Job summary

Zachary Piper Solutions is seeking a Data Cloud Engineer to support a long-term DoD modernization program at Fort Meade, MD. You will design, build, and maintain scalable cloud-based data solutions for mission-critical analytics in secure DoD environments.

Responsibilities include designing ETL/ELT pipelines with Azure Synapse, Databricks, and AWS Glue; developing PySpark, Spark SQL, Python, and SQL; implementing CDC, data lakes/warehouses, data quality and governance; and enabling CI/CD

Qualifications

• 10+ years of experience designing and supporting production-grade cloud data pipelines.
• Bachelor's degree in a related technical field.
• Active Secret clearance required.

Responsibilities

• Design and maintain scalable ETL/ELT data pipelines using cloud technologies.
• Develop large-scale data transformations with PySpark, Spark SQL, Python, and SQL.
• Build data ingestion frameworks for structured and unstructured data.
• Implement CDC and rerunnable pipeline architectures.
• Design and support cloud-based data lakes, data warehouses, and analytics-ready datasets.
• Implement data quality and governance controls including RBAC and lineage.
• Monitor and troubleshoot data pipelines for performance and reliability.
• Implement CI/CD pipelines for automated deployments.
• Ensure compliance with RMF, STIG, FedRAMP, and DoD security requirements.

Skills

Azure Synapse

Databricks

Python

SQL

ETL/ELT

CI/CD

RMF/STIG/FedRAMP

Change Data Capture

Pyspark

Education

Bachelor's degree in a related technical field

Tools

Azure Data Factory

AWS Glue

Databricks

Job description

Zachary Piper Solutions is seeking a Data Cloud Engineer to support a long-term Department of Defense modernization program at Fort Meade, MD. The Data Cloud Engineer will design, build, and maintain scalable cloud-based data solutions supporting enterprise modernization and mission-critical analytics within secure DoD environments.

Responsibilities for the Data Cloud Engineer Include:

• Design and maintain scalable ETL/ELT data pipelines using Azure Synapse, Databricks, AWS Glue, and related cloud technologies.
• Develop large-scale data transformations using PySpark, Spark SQL, Python, and SQL.
• Build data ingestion frameworks for structured and unstructured data, including relational databases, CSV, JSON, and REST APIs.
• Implement full and incremental data loads, Change Data Capture (CDC), and rerunnable pipeline architectures.
• Design and support cloud-based data lakes, data warehouses, and analytics-ready datasets.
• Implement data quality and governance controls including schema validation, RBAC, lineage, cataloging, and credential management.
• Monitor and troubleshoot data pipelines for performance, failures, latency, logging, and operational reliability.
• Implement CI/CD pipelines supporting automated deployments and environment promotion.
• Partner with cloud and cybersecurity teams to maintain compliance with RMF, STIG, FedRAMP, and DoD security requirements.
• Support cloud data migration, integration, and modernization initiatives across Azure and AWS environments.

Qualifications for the Data Cloud Engineer Include:

• 10+ years of experience designing and supporting production-grade cloud data pipelines.
• Strong experience with Azure Synapse, Databricks, Azure Data Factory, and/or AWS data services.
• Advanced experience with PySpark, Spark SQL, Python, and SQL.
• Experience with ETL/ELT, CDC, data lakes/warehouses, data quality, and governance.
• Experience implementing CI/CD and automated deployments for cloud data environments.
• Bachelor's degree in a related technical field.
• Active Secret clearance required.

Compensation for the Data Cloud Engineer Includes:

• Salary Range: $110,000 - $140,000 depending on experience.
• Comprehensive Benefits: Medical, Dental, Vision, 401(k), PTO, and Paid Holidays.

Keywords:

#LI-CR2 #LI-REMOTE

Data Cloud Engineer, Cloud Data Engineer, Senior Data Engineer, Data Engineer, Azure Data Engineer, AWS Data Engineer, Big Data Engineer, Data Platform Engineer, Azure, AWS, Azure Synapse, Synapse Analytics, Databricks, Azure Databricks, Azure Data Factory, ADF, AWS Glue, Redshift, Amazon Redshift, S3, PySpark, Apache Spark, Spark SQL, Python, SQL, ETL, ELT, Data Pipeline, Data Pipelines, Data Engineering, Data Lake, Data Warehouse, Data Lakehouse, Cloud Data Platform, Big Data, Data Integration, Data Migration, REST API, JSON, CDC, Change Data Capture, Incremental Loads, Data Quality, Data Governance, Schema Validation, Schema Drift, RBAC, Data Lineage, Data Catalog, CI/CD, DevOps, Oracle, RMF, STIG, FedRAMP, DoD, DISA, Fort Meade, Secret Clearance