Data Cloud Engineer

Piper Companies; United States; Remote; USD 120,000 - 140,000; Full time

Benefits offered by this job

Fully remote

Job summary

Zachary Piper Solutions is seeking an experienced Data Cloud Engineer for a fully remote opportunity. This position supports a long-term Department of Defense modernization initiative focused on multi-cloud data integration, secure data architecture, and mission-critical analytics.

Active Secret Clearance is required. The role involves designing, building, and operating scalable cloud-based ETL/ELT pipelines using Azure and AWS, with PySpark, Spark SQL, Python, and SQL expertise.

Qualifications

• Active Secret Clearance required.
• Bachelor's degree in Computer Science, Engineering, Information Systems, Data Science, or related field.
• 8+ years of data engineering experience.
• Strong hands-on experience with PySpark, Spark SQL, Python, and SQL.
• Experience with Azure Synapse Analytics, Databricks, and/or AWS data services.
• Experience building and supporting production-grade ETL/ELT pipelines.
• Experience with data ingestion, integration, transformation, and CDC.
• Experience troubleshooting production data environments and implementing monitoring and logging.

Responsibilities

• Design, develop, and maintain scalable cloud-based ETL/ELT pipelines using Azure and AWS technologies.
• Build and optimize large-scale data transformations using PySpark and Spark SQL.
• Develop data ingestion frameworks supporting structured and unstructured data, including databases, APIs, JSON, and flat files.
• Implement full and incremental data loading strategies, including CDC.
• Design and support enterprise data lakes, data warehouses, and analytics environments.
• Implement data quality, governance, and security controls including schema validation and RBAC.
• Monitor, troubleshoot, and optimize production data pipelines.
• Support CI/CD implementation and automated deployments.
• Partner with cloud engineering, cybersecurity, DevOps, and software development teams.

Skills

PySpark

Spark SQL

Python

SQL

Education

Bachelor's degree (CS/Engineering/IS/Data Science)

Tools

Azure Synapse Analytics

Databricks

AWS data services

Azure Data Factory

AWS Glue

Redshift

Amazon S3

Job description

Zachary Piper Solutions is seeking an experienced Data Cloud Engineer for a fully remote opportunity. This position offers the opportunity to support a long-term Department of Defense modernization initiative focused on multi-cloud data integration, secure data architecture, and mission-critical analytics. Candidates must hold an active Secret Clearance.

Responsibilities

• Design, develop, and maintain scalable cloud-based ETL/ELT pipelines using Azure and AWS technologies.
• Build and optimize large-scale data transformations using PySpark and Spark SQL.
• Develop data ingestion frameworks supporting structured and unstructured data, including databases, APIs, JSON, and flat files.
• Implement full and incremental data loading strategies, including Change Data Capture (CDC).
• Design and support enterprise data lakes, data warehouses, and analytics environments.
• Implement data quality, governance, and security controls including schema validation and RBAC.
• Monitor, troubleshoot, and optimize production data pipelines.
• Support CI/CD implementation and automated deployments.
• Partner with cloud engineering, cybersecurity, DevOps, and software development teams.

Required Qualifications

• Active Secret Clearance.
• Bachelor's degree in Computer Science, Engineering, Information Systems, Data Science, or related field.
• 8+ years of data engineering experience.
• Strong hands-on experience with PySpark, Spark SQL, Python, and SQL.
• Experience with Azure Synapse Analytics, Databricks, and/or AWS data services.
• Experience building and supporting production-grade ETL/ELT pipelines.
• Experience with data ingestion, integration, transformation, and CDC.
• Experience troubleshooting production data environments and implementing monitoring and logging.

Preferred Qualifications

• Experience with Azure Data Factory, AWS Glue, Redshift, Amazon S3, and Oracle.
• Experience supporting federal or DoD cloud modernization programs.
• Familiarity with RMF, STIG, and FedRAMP.
• Experience working in Agile development environments.

Salary and Benefits Include:

• Salary: $120,000 - $140,000 depending on experience