Hybrid Remote Senior Spark/Scala Data Engineer (Contract)

DataJobs; Richmond (VA); Hybrid; USD 91,000 - 98,000; Full time

Benefits offered by this job

401(k)

Dental insurance

Health insurance

Vision insurance

Job summary

Astyra Corporation is seeking a Senior Data Engineer to modernize legacy ETL workloads and deliver Spark-based data pipelines. This contract role is based in Richmond, VA with hybrid remote work, focusing on accurate data movement and validation between legacy and target platforms.

You will design, build, and optimize enterprise data pipelines using Spark and Scala, migrate legacy ETL to modern architectures, and implement data quality checks across systems.

Qualifications

• 7+ years of data engineering experience.
• Experience designing and developing enterprise data pipelines.
• Advanced Spark experience.
• Strong data transformation, data quality and reconciliation knowledge.
• Experience migrating legacy ETL to Spark-based architectures.
• Experience supporting production data-processing workloads.
• Familiarity with modern CI/CD and software development practices.

Responsibilities

• Design, build, and maintain scalable data pipelines using Apache Spark and Scala.
• Migrate legacy ETL workloads to Spark-based framework.
• Develop and enhance data transformation and processing solutions.
• Perform data comparison, reconciliation, and validation between legacy and target platforms.
• Identify and troubleshoot data discrepancies throughout the migration process.
• Create data quality checks and controls to ensure completeness and accuracy.
• Optimize Spark jobs and pipelines for performance, scalability, and reliability.
• Collaborate with data engineers, architects, and other technical teams on modernization initiatives.
• Support testing, deployment, troubleshooting, and ongoing maintenance of production data pipelines.
• Contribute to broader feature engineering and data modernization efforts.

Skills

7+ years experience

Data engineering

Troubleshooting

CI/CD familiarity

Data quality & reconciliation

Data transformation

Production data workloads

Legacy modernization

Tools

Apache Spark

Scala

Python

CI/CD tooling

Job description

Astyra Corporation is seeking a Senior Data Engineer to modernize legacy ETL workloads and deliver Spark-based data pipelines. This contract role is based in Richmond, VA with hybrid remote work, focusing on accurate data movement and validation between legacy and target platforms.

You will design, build, and optimize enterprise data pipelines using Spark and Scala, migrate legacy ETL to modern architectures, and implement data quality checks across systems.