Senior Data Engineer - PySpark/Scala on AWS (Hybrid)

Pyramid, Inc; McLean (VA); Hybrid; USD 76.000 - 83.000; Vollzeit

Zusammenfassung

Pyramid Consulting, Inc. is seeking a Data Engineer for a 6+ month contract in McLean, VA with hybrid work arrangement. The role focuses on building scalable data pipelines using PySpark and Scala, and designing cloud-based data processing on AWS.

Strong experience in ETL/ELT, data transformation, and collaboration with cross-functional teams is required. The candidate will optimize jobs for performance and reliability, work with AWS data services, and ensure data quality, security, and

Qualifikationen

• 5+ years of hands-on PySpark experience.
• 5+ years of programming in Scala.
• 5+ years of AWS cloud services and data engineering.

Aufgaben

• Develop and maintain scalable data pipelines using PySpark and Scala.
• Design data processing solutions on AWS cloud.
• Build efficient ETL/ELT pipelines for large-scale data.
• Optimize PySpark/Scala jobs for performance and reliability.
• Work with AWS data services and cloud architectures.
• Transform, cleanse, validate and integrate data.
• Troubleshoot pipeline issues and improve system performance.
• Collaborate with data architects, analysts, developers and stakeholders.

Kenntnisse

PySpark

Scala

AWS

Data pipelines

Tools

AWS

ETL/ELT

Jobbeschreibung

Pyramid Consulting, Inc. is seeking a Data Engineer for a 6+ month contract in McLean, VA with hybrid work arrangement. The role focuses on building scalable data pipelines using PySpark and Scala, and designing cloud-based data processing on AWS.

Strong experience in ETL/ELT, data transformation, and collaboration with cross-functional teams is required. The candidate will optimize jobs for performance and reliability, work with AWS data services, and ensure data quality, security, and