Senior Data Engineer: Spark, Databricks & AWS Pipelines

ICF Olson; Reston (VA); Remote; USD 99,000 - 168,000; Full time

Job summary

ICF in Reston, Virginia is hiring a Senior Data Engineer to design, build, and optimize scalable data pipelines using Spark, Hive, Airflow, and Databricks. You will develop data processing workflows on Databricks, create APIs for data access, and build dashboards in AWS QuickSight.

The role requires strong programming in Python/Java, plus cloud experience and data governance awareness. Join a mission-driven team focused on delivering reliable data solutions, improving data quality, and enabling

Qualifications

• Bachelor's degree required; 7+ years hands-on software development experience; 4+ years in data pipelines using Python/Java and cloud tech; ability to obtain Public Trust clearance; US residency and continuous US presence for 3 of last 5 years.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Spark, Hive, and Airflow.
• Develop and deploy data processing workflows on Databricks.
• Develop API services to access and integrate data.
• Create interactive data visualizations and reports using AWS QuickSight.
• Build infrastructure for ETL from diverse sources using AWS and SQL tech.
• Monitor and optimize data infrastructure performance.
• Write unit and integration tests; participate in CI/CD and IaC with DevOps.

Skills

Spark

Hive

Airflow

Databricks

Python

Java

SQL

Scala

Cloud computing

Education

Bachelor's degree

Tools

AWS

Redshift

S3

EMR

Jupyter Notebooks

PostgreSQL

Cassandra

Job description

ICF in Reston, Virginia is hiring a Senior Data Engineer to design, build, and optimize scalable data pipelines using Spark, Hive, Airflow, and Databricks. You will develop data processing workflows on Databricks, create APIs for data access, and build dashboards in AWS QuickSight.

The role requires strong programming in Python/Java, plus cloud experience and data governance awareness. Join a mission-driven team focused on delivering reliable data solutions, improving data quality, and enabling