Senior Data Engineer - Databricks, AWS, Remote

ICF; United States; Remote; USD 99,000 - 168,000; Full time

Job summary

ICF is seeking a Senior Data Engineer in the United States to design and maintain scalable data pipelines using Spark, Hive, and Airflow. You will deploy workflows on Databricks, build API services, and create interactive reports with AWS QuickSight.

Ideal candidates have 7+ years of development, 4+ years building pipelines with Python/Java in cloud environments, and eligibility for a Public Trust clearance.

Qualifications

• Bachelor's degree required.
• 7+ years of hands-on software development experience.
• 4+ years building data pipelines using Python, Java, and cloud technologies with Spark/Hive.
• Eligible to obtain and maintain a Public Trust clearance.
• US residency and work authorized in the US.

Responsibilities

• Design, develop, and maintain scalable data pipelines with Spark, Hive, and Airflow.
• Develop and deploy data workflows on Databricks.
• Create API services for data access and integration.
• Build interactive data visualizations with AWS QuickSight.
• Ensure data security, privacy, and timely data refreshes.
• Collaborate with DevOps on CI/CD and IaC; perform code reviews.

Skills

Spark

Python

Java

Airflow

AWS QuickSight

Education

Bachelor's degree

Tools

Databricks

Hive

SQL

Terraform

GitHub Actions

Job description

ICF is seeking a Senior Data Engineer in the United States to design and maintain scalable data pipelines using Spark, Hive, and Airflow. You will deploy workflows on Databricks, build API services, and create interactive reports with AWS QuickSight.

Ideal candidates have 7+ years of development, 4+ years building pipelines with Python/Java in cloud environments, and eligibility for a Public Trust clearance.