Data Engineer II: Spark & Hadoop Pipelines (Hybrid)

Delan Associates, Inc; Arlington (VA); Hybrid; USD 90,000 - 130,000; Full time

Job summary

Delan Associates, Inc is seeking a Data Engineer II to join the Analytics Solutions Team. The role focuses on extracting data from the warehouse, transforming it, and building dashboards to deliver actionable insights.

The ideal candidate has strong hands-on experience with Spark, Hadoop, Python, and SQL, and has built scalable data pipelines in large environments. Hybrid work with 3 days in Arlington area.

Qualifications

• Experience as a Data Engineer or similar data-focused role.
• Strong SQL development and query optimization experience.
• Hands-on experience with Apache Spark (PySpark, Spark SQL, Spark Streaming), Hadoop ecosystem (HDFS/Ozone, Hive, YARN), and Python.
• Experience building and maintaining ETL pipelines.
• Understanding of data modeling and database design principles.
• Ability to troubleshoot and resolve complex data issues independently.
• Experience validating and testing data for quality and consistency.
• Strong written and verbal communication skills.

Responsibilities

• Design, develop, and support enterprise-scale ETL processes and data pipelines.
• Build scalable and efficient data processing solutions that ensure timely delivery of business-critical data.
• Develop and optimize big data workflows using Apache Spark and Hadoop technologies.
• Partner with Data Engineers, Analysts, and business stakeholders to deliver high-quality data solutions.
• Troubleshoot data issues and implement solutions that maintain data integrity and quality.
• Support dashboard and reporting initiatives by delivering clean, transformed datasets.
• Utilize SQL and database technologies to improve data processing performance and efficiency.
• Participate in automation initiatives to streamline operational and data engineering processes.
• Follow engineering best practices including code reviews, version control, testing, and data validation.
• Ensure compliance with internal policies and industry regulations.

Skills

Spark

Hadoop

Python

SQL

Education

Bachelor's degree in Engineering, Computer Science, or related field

Job description

Delan Associates, Inc is seeking a Data Engineer II to join the Analytics Solutions Team. The role focuses on extracting data from the warehouse, transforming it, and building dashboards to deliver actionable insights.

The ideal candidate has strong hands-on experience with Spark, Hadoop, Python, and SQL, and has built scalable data pipelines in large environments. Hybrid work with 3 days in Arlington area.