Data Engineer II — Spark, Hadoop & Python (Hybrid)

Delan Associates, Inc; United States; Hybrid; USD 110,000 - 140,000; Full time

Job summary

Delan Associates, Inc. seeks an experienced Data Engineer II to join the Analytics Solutions Team. You will design and build enterprise-scale ETL pipelines, transform data, and deliver reporting dashboards that provide actionable insights to clients.

This hybrid role requires Spark, Hadoop, and Python proficiency, strong SQL and data modeling skills, and collaboration with stakeholders across the organization. Arlington area preferred.

Qualifications

• Experience as a Data Engineer or similar data-focused role.
• Strong SQL development and query optimization experience.
• Hands-on experience with Apache Spark (PySpark, Spark SQL, Spark Streaming).
• Hands-on experience with Hadoop ecosystem (HDFS/Ozone, Hive, YARN).
• Hands-on experience with Python.

Responsibilities

• Design, develop, and support enterprise-scale ETL processes and data pipelines.
• Build scalable and efficient data processing solutions that ensure timely delivery of business-critical data.
• Develop and optimize big data workflows using Apache Spark and Hadoop technologies.
• Partner with Data Engineers, Analysts, and business stakeholders to deliver high-quality data solutions.
• Troubleshoot data issues and implement solutions that maintain data integrity and quality.
• Support dashboard and reporting initiatives by delivering clean, transformed datasets.
• Utilize SQL and database technologies to improve data processing performance and efficiency.
• Participate in automation initiatives to streamline operational and data engineering processes.
• Follow engineering best practices including code reviews, version control, testing, and data validation.
• Ensure compliance with Mastercard internal policies and industry regulations.

Skills

Spark

Hadoop

Python

SQL

Education

Bachelor's degree

Job description

Delan Associates, Inc. seeks an experienced Data Engineer II to join the Analytics Solutions Team. You will design and build enterprise-scale ETL pipelines, transform data, and deliver reporting dashboards that provide actionable insights to clients.

This hybrid role requires Spark, Hadoop, and Python proficiency, strong SQL and data modeling skills, and collaboration with stakeholders across the organization. Arlington area preferred.