Data Engineer II: Spark & Hadoop ETL (Onsite)

Accord Technologies Inc; Arlington (VA); On-site; USD 120,000 - 180,000; Part time

Job summary

Accord Technologies Inc is seeking a Data Engineer II to design, implement, and maintain scalable enterprise ETL processes and robust data pipelines for a global client base. You will develop optimized data processing code and leverage Spark/Hadoop to ensure timely data availability.

Collaborate with senior engineers to maintain high data quality and support cross-client data delivery. Candidates should have 9+ years in data engineering with hands-on experience in Spark, Hadoop, Python, and SQL,

Qualifications

• 9+ years of hands-on data engineering experience building large-scale ETL pipelines using Apache Spark, Hadoop, Python, and SQL.
• Strong background in the payments or financial sector.
• Proficiency in writing and optimizing SQL queries to retrieve, manipulate, and analyze data efficiently.
• Hands-on experience with Apache Spark (PySpark, Spark SQL, Spark Streaming) and the Hadoop ecosystem (HDFS/Ozone, Hive, YARN).
• Understanding data modeling concepts and database design to support scalable data solutions.
• Familiarity with Python.
• Ability to analyze and troubleshoot data issues and provide solutions with minimal supervision.
• Basic knowledge of testing and validating data to ensure accuracy and consistency in data pipelines.
• Excellent verbal and written communication skills, with the ability to articulate complex ideas clearly to both technical and non-technical stakeholders.

Responsibilities

• Design, implement, and maintain scalable enterprise ETL processes and robust data pipelines for a global client base.
• Develop and optimize code for data processing, ensuring timely availability and accessibility of data.
• Leverage big data processing frameworks such as Apache Spark and Hadoop to build and optimize data pipelines.
• Collaborate with senior engineers to address data challenges and maintain high data quality.
• Assist in data delivery, working alongside Data Engineers and Analysts to support accurate, high‑value data solutions across various clients and industries.
• Build strong working relationships with team members and clients, contributing to local and global projects.
• Apply industry best practices, including version control, code reviews, and data validation, to ensure quality in data processes.
• Use SQL and other database technologies to optimize data processing and reduce the time required to handle large data sets.
• Design, implement, and maintain data pipelines using ETL frameworks, orchestration tools, and distributed data processing engines.
• Participate in automation of routine data tasks and streamlining processes.
• Comply with all Mastercard internal policies and adhere to external regulations.

Skills

ETL pipelines

Data engineering

SQL proficiency

Python

Spark

Hadoop

Communication

Problem solving

Education

Bachelor’s degree in a quantitative discipline

Tools

Apache Spark

Hadoop ecosystem (HDFS/Ozone, Hive, YARN)

SQL

Python programming

Job description

Accord Technologies Inc is seeking a Data Engineer II to design, implement, and maintain scalable enterprise ETL processes and robust data pipelines for a global client base. You will develop optimized data processing code and leverage Spark/Hadoop to ensure timely data availability.

Collaborate with senior engineers to maintain high data quality and support cross-client data delivery. Candidates should have 9+ years in data engineering with hands-on experience in Spark, Hadoop, Python, and SQL,