Big Data Engineer – Python/Scala, Spark, Hadoop - Full Time

Programmers.io; Arlington (VA); Hybrid; USD 110,000 - 170,000; Full time

Job summary

Programmers.io seeks a Big Data Engineer for a full-time hybrid role in Arlington, VA. You will translate business needs into robust data pipelines across teams.

You'll design end-to-end pipelines with Python/Scala, Spark and Hadoop, ensuring data quality and scalable analytics. Experience with Hive, Airflow, NiFi, and Sqoop is a plus.

We value production-grade data apps and ML workflows, with cloud familiarity (Databricks or AWS) as a bonus. Onsite 3 days per week.

Qualifications

• End-to-end data pipelines across business, data management and technical teams.
• Production-grade data-driven applications and analytics workflows.
• Strong Python/Scala and SQL skills with Spark/Hadoop expertise.

Responsibilities

• Move between business, data management and technical teams to identify solutions.
• Build robust data pipelines end-to-end with data quality focus.
• Leverage Python/Scala, Spark and Hadoop tools to deliver data products and platforms.
• Deploy data-driven applications and processing workflows at scale.
• Implement ML systems in Java, Scala or Python and deliver analytics across stages.

Skills

Data pipelines

Python

Scala

Spark

SQL

Data quality

ML systems

Feature engineering

Cloud fundamentals

Tools

Hive

Airflow

NiFi

Sqoop

Job description

One of our client is Looking Big Data Engineer for Full Time Role at Arlington VA (Hybrid)

Location- Arlington, VA (3 Days onsite/week)

Terms- Full Time

Job Summary:

• Ability to easily move between business, data management, and technical teams; ability to quickly intuit the business use case and identify technical solutions to enable it
• Experience building robust and efficient data pipelines end-to-end with a strong focus on data quality
• High proficiency in using Python or Scala, Spark, Hadoop platforms & tools (Hive, Airflow, NiFi, Scoop), SQL to build Big Data products & platforms
• Experience in building and deploying production-level data-driven applications and data processing workflows/pipelines and/or
• Implementing machine learning systems at scale in Java, Scala, or Python and deliver analytics involving all phases like data ingestion, feature engineering, modeling, tuning, evaluating, monitoring, and presenting
• Cloud knowledge (Databricks or AWS ecosystem) is a plus but not required