Top Big Data Engineer Job in Rockville, MD | 5+ Years

Rishi Writes

Rockville (MD)

Hybrid

USD 120,000 - 170,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Rishi Writes in Rockville, MD is seeking a highly skilled Big Data Engineer to design, develop and optimize large-scale data processing systems in a hybrid role.

You will build data pipelines with Hadoop, Spark and AWS services, collaborating with data scientists and analysts to deliver scalable, reliable solutions. This position requires 5+ years of distributed-systems experience and a strong focus on performance tuning at petabyte scale.

Qualifications

  • Requires Big data engineering experience with Hadoop, Hive and Trino at petabyte scale.
  • Proficient in Spark development, performance tuning and troubleshooting.
  • Advanced SQL and data modeling skills are required.

Responsibilities

  • Design, develop and maintain large-scale data processing pipelines using Hadoop, Spark, Python and Scala.
  • Implement data ingestion, storage, transformation and analysis solutions that are scalable and reliable.
  • Optimize existing pipelines for performance, scalability and reliability.
  • Collaborate with cross-functional teams to translate business needs into technical solutions.
  • Run tests to protect data accuracy and pipeline strength.
  • Create and maintain automated tests for data pipelines.
  • Monitor and troubleshoot pipelines in production.
  • Support data scientists and analysts.
  • Stay current with emerging big data technologies.

Skills

Hadoop
Spark
Python
Scala
SQL
AWS
LLM tooling
GitHub Copilot

Education

Bachelor's degree in Computer Science, Information Systems or related field

Tools

Hive
Trino
EMR
Athena

Job description

About the Role

We are looking for a highly skilled Big Data Engineer to design, develop and optimize large-scale data processing systems in Rockville, MD. This is a hybrid role, and the source notes two open positions.

You will work with cross-functional teams to build data pipelines, implement data integration solutions and keep big data platforms fast, scalable and reliable. Deep experience with distributed systems, cloud platforms and technologies such as Hadoop and Spark is central.

Key Responsibilities
  • Design, develop and maintain large-scale data processing pipelines using Hadoop, Spark, Python and Scala.
  • Implement data ingestion, storage, transformation and analysis solutions that are scalable and reliable.
  • Optimize existing pipelines for performance, scalability and reliability.
  • Work with teams to turn business requirements into technical solutions.
  • Run unit, integration and system tests to protect data accuracy and pipeline strength.
  • Write and maintain automated unit, integration and end-to-end tests.
  • Monitor and troubleshoot pipelines in production.
  • Support data scientists and analysts.
  • Stay current with emerging big data technologies.
Required Skills and Qualifications
  • Big data engineering with Hadoop, Hive and Trino, including experience with petabytes of data and distributed systems.
  • Apache Spark development and performance tuning, including Spark internals and troubleshooting at petabyte scale.
  • Advanced SQL, including window functions, multi-table joins and query optimization.
  • AWS cloud engineering, including S3, EMR, Glue and Athena, and scaling clusters.
  • Python or Scala programming.
  • AI and LLM tooling, including GitHub Copilot and prompt engineering.
  • A Bachelor's degree in Computer Science, Information Systems or a related field with at least 5 years of related experience, or equivalent training or work experience.
  • Technical expertise in object-oriented and database concepts and experience delivering enterprise-quality solutions in an iterative or Agile environment.
Preferred Skills

The listing names these secondary skills as nice to have:

  • CI/CD
  • Java
  • Systems troubleshooting
  • ETL
  • Automated testing

A Master's degree is also listed as preferred.

Tools and Technologies
  • Hadoop, Hive and Trino
  • Apache Spark
  • AWS: S3, EMR, Glue and Athena
  • Python or Scala, and SQL
Work Environment and Collaboration

The listing describes a hybrid role in Rockville, MD, working with cross-functional teams, data scientists and analysts.

Why This Big Data Engineer Role Stands Out

Editorial note: this section is general guidance for readers. It is not part of the confirmed job requirements.

The listing stresses experience with petabytes of data and distributed systems. It centers on Hadoop, Hive, Trino and Spark, backed by AWS services such as S3, EMR, Glue and Athena.

It also lists AI and LLM tooling, including GitHub Copilot and prompt engineering, among its top skills.

Who Will Thrive Here

The listing asks for Spark performance tuning, advanced SQL and automated unit, integration and end-to-end testing.

The source notes that two positions are open.

Professional Expectations

Editorial note: this section is general guidance for readers. It is not part of the confirmed job requirements.

The listing asks for testing, production monitoring and troubleshooting, and performance optimization.

Frequently Asked Questions
1. What does a Big Data Engineer do?

You build, optimize and monitor large-scale data pipelines.

2. Which skills are essential?

Hadoop, Spark, AWS, advanced SQL and Python or Scala.

3. How much experience is required?

At least 5 years of related experience, or equivalent training or work experience.

4. Is a degree required?

A Bachelor's degree in a related field, or equivalent experience.

5. Which tools will I use?

Hadoop, Hive, Trino, Spark and AWS services such as S3, EMR, Glue and Athena.

6. Where is this job located?

Rockville, MD.

7. Is the role onsite, hybrid or remote?

Hybrid.

8. What type of employment is this?

The listing does not state the employment type. Please ask when you apply.

9. What does a typical day look like?

The listed duties include building and optimizing pipelines, testing and monitoring production pipelines. This is a general picture based on the listed duties, not a confirmed daily schedule.

10. What communication skills matter most?

The listing mentions collaboration with cross-functional teams, data scientists and analysts.

11. Which concepts should I understand well?

Spark internals, distributed systems and petabyte-scale performance tuning.

12. Are there any preferred skills?

Yes. CI/CD, Java, ETL, automated testing and troubleshooting.

13. How do I apply?

Email your resume to jobs@rishiwrites.com with the job title in the subject line.

14. What should I include in my application?

Your resume and the data volumes you have handled.

15. What qualities suit this role?

Rigor, curiosity about new technology and calm production troubleshooting. These are general suggestions, not confirmed requirements.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Big Data Engineer: Spark, Hadoop & Cloud Pipelines
Senior Big Data Engineer: Spark, Hadoop & Cloud Pipelines

Compunnel Inc. • Tysons (VA)

On-site
USD 150,000 - 190,000
Big Data Engineer
Big Data Engineer

Compunnel Inc. • Tysons (VA)

On-site
USD 150,000 - 190,000
Senior Data Engineer
Senior Data Engineer

Unisys • McLean (VA)

On-site
USD 120,000 - 180,000
Senior Big Data Engineer — Spark, Hadoop on AWS (Hybrid)
Senior Big Data Engineer — Spark, Hadoop on AWS (Hybrid)

Rishi Writes • Rockville (MD)

Hybrid
USD 120,000 - 170,000
Big Data Engineer
Big Data Engineer

Cymertek Corporation • Reston (VA)

On-site
USD 110,000 - 170,000
Big Data Engineer
Big Data Engineer

Good co India • United States

Remote
USD 120,000 - 180,000
Senior Data Engineer
Senior Data Engineer

Rishi Writes • Plano (TX)

On-site
USD 140,000 - 160,000
Data Engineer Sr
Data Engineer Sr

Sequoia Connect • United States

Remote
USD 120,000 - 160,000
Big Data Engineer
Big Data Engineer

TechDigital Group • Jersey City (NJ)

On-site
USD 90,000 - 150,000
Data Engineer | Seattle, WA
Data Engineer | Seattle, WA

Tech Junction Ltd • Seattle (WA)

Hybrid
USD 120,000 - 180,000