Principal PySpark Engineer – AWS/EMR

BigRio; New Jersey; Remote; USD 85,000 - 100,000; Full time

Job summary

BigRio, a technology consulting firm, is seeking a Principal PySpark Engineer. This role requires strong experience in building data pipelines using Apache Spark on AWS EMR, focusing on coding and GxP-compliant environments. Ideal candidates should have substantial programming skills in Python and experience with Databricks, aiming to contribute directly to robust data solutions. This is a remote opportunity with occasional office presence required.

Qualifications

• 8-10 years in software or data engineering focused on distributed systems.
• Hands-on experience with Apache Spark, PySpark, and AWS EMR.
• Proven ability to design and develop data pipelines.

Responsibilities

• Design, develop, and maintain distributed ETL data pipelines using PySpark on AWS EMR.
• Work within a GxP-compliant environment to ensure data integrity.
• Collaborate with teams to deliver end-to-end data solutions.

Skills

PySpark

AWS

Python

Databricks

Distributed Computing

GxP Compliance

Job description

Direct message the job poster from BigRio

Sr. Director, Talent Acquisition and HR at BigR.io

Job Title: Principal PySpark Engineer – AWS/EMR

Location: Remote- New England, NY, and New Jersey (EST Time Zone Preferred)-

5 Days a month in the Office

Duration: 6 Months Contract

About BigRio:

BigRio is a remote-based, technology consulting firm with headquarters in Boston, MA. We deliver software solutions ranging from custom development and software implementation to data analytics and machine learning/AI integrations. As a one-stop shop, we attract clients from a variety of industries due to our proven ability to deliver cutting-edge, cost-effective software solutions.

Job Overview:

We are seeking Principal PySpark Engineers with strong hands-on experience in building distributed data pipelines using Apache Spark on AWS EMR. The ideal candidate is proficient in Python, has worked with Databricks, and has a solid understanding of GxP-compliant environments. This is a coding-heavy role — not DevOps or AWS administration — where you’ll contribute directly to the architecture and development of robust data solutions in a highly regulated, cloud-native environment.

Key Responsibilities:

• Design, develop, and maintain distributed ETL data pipelines using PySpark on AWS EMR
• Work within a GxP-compliant environment, ensuring data integrity and regulatory alignment
• Write clean, scalable, and efficient PySpark code for large-scale data processing
• Utilize AWS cloud services for pipeline orchestration, compute, and storage
• Collaborate closely with cross-functional teams to deliver end-to-end data solutions
• Participate in code reviews, testing, and deployment of pipeline components
• Ensure performance optimization, fault tolerance, and scalability of data workflows

Required Qualifications:

• Hands-on experience with writing clean, scalable, and efficient PySpark code for large-scale data processing
• Recent Hands-on Experience is Must.
• 8–10 years of experience in software or data engineering with a focus on distributed systems
• Deep hands-on experience with Apache Spark, PySpark, and AWS (especially EMR)
• Experience building pipelines using Databricks is required.
• Strong programming skills in Python
• Solid understanding of cloud-native architectures
• Familiarity with GxP compliance and working in regulated data environments
• Proven ability to independently design and develop data pipelines (not a DevOps/AWS admin role)
• Experience with distributed computing and high-volume ETL pipelines

Equal Opportunity Statement:

BigRio is an equal-opportunity employer. We prohibit discrimination and harassment of any kind based on race, religion, national origin, sex, sexual orientation, gender identity, age, pregnancy, status as a qualified individual with disability, protected veteran status, or other protected characteristic as outlined by federal, state, or local laws. BigRio makes hiring decisions based solely on qualifications, merit, and business needs at the time. All qualified applicants will receive equal consideration for employment.

Seniority level

• Seniority level

  Mid-Senior level

Employment type

• Employment type

  Contract

Job function

• Industries

  Software Development

Referrals increase your chances of interviewing at BigRio by 2x

Sign in to set job alerts for “Software Engineer” roles.

New Jersey, United States $85,000.00-$100,000.00 2 days ago

New Jersey, United States $115,000.00-$185,000.00 2 months ago

Trenton, NJ $120,000.00-$150,000.00 5 hours ago

United States $150,000.00-$200,000.00 1 day ago

Gen AI With Cloud- 100% Remote (Fulltime)

Lead Java Developer With Backend -(Fulltime) -100% Remote

Senior Terraform Developer (Fulltime)- Remote

Site Reliability Engineer - 100 % Remote

Lead Java Full Stack Developer - (Fulltime) 100% Remote

New Jersey, United States $110.00-$120.00 1 week ago

Lead Java Full Stack Developer - (Fulltime) 100% Remote

Jersey City, NJ $110,000.00-$115,000.00 4 weeks ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.