Spark Developer Lead – Cloud/Data Pipelines (6 Month Contract – W2)

Mogi I/O : OTT/Podcast/Short Video Apps for you; Owings (MD); On-site; USD 65,000 - 112,000; Full time

Job summary

A leading company is looking for a Spark Developer Lead to build, optimize, and scale big data solutions using Apache Spark and AWS. This on-site role requires hands-on experience in PySpark and a strong background in data engineering, cloud architecture, and ETL development.

Qualifications

• 10+ years of experience in big data and distributed systems.
• 7+ years of AWS cloud computing experience.
• Strong communication and team collaboration abilities.

Responsibilities

• Design and lead PySpark-based big data applications.
• Architect scalable ETL workflows.
• Optimize Spark jobs for performance.

Skills

PySpark

Apache Spark

Python

AWS

SQL

NoSQL

ETL

Education

Bachelor’s degree in Computer Science

Tools

Airflow

DBT

Docker

Kubernetes

Job description

Spark Developer Lead – Cloud/Data Pipelines (6 Month Contract – W2)

Spark Developer Lead – Cloud/Data Pipelines (6 Month Contract – W2)

2 days ago Be among the first 25 applicants

• It is a 100% on-site role based in Owings Mills, MD.
• Candidates must have strong hands-on experience in PySpark, Apache Spark, Python, and AWS along with data engineering, cloud architecture, and ETL pipeline development.
• Visa sponsorship and relocation assistance are not available.
• Work Type: Full-time | On-site
• Contract Duration: 6 Months
• Notice Period: Immediate Joiner / Up to 30 Days Preferred
  
  

Note

• This is a W2 position.
• It is a 100% on-site role based in Owings Mills, MD.
• Candidates must have strong hands-on experience in PySpark, Apache Spark, Python, and AWS along with data engineering, cloud architecture, and ETL pipeline development.
• Visa sponsorship and relocation assistance are not available.
• Work Type: Full-time | On-site
• Contract Duration: 6 Months
• Notice Period: Immediate Joiner / Up to 30 Days Preferred
  
  

Job Summary

• We are hiring on behalf of a global IT services provider for a Lead PySpark Developer to build, optimize, and scale big data solutions using Apache Spark and AWS. The role will lead the design and deployment of ETL pipelines, collaborate with cross-functional teams, and drive cloud data platform initiatives.
  
  

Key Responsibilities

• Design and lead PySpark-based big data applications and data pipelines
• Architect scalable ETL workflows for structured/unstructured data
• Optimize Spark jobs for performance, caching, and partitioning
• Collaborate with data engineers, scientists, and business users
• Work with AWS services and cloud-native data platforms
• Apply DevOps and CI/CD practices to data engineering workflows
• Mentor junior developers, conduct code reviews, and enforce standards
• Ensure data governance, security, and compliance
  
  

Key Requirements

• Bachelor’s degree in Computer Science or equivalent
• 10+ years of experience in big data and distributed systems
• 7+ years of AWS cloud computing experience
• Expert in PySpark, Apache Spark, and Python
• Proficient in SQL and NoSQL (DB2, PostgreSQL, Snowflake)
• Experience with workflow tools like Airflow, and platforms like DBT
• Exposure to CI/CD, Docker, Kubernetes, and DevOps pipelines
• Excellent problem-solving and leadership skills
• Strong communication and team collaboration abilities

Seniority level

• Seniority level

  Mid-Senior level

Employment type

• Employment type

  Contract

Job function

• Job function

  Information Technology

• Industries

  Software Development

Referrals increase your chances of interviewing at Mogi I/O : OTT/Podcast/Short Video Apps for you by 2x

Get notified about new Project Lead Developer jobs in Owings, MD.

Maryland, United States $65,000 - $112,000 2 weeks ago

Assistant Project Manager (Mechanical Construction)

Washington DC-Baltimore Area $90,000 - $110,000 1 week ago

Maryland, United States
$77,040.00
-
$175,800.00
3 days ago

Project Manager (DODEA Cyber Security Manager Lead)

CDA Lending and Risk Analyst II (C24024/49299) (C24025/49300) Project Manager

Maryland, United States
$31.16
-
$36.23
5 days ago

Planning Project Manager (Transportation Engineering Manager I)

CDA Lending and Risk Analyst II (C24024/ 49299) (C24025/ 49300) (Project Manager)

Prince George's County, MD
$31.16
-
$36.23
1 month ago

Epic Service Area Consolidation Project Manager

Annapolis, MD
$70,000.00
-
$100,000.00
3 days ago

Annapolis, MD
$50,000.00
-
$90,000.00
2 days ago

ServiceNow Solution Architect / Senior Developer

Washington DC-Baltimore Area
$140,000.00
-
$200,000.00
4 days ago

Senior Project Manager, Asset Management

Senior Electrical Engineer / Project Manager

Maryland, United States
$129,029.52
-
$187,530.72
1 week ago

Capitol Heights, MD $100,000 - $140,000 2 weeks ago

Senior Pre-Construction Manager, WWRGD Design, Development and Construction

Washington DC-Baltimore Area $81,200 - $181,600 2 days ago

Electrical Distribution Project Manager *

Washington DC-Baltimore Area $80,000 - $100,000 6 days ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.