Lead PySpark Developer (7232-1) Owings Mills, MD

ESR Healthcare; Owings Mills (MD); On-site; USD 120,000 - 150,000; Full time

Job summary

A leading company in the IT sector is seeking a Lead PySpark Developer to join their team in Owings Mills, MD. The ideal candidate will have strong expertise in AWS, big data solutions, and team leadership. This role will involve designing and optimizing ETL workflows, collaborating with various teams, and mentoring junior developers, making it a significant position with clear impact on project success.

Qualifications

• 7+ years in AWS Cloud Computing.
• 10+ years in big data and distributed computing.
• Experience with PySpark and SQL databases.

Responsibilities

• Lead design and development of PySpark-based solutions.
• Optimize ETL pipelines for data handling.
• Mentor junior developers and ensure best practices.

Skills

PySpark

Apache Spark

Python

SQL

NoSQL

DB2

PostgreSQL

Big Data

Cloud Computing

ETL

Education

Bachelor’s degree

Tools

AWS

Docker

Kubernetes

Job description

Lead PySpark Developer (7232-1) Owings Mills, MD

PySpark Developer, Amazon Web Service (AWS), Cloud Computing, Big Data, Apache Spark, Python, SQL, NoSQL, DB2, PostgreSQL, Snowf

If you post this job on a job board, please do not use company name or salary.
Experience level: Mid-senior Experience required: 7 Years Education level: Bachelor’s degree Job function: Information Technology Industry: Information Technology and Services Pay rate : View hourly payrate Total position: 1 Relocation assistance: No Visa sponsorship eligibility: No
Job Description:

7+ years of experience in Amazon Web Service(AWS) Cloud Computing.
10+ years of experience in big data and distributed computing.
Very Strong hands-on experience with PySpark, Apache Spark, and Python.
Strong Hands on experience with SQL and NoSQL databases (DB2, PostgreSQL, Snowflake, etc.).
Proficiency in data modeling and ETL workflows.
Proficiency with workflow schedulers like Airflow.
Hands on experience with AWS cloud-based data platforms.
Experience in DevOps, CI/CD pipelines, and containerization (Docker, Kubernetes) is a plus.
Strong problem-solving skills and ability to lead a team
DBT, AWS Astronomer
Lead the design, development, and deployment of PySpark-based big data solutions.
Architect and optimize ETL pipelines for structured and unstructured data.
Collaborate with Client, data engineers, data scientists, and business teams to understand requirements and provide scalable solutions.
Optimize Spark performance through partitioning, caching, and tuning.
Implement best practices in data engineering (CI/CD, version control, unit testing).
Work with cloud platforms like AWS.
Ensure data security, governance, and compliance.
Mentor junior developers and review code for best practices and efficiency.
Additional Notes:

Please submit the candidate's resume in PDF format.
Please note that TCS does not consider former full-time employees (FTEs) for rehire. Additionally, individuals who have previously worked at TCS as contractors must observe a minimum waiting period of six months before being eligible for re-engagement.
MUST HAVE:

7+ years of experience in Amazon Web Service(AWS) Cloud Computing.
10+ years of experience in big data and distributed computing.
Experience with PySpark, Apache Spark, and Python.
Experience with SQL and NoSQL databases (DB2, PostgreSQL, Snowflake, etc.).
Hands on experience with AWS cloud-based data platforms.
Experience in DevOps, CI/CD pipelines, and containerization (Docker, Kubernetes) is a plus.