Python Data Engineer — PySpark, Pytest & Real‑Time Data

Synechron; Pittsburgh (Allegheny County); On-site; USD 105,000 - 110,000; Full time

Benefits offered by this job

Competitive pay

Global offices

Paid leave

Parental leave

Insurance plan

Retirement plan

Education benefits

Udemy for Business

Training opportunities

Diverse culture

Job summary

Synechron, a global consulting firm, is seeking a Python Data Engineer with hands-on PySpark and Pytest experience to design, develop, test, and maintain scalable data pipelines and processing solutions.

The ideal candidate will work with Python, PySpark, Django, Flask, MongoDB, Kafka and REST APIs, contributing to ETL/ELT pipelines, implementing tests with Pytest, and collaborating with data scientists, software engineers, QA, and DevOps.

Qualifications

• Experience in Python development and data engineering.
• Strong knowledge of Python, PySpark, Django, Flask, MongoDB, Kafka and Pytest.
• Experience with REST APIs, ETL/ELT pipelines, and distributed data processing.
• Proficiency in SQL and working with JSON, CSV, and Parquet data formats.
• Experience with Git, automated testing, and Agile methodologies.
• Strong analytical, communication, and problem-solving skills.

Responsibilities

• Develop data pipelines and applications using Python and PySpark.
• Build REST APIs and backend services using Django and Flask.
• Design and manage MongoDB databases, schemas, indexes, and queries.
• Develop Kafka producers and consumers for real-time data processing.
• Implement data transformation, validation, cleansing, and integration workflows.
• Write unit and integration tests using Pytest.
• Optimize application and Spark-job performance.
• Troubleshoot data, application, and production issues.
• Collaborate with data scientists, software engineers, QA, and DevOps teams.
• Participate in code reviews, Agile ceremonies, and CI/CD activities.

Skills

Python

PySpark

Django

Flask

MongoDB

Kafka

Pytest

Tools

Docker

CI/CD tools

Job description

Synechron, a global consulting firm, is seeking a Python Data Engineer with hands-on PySpark and Pytest experience to design, develop, test, and maintain scalable data pipelines and processing solutions.

The ideal candidate will work with Python, PySpark, Django, Flask, MongoDB, Kafka and REST APIs, contributing to ETL/ELT pipelines, implementing tests with Pytest, and collaborating with data scientists, software engineers, QA, and DevOps.