Python Data Engineer: Build Scalable ETL Pipelines

mthree; New York (NY); On-site; USD 85,000 - 100,000; Full time

Job summary

mthree is seeking an intermediate Python Developer in Data Engineering for a client in the financial industry in New York. You will design, build, and maintain scalable data pipelines using Python and related technologies, collaborating with cross-functional, cross-regional teams.

The role focuses on ETL, data quality, and performance, with opportunities to tackle distributed systems, APIs, and data storage solutions while growing your skills in a fast-moving consultancy environment.

Qualifications

• Proficiency in Python, with a strong understanding of object-oriented programming principles.
• Data Structures and Algorithms: familiarity with common data structures and their practical usage.
• Experience with data libraries such as Django, Flask, NumPy, Pandas.
• Familiarity with PostgreSQL, MySQL and NoSQL databases like MongoDB.
• Knowledge of ETL tools (Airflow, Kafka) and data storage technologies.
• Experience with orchestration tools and cloud platforms (Ansible, Terraform/Kubernetes, AWS, Azure, GCP).
• Familiarity with modern CI/CD frameworks (Git, Jenkins, Github) and Linux.

Responsibilities

• Design, develop, and maintain software applications and scalable data pipelines using Python.
• Optimize data workflows for large datasets and ensure data quality.
• Write clean, maintainable code meeting industry standards.
• Develop and manage databases and applications; implement ETL processes.
• Write and document effective APIs; test software at scale for performance.
• Collaborate with teams to define requirements and participate in code reviews.
• Troubleshoot, debug, and resolve performance issues and bugs.
• Document code and projects to aid future maintenance.

Skills

Python programming

OOP concepts

Data structures & algorithms

Problem solving

Collaboration

Tools

Django

Flask

NumPy

Pandas

PostgreSQL

MySQL

MongoDB

Airflow

Kafka

Ansible

Terraform

Kubernetes

AWS

Azure

GCP

Git

Jenkins

Github

Linux

Job description

mthree is seeking an intermediate Python Developer in Data Engineering for a client in the financial industry in New York. You will design, build, and maintain scalable data pipelines using Python and related technologies, collaborating with cross-functional, cross-regional teams.

The role focuses on ETL, data quality, and performance, with opportunities to tackle distributed systems, APIs, and data storage solutions while growing your skills in a fast-moving consultancy environment.