Python HPC Pipeline Engineer

E-Solutions; United States; On-site; USD 160,000 - 200,000; Full time

Job summary

A leading technology company seeks a skilled Python developer to join their dynamic team. You will focus on developing and maintaining HPC pipelines, ensuring high standards of code quality. This role requires expertise in Python, Bash scripting, and familiarity with Docker and Google Cloud. Strong communication and teamwork skills are essential for success in this collaborative environment.

Qualifications

• 5-8 years strong experience in Python.
• Experience with Docker and GCP.
• Ability to work independently and in teams.

Responsibilities

• Develop and maintain HPC pipelines.
• Write unit tests and troubleshoot performance.
• Collaborate with teams across time zones.

Skills

Python

Bash scripting

Docker

Google Cloud

HPC Operations

Data Structures

Object-oriented programming

Git

Education

B Tech, MS or PhD in Computer Science

Job description

Direct message the job poster from E-Solutions

We are looking for an expert developer who is highly skilled with Python. Primary focus will be developing and maintaining the HPC pipelines, Write unit test cases, Interact with teams in different timezone, migration of Python pipeline from GCP to HPC clusters, following generally accepted practices and workflows. Ensuring that you produce robust, secure, modular, and maintainable code. You will coordinate with other team members, including backend, Devops engineers. Your commitment to team collaboration, perfect communication, and a quality product is crucial.

Qualifications and Skills:

• B Tech, MS or PhD degree in Computer Science or similar.

• 5-8 years of strong hands-on experience in Python.

• Strong hands-on experience in Bash scripting.

• Strong knowledge of Docker, Google Cloud Clusters (Compute engine, GCS)

• Should have working experience of on-prem cluster.

• Good to have knowledge about HPC Operations work.

• Good communication skills and ability to work independently

• Expertise in understanding and analyzing requirements

• Participate in pair programming and peer reviews. You'll collaborate with other engineers on problems of all shapes and sizes, from code to new ideas and methodologies.

• Incorporate automated testing into development and maintenance procedures.

• Strong Knowledge of data structure, functional programming and object-oriented programming paradigms

• Ability to write efficient, secure, well-documented, and clean Python code

• Proficiency with modern development tools, like Git

• Experience with both consuming and designing Pipelines setups.

• Suggest any enhancements or changes that are required to stay up with modern security and development best practices.

• Good to have GCP.

Responsibilities:

• Designs, analyzes, and documents cluster pipelines

• Investigates and analyzes verbal and written requests for pipelines.

• Excellent teamwork and communication abilities

• Write backend code/scripts in programming languages like Python / BASH

• Maintains high standards of quality for code, functional specification documentation, and deliverables

• Self-motivated and self-managing, with strong organizational skills

• Ability to work with tight deadlines and multiple competing priorities

• Write efficient, secure, clean, scalable and robust Python code that is effective

• Test and troubleshoot the pipeline to ensure its performance.

• Ability to optimize the pipeline for performance

• Interact with development teams to develop a strong understanding of the project and testing objectives.

• Participate in troubleshooting of issues with different teams to drive towards root cause identification and resolution

• Documentation skills to track the development and implementations

• Effective communication skills: Regularly achieve consensus with peers, and clear status updates.

Desired Skills:

• LSF jobs, DDN’s understanding [Must]

• Google Cloud Platform / HPC clusters knowledge [Must]

• Data Structures / Object oriented programming concepts [Must]

• Experience in building Google cloud solutions and/or microservices. [optional]

• Knowledge of Git [Must]

Seniority level

• Seniority level

  Mid-Senior level

Employment type

• Employment type

  Contract

Job function

• Job function

  Information Technology

• Industries

  IT Services and IT Consulting

Referrals increase your chances of interviewing at E-Solutions by 2x

Get notified about new Python Developer jobs in United States.

Software Engineer, Python - AI Training (Freelance, Remote)

Software Engineer (Python), Data Platform

Chicago, IL $165,000.00-$195,000.00 3 days ago

United States $160,000.00-$200,000.00 5 days ago

Full-Stack Software Engineer (New graduates: United States)

United States $70,000.00-$100,000.00 3 days ago

United States $80,000.00-$200,000.00 3 days ago

Backend Engineer with Python (100% remote or hybrid)

Johnston, RI $99,000.00-$130,000.00 3 weeks ago

New York City Metropolitan Area 2 weeks ago

Search - Developer Tools - Python - Software Engineer I

New York, NY $140,000.00-$150,000.00 4 days ago

Full Stack Engineer (L5), N-Tech Software Engineering

United States $100,000.00-$900,000.00 1 week ago

Backend Software Developer (AWS / Python)

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.