HPC Dataflow Analyst

axiomconsultants; United States; On-site; USD 80,000 - 110,000; Full time

Job summary

AxiomConsultants is looking for a skilled HPC Dataflow Analyst to manage and optimize data flow systems. Responsibilities include providing support, handling data migrations, and ensuring efficient data transfer protocols.

Candidates should hold a Bachelor's degree and have significant experience in cloud migrations, along with a strong proficiency in relevant programming languages. Join us to enhance data management and dissemination strategies.

Qualifications

• 6–8 years of professional experience in relevant field.
• Demonstrated experience migrating HPC environments to the cloud.

Responsibilities

• Provide O&M support for data flow systems.
• Identify user requirements and write implementation instructions.
• Provide guidance to government contracted vendors.
• Support software for data transfer including troubleshooting.
• Migrate data across new architectures/platforms.
• Troubleshoot data issues related to software and networks.
• Implement real-time monitoring solutions for applications.

Skills

Data flow management

Linux-based systems

Perl

Python

C

Bash scripting

Education

Bachelor's degree in a relevant field

Job description

Axiom is seeking a talented HPC Dataflow Analyst to join the team. This individual will primarily be responsible for forecast products and model guidance and will work closely with the IDP team on all data dissemination within the NCO environment.

NCEP Central Operations (NCO) provides and exchanges over 12TB per day of all different data types and formats with the NWS offices and centers, core Government partners, the education sector, and international partners to support the mission. The data types consist of Watches/Warnings and Advisories (WWAs), forecast products, model guidance, observational, radar, and satellite data, which are freely provided to the general public via web services. The inbound and outbound data transmissions and processing are supported by two Dataflow Teams within NCO: IDP Dataflow and HPC Dataflow.

Responsibilities

• Provide tier 2 O&M support during normal business hours and off-hours in an on‑call rotation for all data flow systems by: retrieving, disseminating, ingesting, processing, testing, archiving, verifying, optimizing, troubleshooting, timing, reporting, and monitoring data feeds.
• For all new data related requests, identify user requirements, write detailed implementation instructions, test changes in development whenever possible, communicate the change timelines with all users (requestor and/or Government contracted vendor partners), successfully execute NCO approved instructions in operations following all NCO CM policies, and verify all changes worked according to requirements.
• Provide guidance, optimizations, and recommendations, to all government contracted vendor partners with respect to data flowing between systems.
• Provide thorough end‑to‑end data checkouts for implementations and/or upgrades of applications/models by following the established standard procedures.
• Support all software used to transfer data products by being able to: update, modify, test, troubleshoot, and optimize code and scripts written in languages such as: Perl, Python, C, and bash.
• Be experienced with data transfer protocols on Linux based systems to transfer data in the most efficient methods using: Local Data Manager (LDM), File Transfer Protocol (FTP), FTP Secure (FTPs), Secure Copy (SCP), and Remote Sync (RSync). In addition, must have knowledge and experience transferring data in and out of cloud environments.
• Migrate data and software across new architecture and/or platforms by defining requirements, designing architectural solutions, porting, modifying, testing, and validating.
• Troubleshoot and resolve data issues related to software, hardware, network, and data, providing the root cause and recommendations for improvement using all tools and resources available.
• Perform failovers and validation steps following the approved procedures during both routine and emergency situations.
• Keep all code, software, and configuration settings identical between all applicable HPC and Virtual Machine (VM) servers unless approved by the Government.
• Develop and maintain graphing trends for all critical components of data transfers to include but not limited to: transfer rates, volumes, transfer queue sizes, and timeliness of data.
• Maintain up-to-date documentation for all systems, software, partners, and components to include but not limited to: function, versions, troubleshooting tips, standard procedures, and failover instructions.
• Create and implement real‑time monitoring solutions that detect problems/issues/errors before a customer reports them for all applications, software, and data feeds.

Required Skills/Experiences

• Demonstrated experience in migrating high‑performance computing (HPC) environments to the cloud.
• Bachelor's degree in a relevant field and 6–8 years of professional experience.