Data Operations Engineer II — Remote Data Pipelines & Automation

AccuWeather; State College (Centre County); Ibrido; USD 100.000 - 140.000; Tempo pieno

Descrizione del lavoro

AccuWeather is hiring a Data Operations Engineer II to build, monitor, and maintain data ingestion systems that move, store, and transform data across platforms. You will develop scripts in Python, PowerShell, or similar, own pipelines, and ensure data integrity in a global Agile environment.

You will work with Databricks, Data Marketplace, and Ad Platform, administer GoAnywhere, implement CI/CD pipelines, and contribute to runbooks and documentation to support reliable operations.

Competenze

• BA/BS in Computer Science, Information Technology, or related field; 3+ years in data operations, scripting, automation.
• Proficiency with Python, PowerShell, Bash or similar scripting languages.
• Experience with Azure, Databricks, Snowflake, and SQL Server; relational database management.

Mansioni

• Build, monitor, and maintain data ingestion processes and pipelines across platforms; administer Databricks, Data Marketplace, and Ad Platform.
• Develop and optimize scripts to automate data collection and monitoring in Python/PowerShell/Bash.
• Own the operational state of defined data pipelines; monitor incidents and maintenance procedures.
• Manage SQL databases for data loading and deployment (Forecast, current, historical, locations).
• Develop unit tests for Data Ops code and ensure deployment readiness.
• Administer GoAnywhere and other managed file transfer platforms.
• Build end-to-end monitoring and automated alerts to optimize pipeline health and notify appropriately.
• Create and maintain CI/CD pipelines (GitHub Actions, Azure DevOps) for reliable deployments.
• Execute scheduled maintenance, apply patches, and perform preventive checks to avoid incidents.
• Document scripts, workflows, pipelines, and deployment procedures; contribute to runbooks.

Conoscenze

Python

PowerShell

Bash

Azure

Databricks

Snowflake

SQL Server

SQL

C++

Java

DataDog

CI/CD

GoAnywhere

AWS

GCP

Git

Formazione

BA/BS in Computer Science or related field

Strumenti

GoAnywhere

Databricks

SQL Server

Git

CI/CD tools

Azure

Snowflake

DataDog

Descrizione del lavoro

AccuWeather is hiring a Data Operations Engineer II to build, monitor, and maintain data ingestion systems that move, store, and transform data across platforms. You will develop scripts in Python, PowerShell, or similar, own pipelines, and ensure data integrity in a global Agile environment.

You will work with Databricks, Data Marketplace, and Ad Platform, administer GoAnywhere, implement CI/CD pipelines, and contribute to runbooks and documentation to support reliable operations.