Senior Data Engineer — Hybrid Cloud Pipelines

Motion Recruitment Partners LLC; Charlotte, Northern (NC, KY); Hybrid; USD 90,000 - 130,000; Full time

Job summary

Motion Recruitment Partners LLC is recruiting for a Data Engineer in Charlotte, NC, operating in a hybrid setup. The role targets building scalable batch and real-time data pipelines across on-premises and Google Cloud platforms, with migration from legacy tools to modern cloud-based solutions.

You will optimize BigQuery transformations, participate in data governance, and implement CI/CD practices using Autosys, Airflow, and other tooling, while maintaining secure data handling and

Qualifications

• 4+ years of Data Engineering experience or equivalent
• 4+ years PL/SQL and SQL with Oracle/Teradata; Python and/or BigQuery
• 4+ years Ab Initio to build graphs, Psets, tuning
• 3+ years Python; PySpark for distributed processing
• 3+ years ETL, data warehousing concepts, data modeling

Responsibilities

• Build and maintain scalable batch and near real-time data pipelines using Ab Initio, Python, PySpark, PL/SQL and SQL to ingest, transform and publish curated datasets across on-prem and Google Cloud platforms
• Develop and optimize BigQuery transformations and data models with partitioning, clustering and cost/performance tuning
• Support modernization/migration from Teradata and Ab Initio workflows to GCP/BigQuery, including cutovers
• Implement orchestration for pipelines using Autosys and migrate toward Cloud Composer (Airflow)
• Apply data governance using Dataplex: metadata, classification, lineage
• Build and operationalize data quality controls using Informatica Data Quality
• Maintain operational excellence: monitoring, alerting, runbooks and incident analysis
• Apply secure data engineering practices and audit-friendly documentation
• Partner with stakeholders to translate requirements into data contracts and datasets
• Use AI-assisted coding tools to accelerate development with strong code review and testing

Skills

Data Engineering

PL/SQL

SQL

Oracle

Teradata

Python

BigQuery

Ab Initio

PySpark

ETL

Data Warehousing

Data Modeling

Tools

Autosys

Airflow

Git

Jenkins

uDeploy

Dataplex

Informatica Data Quality

Cloud Composer

Job description

Motion Recruitment Partners LLC is recruiting for a Data Engineer in Charlotte, NC, operating in a hybrid setup. The role targets building scalable batch and real-time data pipelines across on-premises and Google Cloud platforms, with migration from legacy tools to modern cloud-based solutions.

You will optimize BigQuery transformations, participate in data governance, and implement CI/CD practices using Autosys, Airflow, and other tooling, while maintaining secure data handling and