Senior Data Engineer - Hybrid Cloud Pipelines

Motion Recruitment; Charlotte (NC); Hybrid; USD 110,000 - 140,000; Full time

Job summary

Motion Recruitment is seeking a Software Engineer on a long-term contract in Charlotte, NC (Hybrid) to design and build scalable data pipelines and modernize data platforms across on-prem and Google Cloud. The role emphasizes hands-on development with AB Initio, Python, and SQL, plus migration to BigQuery.

The ideal candidate has 4+ years in data engineering, strong SQL/PL/SQL skills, and experience with ETL, data warehousing, and governance.

Qualifications

• 4+ years of Data Engineering experience or equivalent demonstrated through one or more: work experience, training, military experience, education.
• 4+ years PL/SQL and SQL skills with proven experience in Oracle, Teradata, Python and/or BigQuery: complex query development, tuning, and debugging.
• 4+ years Ab Initio skills with proven experience to build complex graphs, Psets, performance tuning.
• 3+ years programming skills in Python; hands-on PySpark for distributed data processing
• 3+ years of ETL/ETL design, data warehousing concepts, and data modeling best practices

Responsibilities

• Build and maintain scalable batch and near real-time data pipelines using AB Initio, Python, PySpark, PL/SQL and SQL.
• Develop and optimize BigQuery transformations and data models, including partitioning, clustering, and cost/performance tuning.
• Support modernization/migration from Teradata and Ab Initio workflows to GCP/BigQuery with re-platforming and cutovers.
• Implement orchestration and scheduling for pipelines using Autosys and migrating toward Airflow (Google Cloud Composer).
• Apply data governance and discovery practices using Dataplex: metadata management, classification, and lineage.
• Build and operationalize data quality controls using Informatica Data Quality: profiling, rules, thresholds, and exception handling.
• Ensure operational excellence: monitoring, alerting, runbooks, incident triage, and continuous improvements.
• Implement secure data engineering practices: least-privilege access, PII handling, retention controls and audit-friendly docs.
• Partner with product, analytics, and engineering to translate requirements into data contracts and documentation.
• Must-have: use AI-assisted coding tools to accelerate development with strong code reviews and secure coding.
• Collaborate with Architects to design, develop, test and support reliable reusable data pipelines.
• Analyze complex business requirements and generate technical specifications for ETL processes.
• Serve as a technical resource guiding less experienced staff and solving complex middleware problems.

Skills

Data engineering

PL/SQL

SQL

Python

PySpark

BigQuery

Ab Initio

ETL design

Data warehousing

Data modeling

AI-assisted coding

GCP knowledge

Tools

Autosys

Airflow

Git

Jenkins

uDeploy

Dataplex

Informatica Data Quality

Job description

Motion Recruitment is seeking a Software Engineer on a long-term contract in Charlotte, NC (Hybrid) to design and build scalable data pipelines and modernize data platforms across on-prem and Google Cloud. The role emphasizes hands-on development with AB Initio, Python, and SQL, plus migration to BigQuery.

The ideal candidate has 4+ years in data engineering, strong SQL/PL/SQL skills, and experience with ETL, data warehousing, and governance.