Senior Data Engineer - Real-Time ETL & Cloud Migration

Staffing Spot, Inc.; Charlotte (NC); Hybrid; USD 110,000 - 150,000; Full time

Job summary

Staffing Spot, Inc. is hiring a data engineer to build scalable batch and real-time data pipelines, using Ab Initio, Python, PySpark, SQL and PL/SQL. You will review designs and transform blueprints into reusable code, and transform mortgage data from multiple systems.

You will collaborate with product owners, architects, engineers and stakeholders; support migration to Google Cloud; and drive CI/CD and production reliability in a hybrid data environment.

Qualifications

• 4+ years of Data Engineering experience supporting data pipelines.
• 4+ years PL/SQL and SQL with Oracle, Teradata or BigQuery; complex queries.
• 4+ years Ab Initio to build graphs, Psets and performance tuning.
• 3+ years Python and PySpark for distributed data processing.

Responsibilities

• Build and maintain scalable batch and near real-time data pipelines.
• Develop ETL solutions using Ab Initio, Python, PySpark, SQL and PL/SQL.
• Review designs and translate blueprints into reusable code.
• Integrate, cleanse, normalize, and transform data from mortgage systems.
• Partner with Product Owners, Architects, Engineers, and stakeholders.
• Support migration and modernization to Google Cloud.
• Build, schedule, orchestrate, and deploy solutions via CI/CD.
• Support monitoring, incident response, and production reliability.

Skills

Python

PySpark

SQL

PL/SQL

Ab Initio

BigQuery

ETL

Data Warehousing

Data Modeling

GCP

Airflow

Autosys

CI/CD

Git

Dataplex

Informatica Data Quality

Tools

Autosys

Airflow

Git

Jenkins

uDeploy

Cloud Composer

Job description

Staffing Spot, Inc. is hiring a data engineer to build scalable batch and real-time data pipelines, using Ab Initio, Python, PySpark, SQL and PL/SQL. You will review designs and transform blueprints into reusable code, and transform mortgage data from multiple systems.

You will collaborate with product owners, architects, engineers and stakeholders; support migration to Google Cloud; and drive CI/CD and production reliability in a hybrid data environment.