Data Engineer - Python, dbt & Oracle

Interon IT Solutions; Jersey City (NJ); Hybrid; USD 120,000 - 180,000; Full time

Job summary

Interon IT Solutions in Jersey City, NJ is seeking a senior Data Engineer to modernize a large data platform. You will migrate 700-800 legacy Perl scripts and RunMyJobs/PL/SQL workflows to Python, dbt, Airflow, and Oracle 19c.

You will design production Airflow DAGs, develop custom operators, implement data quality and lineage monitoring, and collaborate with DBAs to ensure reliable, observable pipelines across on‑prem systems.

Qualifications

• 5+ years of Python development for automation and enterprise ETL pipelines.
• 2+ years of producing production-grade Airflow DAGs with custom operators and sensors.
• Strong experience with Oracle SQL, PL/SQL development, complex queries, and performance tuning.
• Experience modernizing or replacing legacy schedulers like RunMyJobs or AutoSys.

Responsibilities

• Assess, redesign, and migrate legacy Perl scripts and RunMyJobs/PL/SQL workflows to Python scripts and Airflow DAGs.
• Design, build, and deploy production-grade Airflow DAGs for interdependent batch workflows.
• Develop custom Airflow operators and sensors for Oracle 19c, REST APIs, and file transfers.
• Build and maintain data transformation workflows using dbt.
• Write and optimize complex Oracle SQL/PLSQL queries for performance.
• Collaborate with DBAs and application teams to document dependencies and improve data quality.

Skills

Python

Airflow

dbt

Oracle 19c

PL/SQL

REST API

SFTP/FTP

Linux/Unix

RunMyJobs

ETL pipelines

Tools

Git

Jenkins

Bitbucket

Job description

Job Title: Data Engineer Python, dbt & Oracle

Location: Jersey City, NJ Hybrid, 3 days onsite

Worksite: 185 Hudson Street, Suite 1150, Jersey City, NJ 07311

Work Authorization: U.S. Citizens and Green Card holders only

Candidate Requirement: Local candidates only

Interview Process: First-round video interview followed by a two-hour onsite final interview

Position Overview

Our Client is seeking an experienced Data Engineer to support a major data infrastructure modernization initiative. This project involves assessing and modernizing approximately 700 800 legacy Perl scripts and RunMyJobs/PL/SQL scheduling workflows.

The selected candidate will help migrate these legacy processes to a modular, observable, and testable data platform using Python, dbt, Apache Airflow, and Oracle 19c.

Key Responsibilities

• Assess, redesign, and migrate legacy Perl scripts, RunMyJobs workflows, and PL/SQL scheduling logic to Python scripts and Apache Airflow DAGs.
• Design, build, and deploy production-grade Airflow DAGs for complex, interdependent enterprise batch workflows.
• Develop custom Airflow operators and sensors for Oracle 19c, REST APIs, SFTP/FTP transfers, and external triggers.
• Build and maintain data transformation workflows using dbt.
• Write and optimize complex Oracle SQL and PL/SQL queries.
• Improve the performance and reliability of data pipelines operating across on-premises systems.
• Implement error handling, retry processes, SLA alerts, execution logs, audit trails, and data-lineage monitoring.
• Troubleshoot failed or partially completed jobs across application logs, databases, and Linux/Unix filesystems.
• Work closely with DBAs and application teams to document job dependencies, identify critical processing paths, and improve data quality and recoverability.
• Support parameter-driven automation and reusable pipeline-development standards.

Required Qualifications

• Strong hands-on experience with dbt and Oracle 19c. Both are mandatory.
• At least five years of Python development experience, particularly in automation, scripting, and enterprise ETL pipelines.
• At least two years of experience developing and deploying production-grade Apache Airflow DAGs, custom operators, and sensors.
• Strong experience with Oracle SQL, PL/SQL development, complex queries, and performance tuning.
• Advanced knowledge of Linux/Unix environments and shell-based processing.
• Experience modernizing or replacing enterprise job schedulers such as RunMyJobs or AutoSys.
• Experience developing REST API integrations and SFTP/FTP file-transfer processes.
• Strong understanding of batch processing, job dependencies, failure recovery, and operational monitoring.
• Proven ability to troubleshoot issues across application logs, databases, filesystems, and data pipelines.

Preferred Qualifications

• Experience modernizing or replacing legacy Perl applications or scripts.
• Previous experience in banking, investment management, or financial services.
• Familiarity with financial data such as holdings, transactions, net asset values, and tax lots.
• Experience with Git or Bitbucket, Jenkins, and CI/CD-based deployment processes.
• Knowledge of operational risk, data governance, audit controls, and data lineage within financial systems.