Lead Data Engineer - Oracle & PySpark ETL

Conch Technologies Inc; Charlotte (NC); On-site; USD 130,000 - 180,000; Full time

Job summary

Conch Technologies, Inc seeks a Lead Data Engineer in Charlotte, NC for a 12+ month contract. You will design, implement, and maintain data pipelines, emphasizing Oracle RDMS, SQL, PL/SQL, and large-scale data processing.

Responsibilities include building ETL jobs with PySpark, Python, and Autosys, managing NDM file transfers, and collaborating with cross-functional teams to ensure data quality and availability for analytics.

Qualifications

• Advanced Oracle RDMS expertise with object creation (tables, views, indexes).
• Strong SQL and PL/SQL skills for data manipulation and analysis.
• Experience with data pipelines, ETL, and large-scale processing.
• Proficiency in shell scripting for automation and data tasks.

Responsibilities

• Design, develop, and optimize complex SQL queries and PL/SQL scripts for ETL.
• Create and manage Oracle database objects including tables, views, indexes.
• Implement and manage file transfers using NDM for secure data movement.
• Utilize Autosys for job scheduling and monitoring of data processes.
• Develop data workflows using Python and PySpark for large-scale processing.
• Collaborate with cross-functional teams to deliver data solutions and ensure data quality.

Skills

Advanced Oracle RDMS

SQL

PL/SQL

K shell scripting

sed/awk/grep

NDM (Connect:Direct)

Autosys

Python

PySpark

GitHub

GitHub Copilot

Tools

NDM (Connect:Direct)

Autosys

GitHub

GitHub Copilot

Job description

Conch Technologies, Inc seeks a Lead Data Engineer in Charlotte, NC for a 12+ month contract. You will design, implement, and maintain data pipelines, emphasizing Oracle RDMS, SQL, PL/SQL, and large-scale data processing.

Responsibilities include building ETL jobs with PySpark, Python, and Autosys, managing NDM file transfers, and collaborating with cross-functional teams to ensure data quality and availability for analytics.