Lead Data Engineer

Conch Technologies Inc; Charlotte (NC); On-site; USD 130,000 - 180,000; Full time

Job summary

Conch Technologies, Inc seeks a Lead Data Engineer in Charlotte, NC for a 12+ month contract. You will design, implement, and maintain data pipelines, emphasizing Oracle RDMS, SQL, PL/SQL, and large-scale data processing.

Responsibilities include building ETL jobs with PySpark, Python, and Autosys, managing NDM file transfers, and collaborating with cross-functional teams to ensure data quality and availability for analytics.

Qualifications

• Advanced Oracle RDMS expertise with object creation (tables, views, indexes).
• Strong SQL and PL/SQL skills for data manipulation and analysis.
• Experience with data pipelines, ETL, and large-scale processing.
• Proficiency in shell scripting for automation and data tasks.

Responsibilities

• Design, develop, and optimize complex SQL queries and PL/SQL scripts for ETL.
• Create and manage Oracle database objects including tables, views, indexes.
• Implement and manage file transfers using NDM for secure data movement.
• Utilize Autosys for job scheduling and monitoring of data processes.
• Develop data workflows using Python and PySpark for large-scale processing.
• Collaborate with cross-functional teams to deliver data solutions and ensure data quality.

Skills

Advanced Oracle RDMS

SQL

PL/SQL

K shell scripting

sed/awk/grep

NDM (Connect:Direct)

Autosys

Python

PySpark

GitHub

GitHub Copilot

Tools

NDM (Connect:Direct)

Autosys

GitHub

GitHub Copilot

Job description

Greetings,

Job Title: Lead Data Engineer
Designation: Lead Data Engineering

Location: Charlotte NC

12+ Months Contract

Job Summary:

We are seeking a highly skilled Lead Data Engineer to join our dynamic team. The ideal candidate will possess strong hands-on experience in advanced Oracle RDMS, SQL, PL/SQL, and various data engineering tools and technologies. This role requires a deep understanding of data architecture, data modeling, and the ability to work collaboratively in a fast-paced environment. The Lead Data Engineer will be responsible for designing, implementing, and maintaining data pipelines and ensuring data integrity and availability for analytics and reporting.

Responsibilities:

• Design, develop, and optimize complex SQL queries and PL/SQL scripts for data extraction, transformation, and loading (ETL).
• Create and manage Oracle database objects including tables, views, indexes, and partitioned objects.
• Implement and manage file transfers using NDM (Connect:Direct) to ensure secure and efficient data movement.
• Utilize Autosys for job scheduling and monitoring to ensure timely execution of data processes.
• Develop and maintain data processing workflows using Python and PySpark for large-scale data processing.
• Leverage GitHub as a code repository for version control and collaboration on data engineering projects.
• Utilize GitHub CoPilot to enhance development efficiency and streamline coding processes.
• Collaborate with cross-functional teams to gather requirements and deliver data solutions that meet business needs.
• Ensure data quality and integrity through rigorous testing and validation processes.
• Stay updated with industry trends and best practices in data engineering and contribute to continuous improvement initiatives.

Mandatory Skills:

• Advanced Oracle RDMS expertise, including object creation (tables, views, indexes, partitioned objects).
• Proficient in SQL and PL/SQL for data manipulation and analysis.
• Strong experience with K shell scripting, including sed, awk, grep, and invoking SQL Plus shell.
• Hands-on experience with NDM (Connect:Direct) for file transfers.
• Familiarity with Autosys for job scheduling and monitoring.
• Proficient in Python and PySpark for data processing and analytics.
• Experience using GitHub as a code repository.
• Comfortable using GitHub CoPilot for AI development efficiencies.

Thanks & Regards,

Naveen Lam |Sr. IT Recruiter
E: [e-mail] | T: 9016273002 *835
Conch Technologies, Inc | www.conchtech.com