Lead Data Engineering

Cloudious LLC; Charlotte (NC); On-site; USD 120,000 - 180,000; Full time

Job summary

Cloudious LLC is seeking a Lead Data Engineer to design, implement, and maintain data pipelines and ensure data integrity for analytics and reporting. The role requires hands-on expertise in Oracle RDMS, SQL, PL/SQL, Python and PySpark, with experience in ETL, scheduling with Autosys, and source control via GitHub and Copilot.

The incumbent will collaborate with cross-functional teams in a fast-paced environment to deliver scalable data solutions and ensure data quality across analytics

Qualifications

• Expert in Oracle RDMS with experience creating objects.
• Strong SQL/PLSQL development and optimization.
• Experience with data pipelines, ETL and data modeling.
• Hands-on scripting in K shell (sed/awk/grep) and SQL Plus.
• Familiar with GitHub workflows and AI-assisted coding.

Responsibilities

• Design, develop, and optimize complex SQL queries and PL/SQL scripts for ETL.
• Create and manage Oracle database objects including tables, views, indexes, and partitioned objects.
• Implement and manage file transfers using NDM (Connect:Direct) for secure data movement.
• Utilize Autosys for job scheduling and monitoring to ensure timely execution of data processes.
• Develop and maintain data processing workflows using Python and PySpark for large-scale data processing.
• Leverage GitHub as a code repository for version control and collaboration on data engineering projects.
• Utilize GitHub CoPilot to enhance development efficiency and streamline coding processes.
• Collaborate with cross-functional teams to gather requirements and deliver data solutions that meet business needs.
• Ensure data quality and integrity through rigorous testing and validation processes.
• Stay updated with industry trends and best practices in data engineering and contribute to continuous improvement initiatives.

Skills

Oracle RDMS

SQL

PL/SQL

K shell scripting

Python

PySpark

GitHub

GitHub Copilot

NDM Connect:Direct

Autosys

SQL Plus

Tools

NDM (Connect:Direct)

Autosys

GitHub

GitHub Copilot

Job description

We are seeking a highly skilled Lead Data Engineer to join our dynamic team. The ideal candidate will possess strong hands‑on experience in advanced Oracle RDMS, SQL, PL/SQL, and various data engineering tools and technologies. This role requires a deep understanding of data architecture, data modeling, and the ability to work collaboratively in a fast‑paced environment. The Lead Data Engineer will be responsible for designing, implementing, and maintaining data pipelines and ensuring data integrity and availability for analytics and reporting.

Responsibilities:

• Design, develop, and optimize complex SQL queries and PL/SQL scripts for data extraction, transformation, and loading (ETL).
• Create and manage Oracle database objects including tables, views, indexes, and partitioned objects.
• Implement and manage file transfers using NDM (Connect:Direct) to ensure secure and efficient data movement.
• Utilize Autosys for job scheduling and monitoring to ensure timely execution of data processes.
• Develop and maintain data processing workflows using Python and PySpark for large-scale data processing.
• Leverage GitHub as a code repository for version control and collaboration on data engineering projects.
• Utilize GitHub CoPilot to enhance development efficiency and streamline coding processes.
• Collaborate with cross-functional teams to gather requirements and deliver data solutions that meet business needs.
• Ensure data quality and integrity through rigorous testing and validation processes.
• Stay updated with industry trends and best practices in data engineering and contribute to continuous improvement initiatives.

Mandatory Skills:

• Advanced Oracle RDMS expertise, including object creation (tables, views, indexes, partitioned objects).
• Proficient in SQL and PL/SQL for data manipulation and analysis.
• Strong experience with K shell scripting, including sed, awk, grep, and invoking SQL Plus shell.
• Hands‑on experience with NDM (Connect:Direct) for file transfers.
• Familiarity with Autosys for job scheduling and monitoring.
• Proficient in Python and PySpark for data processing and analytics.
• Experience using GitHub as a code repository.
• Comfortable using GitHub CoPilot for AI development efficiencies.

Preferred Skills:

• Experience with data warehousing concepts and technologies.
• Knowledge of cloud‑based data solutions (e.g., AWS, Azure, Google Cloud).
• Familiarity with data visualization tools (e.g., Tableau, Power BI).
• Understanding of data governance and compliance standards.