GCP Data Engineer

Compunnel Inc.; Charlotte (NC); On-site; USD 90,000 - 120,000; Full time

Job summary

A leading IT services firm is seeking a Data Engineer for a contract position based in Charlotte, NC. This mid-senior level role involves designing and implementing ETL solutions using Python and AWS. Responsibilities include developing data integration systems and documenting solutions. Ideal candidates will have strong experience with data engineering, AWS services, and SQL programming.

Qualifications

• Mid-Senior level experience in Data Engineering required.
• Hands-on experience with ETL tools and AWS services essential.
• Strong SQL skills for database programming and performance tuning needed.

Responsibilities

• Provide scoping, estimating, planning, design, development, and support services.
• Identify and develop technical design documents.
• Design, develop, and maintain ETL pipelines to support project initiatives.
• Coordinate with vendors and contractors for systems.

Skills

Data Engineering

Python

PySpark

AWS Services

SQL programming

REST APIs

CI/CD

Agile/SCRUM

Tools

AWS Glue

AWS Lambda

Control-M

Bitbucket

Job description

Job Title: Data Engineer

Location: Charlotte, NC (Onsite - 3 days a week)

Duration: Long Term

Job Description:

The primary role of the Data Engineer is to function as a critical member of a data team by designing data integration solutions that deliver business value in line with the company’s objectives.

They are responsible for the design and development of data/batch processing, data manipulation, data mining, and data extraction/transformation/loading into large data domains using Python/Pyspark and AWS tools.

Responsibilities:

• Provide scoping, estimating, planning, design, development, and support services to a project.
• Identify and develop the technical detail design document.
• Work with developers and business areas to design, configure, deploy, and maintain custom ETL Infrastructure to support project initiatives.
• Design and develop data/batch processing, data manipulation, data mining, and data extraction/transformation/loading (ETL Pipelines) into large data domains.
• Document and present solution alternatives to clients, which support business processes and business objectives.
• Work with business analysts to understand and prioritize user requirements.
• Design, development, test, and implement application code.
• Follow proper software development lifecycle processes and standards.
• Quality Analysis of the products, responsible for the Defect tracking and Classification.
• Track progress and intervene as needed to eliminate barriers and ensure delivery.
• Resolve or elevate problems and manage risk for both development and production support.
• Coordinate vendors and contractors for specific projects or systems.
• Maintain deep knowledge and awareness of technical & industry best practices and trends, especially in technology & methodologies.

Skills and Knowledge:

• Developer experience specifically focusing on Data Engineering.
• Hands‑on experience in Development using Python and PySpark as an ETL tool.
• Experience in AWS services like Glue, Lambda, MSK (Kafka), S3, Step functions, RDS, EKS etc.
• Experience with SQL database programming, SQL performance tuning, relational model analysis, queries, stored procedures, views, functions, and triggers.
• Strong technical experience in Design (Mapping specifications, HLD, LLD), Development (Coding, Unit testing).
• Experience with data models, data mining, data analysis and data profiling.
• Experience in working with REST APIs.
• Experience with workload automation tools like Control‑M, Autosys etc.
• Good knowledge in CI/CD DevOps process and tools like Bitbucket, GitHub, Jenkins.
• Strong experience with Agile/SCRUM methodology.
• Experience with other ETL tools (DataStage, Informatica, Pentaho, etc.).
• Knowledge in MDM, Data warehouse and Data Analytics

Seniority level

Mid-Senior level

Employment type

Contract

Job function

Other, Analyst, and Consulting

Industries

IT Services and IT Consulting, IT System Testing and Evaluation, and IT System Data Services

Referrals increase your chances of interviewing at Compunnel Inc. by 2x.