Data Engineer

SDLC Technologies; Charlotte (NC); On-site; USD 90,000 - 150,000; Full time

Job summary

SDLC Technologies in Charlotte, NC is seeking a data engineer/backend developer to design and optimise large-scale data pipelines using PySpark and Python. You will work with data scientists and analysts to translate requirements into robust, scalable solutions running on AWS services.

Responsibilities include troubleshooting issues, performing data quality checks, and tuning job performance. The role emphasizes strong SQL skills, OOP fundamentals, and teamwork to deliver reliable data workflows

Qualifications

• Hands-on experience in data engineering or backend development.
• Experience with AWS ecosystem (S3, Glue, EMR, Redshift, Lambda) and Kafka.
• Proficiency in PySpark and Python for scalable data transformations.
• Strong understanding of OOP principles and design patterns.
• Proficient SQL in writing complex queries and optimizing performance.
• Strong problem-solving and independent troubleshooting skills.
• Excellent communication and collaboration abilities.

Responsibilities

• Design, develop, and optimise large-scale data pipelines.
• Collaborate with data scientists, analysts, and stakeholders to gather requirements.
• Troubleshoot data issues and ensure timely resolutions.
• Conduct code reviews and data quality checks.
• Tune performance of data jobs and manage cloud-native workflows.

Skills

PySpark

Python

SQL

OOP

Data engineering

Problem solving

Communication

Collaboration

Tools

AWS

S3

EMR

Glue

Redshift

Lambda

Kafka

Job description

Design, develop, and optimise large-scale data pipelines using PySpark and Python.

Implement and adhere to best practices in object-oriented programming to build reusable, maintainable code.

Write advanced SQL queries for data extraction, transformation, and loading (ETL).

Collaborate closely with data scientists, analysts, and stakeholders to gather requirements and translate them into technical solutions.

Troubleshoot data-related issues and resolve them in a timely and accurate manner.

Leverage AWS cloud services (e.g., S3, EMR, Lambda, Glue) to build and manage cloud-native data workflows (preferred).

Participate in code reviews, data quality checks, and performance tuning of data jobs.

Required Skills & Qualifications:

• Relevant experience in a data engineering or backend development role.
• Experience working with AWS cloud ecosystem (S3, Glue, EMR, Redshift, Lambda, etc.). Kafka
• Strong hands-on experience with PySpark and Python, especially in designing and implementing scalable data transformations.
• Solid understanding of Object-Oriented Programming (OOP) principles and design patterns.
• Proficient in SQL, with the ability to write complex queries and optimise performance.
• Strong problem-solving skills and the ability to troubleshoot complex data issues independently.
• Excellent communication and collaboration skills.