Data Engineer

Quantum Technologies LLC; Charlotte (NC); On-site; USD 96,000 - 131,000; Full time

Job summary

Quantum Technologies. LLC in Charlotte, NC is seeking a Data Engineer to optimize data pipelines from Kinesis to Oracle, with emphasis on real-time streaming and S3 data integration.

You will build services to access NetApp object stores via the S3 API, create analytical datasets in Parquet/Iceberg formats, and develop housekeeping scripts for logs and archives. Strong Python/Java and distributed data systems experience is required.

Qualifications

• Strong Python/Java and data engineering expertise.
• Experience building and deploying data pipelines using frameworks like Spark, Flink and understanding of distributed data systems.
• Hands-on experience with real-time streaming architectures, REST APIs, and database connectors.
• Proficient with and Relational Databases (e.g. Oracle).
• Experience working with Amazon S3 APIs for data storage and integration.

Responsibilities

• Optimize data pipelines that read from Kinesis and Write into Oracle improve exception handling for schema/data type mismatches and implement alerting for data quality issues.
• Build services to read/write data to NetApp object store using the S3 API.
• Create analytical datasets in Parquet or Iceberg format.
• Develop housekeeping scripts for cleanup, archiving, and compression of logs, Object store files, etc.

Skills

Python

Java

Data engineering

Spark

Flink

REST APIs

Distributed data systems

Oracle

Amazon S3

Job description

US Citizen, Green Card, H-1B, OPT-EAD, CPT, Other

Job Title: Data Engineer Location: Charlotte, NC Duration: 8 Months + Extension Bill Rate: $82/hour (C2C) Job Type: W-2, 1099, C2C Client: To Be Discussed Later Work Authorization: US Citizen, Green Card, OPT-EAD, CPT, H-1B, H4-EAD, Other

Job Description

• Optimize data pipelines that read from Kinesis and Write into Oracle improve exception handling for schema/data type mismatches and implement alerting for data quality issues.
• Build services to read/write data to NetApp object store using the S3 API.
• Create analytical datasets in Parquet or Iceberg format.
• Develop housekeeping scripts for cleanup, archiving, and compression of logs, Object store files, etc.

Core skill requirements

• Strong Python/Java and data engineering expertise.
• Experience building and deploying data pipelines using frameworks like Spark, Flink and understanding of distributed data systems.
• Hands-on experience with real-time streaming architectures, REST APIs, and database connectors.
• Proficient with and Relational Databases (e.g. Oracle).
• Experience working with Amazon S3 APIs for data storage and integration.

We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.