Data Engineer _ Charlotte, NC (Local / W2)

Datum Technologies Group; Charlotte (NC); On-site; USD 110,000 - 150,000; Full time

Job summary

Datumtg in Charlotte, NC seeks a Data Engineer to optimize data pipelines that read from Kinesis and write into Oracle, with better schema handling and data-quality alerting. You will build services to access NetApp object stores via the S3 API and craft analytical datasets in Parquet or Iceberg formats.

The role requires strong Python/Java data engineering skills and hands-on experience with Spark, Flink, and real-time streaming architectures.

Qualifications

• Experience building and deploying data pipelines with Spark and Flink.
• Hands-on with real-time streaming architectures and REST APIs.
• Proficient with Oracle databases and S3-compatible object stores.
• Strong Python/Java programming skills for data engineering.

Responsibilities

• Optimize data pipelines reading from Kinesis and writing into Oracle with improved exception handling and data quality alerting.
• Build services to read/write data to NetApp object store using the S3 API.
• Create analytical datasets in Parquet or Iceberg format.
• Develop housekeeping scripts for cleanup, archiving, and compression of logs and object store files.

Skills

Python

Java

Data Engineering

Spark

Distributed Systems

REST APIs

Oracle

Kinesis

Tools

Apache Spark

Apache Flink

S3 API

Parquet/ Iceberg

NetApp

SQL/PLSQL

Job description

Key responsibilities for the Data Engineer:

• Optimize data pipelines that read from Kinesis and Write into Oracle improve exception handling for schema/data type mismatches and implement alerting for data quality issues.
• Build services to read/write data to NetApp object store using the S3 API.
• Create analytical datasets in Parquet or Iceberg format.
• Develop housekeeping scripts for cleanup, archiving, and compression of logs, Object store files, etc.

Core skill requirements:

• Strong Python/Java and data engineering expertise.
• Experience building and deploying data pipelines using frameworks like Spark, Flink and understanding of distributed data systems.
• Hands-on experience with real-time streaming architectures, REST APIs, and database connectors.
• Proficient with and Relational Databases (e.g. Oracle).
• Experience working with Amazon S3 APIs for data storage and integration.