Data Engineer

VDart Inc; Charlotte (NC); Hybrid; USD 83,000 - 152,000; Full time

Job summary

VDart Inc. is seeking a Data Engineer in Charlotte, NC to design and operate cloud-based data solutions for cybersecurity analytics. The role emphasizes ETL development with AWS Glue, data warehousing, data lakes, and Snowflake, with Terraform IaC and API integration.

You will collaborate with cross-functional teams to ensure secure, scalable data processing and robust analytics capabilities. Hybrid work arrangement and contract engagement are offered.

Qualifications

• This role requires 5+ years of cloud-based data engineering experience.
• Hands-on AWS Glue ETL development and job orchestration.
• Terraform for cloud infrastructure automation and deployment.
• Strong Data Warehousing and Data Lake design experience.
• Proficiency with Snowflake for storage, modeling, and queries.
• Experience crafting API calls for data ingestion from apps.
• Proficiency in SQL, Python, and Spark for data transformations.
• Experience using Kafka in ETL processes and data pipelines.
• Knowledge of data security practices, encryption, and access control.

Responsibilities

• Develop and maintain AWS Glue ETL pipelines for cybersecurity data.
• Design, implement, and optimize Data Warehouses and Data Lakes for security analytics.
• Manage Terraform-based Infrastructure-as-Code deployments.
• Work with Snowflake to optimize secure data storage and performance.
• Create and manage API calls to ingest and exchange data.
• Assist in data governance, security controls, and compliance.
• Monitor production data jobs and troubleshoot failures.
• Participate in code reviews and architecture discussions.

Skills

AWS Glue

Apache Kafka

Terraform

Data Warehousing

Data Lakes

Snowflake

API development

SQL

Python

Spark

RESTful APIs

Tools

SSIS

Job description

Job Title: Data Engineer

Location: Charlotte, NC (Hybrid)

Type: Contract

Job Description

• This role involves working with cross-functional teams to develop, manage, and optimize cloud-based data solutions for cybersecurity analytics.
• The ideal candidate will have experience in AWS Glue, Apache Kafka, Terraform, Data Warehousing, Data Lakes, Snowflake, and API development to ensure secure, scalable, and efficient data processing and storage.

Responsibilities

• Develop and maintain AWS Glue ETL pipelines to process and transform cybersecurity-related data.
• Design, implement, and optimize Data Warehouses and Data Lakes to support security analytics and reporting.
• Manage Terraform-based Infrastructure-as-Code (IaC) deployments.
• Work with Snowflake to develop and optimize secure data storage, querying, and performance tuning for large datasets.
• Create and manage API calls to ingest, transform, and exchange data between front-end applications and backend data systems.
• Assist in data governance, security controls, and compliance within cloud-based data platforms.
• Monitor production data jobs, troubleshoot failures, and execute batch processing efficiently.
• Participate in code reviews, security audits, and architecture discussions to improve data workflows.

Job Requirements

• 5+ years of experience in cloud-based data engineering and big data processing.
• Hands-on experience with AWS Glue for ETL development and job orchestration.
• Expertise in Terraform for cloud infrastructure automation and deployment.
• Strong background in Data Warehousing and Data Lake architectures (AWS S3, Snowflake
• , etc.).
• Proficiency in Snowflake for data storage, modelling, and query performance optimization.
• Experience in creating and managing API calls for data ingestion from front-end applications and external systems.
• Experience in creating SSIS packages/ Debugging the SSIS Packages.
• Knowledge of RESTful APIs, JSON, and authentication mechanisms (OAuth, API keys, etc.).
• Strong skills in SQL, Python, and Spark for data transformations and analysis.
• Hands on experience using Apache Kafka in ETL process
• Experience with data security best practices, encryption, and access control in cloud environments