Senior Data Engineer/Data Engineer

VDart Inc; Columbus (OH); On-site; USD 90,000 - 130,000; Full time

Job summary

VDart Inc. is seeking a Senior Data Engineer/Data Engineer in Columbus, OH to design and optimize data pipelines. You will work with Snowflake, Matillion and AWS services to develop ETL/ELT workflows, analyze data sources and translate business needs into technical solutions.

Strong communication with Sales and Architects is required. The role involves gathering requirements, validating data quality, deploying and testing data jobs in development and staging, and collaborating across onshore

Qualifications

• Translate business logic into scalable technical workflows.
• Collaborate with Sales Compensation stakeholders and architects to gather requirements.
• Analyze target data structures and underlying systems.
• Design data processing pipelines using the defined technology stack.
• Develop ETL/ELT jobs and implement data cleaning and validation steps.
• Deploy and test ETL jobs in development and staging environments.

Responsibilities

• Design data processing pipelines using Snowflake, Matillion and AWS services.
• Develop ETL/ELT workflows for ingestion and transformation.
• Validate data quality with checks and cleaning steps.
• Deploy, test and monitor jobs across environments.

Skills

Data engineering

Requirements gathering

Stakeholder communication

Proactive mindset

Tools

Snowflake

Matillion

AWS S3

AWS Lambda

AWS DynamoDB

CloudWatch

GitHub

Cortex

Job description

Role: Senior Data Engineer/Data Engineer

Location: Columbus, OH

Duration: 6 Months

Job Description

• Technology Stack Reference: Development activities will primarily leverage Snowflake, Matillion, AWS Services (S3, Lambda, DynamoDB, CloudWatch), GitHub, Cortex, and Snowflake Intelligence.
• Tech stack & core skills: Aligned with the existing CDW stack, including Snowflake, Matillion, AWS services
  (S3, Lambda, DynamoDB, CloudWatch), along with GitHub.
• Role expectation: We are looking for an expert Data Engineer with good English speaking and strong communication skills and a proactive mindset to effectively probe and gather requirements as well from the sales team.
• Stakeholder interaction: The resource gets involved in discussions with Sales Compensation stakeholders
  along with Architect and other onshore developers, engaging in discussions to capture/gather
  compensation-related requirements.
• Requirements Gathering: Collaborating with business stakeholders to map, refine, and translate business logic into scalable technical workflows.
• Data Source Analysis: Analyze target data structures and underlying systems.
• Pipeline Design: Design data processing pipelines using the defined technology stack.
• ETL/ELT Development: Development of data ingestion, processing, and Extract, Transform, Load (ETL) jobs.
• Data Cleaning & Validation: Implementing validations, data cleaning steps, and quality checkpoints.
• Deployment & Testing: Deployment and thorough testing of ETL jobs within the development and staging environments.