Senior Data Engineer / 12+yrs/ Local or near by State only

Visionary Innovative Technology Solutions LLC; Virginia; On-site; USD 96,000 - 138,000; Full time

Job summary

Visionary Innovative Technology Solutions LLC is seeking a Senior Data Engineer to lead cloud-native data pipelines on AWS and Snowflake. You will drive Spark-on-EKS, Snowflake capabilities, and PoCs for AI-powered data solutions.

You will mentor mid-level developers, collaborate with stakeholders, and architect end-to-end pipelines using AWS, Snowflake, IICS, and PySpark. Strong leadership and hands-on expertise are essential for success.

Qualifications

• Extensive experience designing cloud-native data pipelines on AWS.
• Strong Snowflake expertise and data governance capabilities.
• Proficient in PySpark and Snowpark for large-scale processing.
• Experience with IICS for file-based ETL ingestion.
• Leadership and mentoring of engineers on complex data projects.

Responsibilities

• Design and deploy containerized PySpark workloads on EKS.
• Build and tune EMR clusters on EC2 or EKS for large-scale processing.
• Orchestrate workflows using AWS Step Functions and Control-M.
• Lead IICS file-based ingestion design and data ingestion pipelines.
• Conduct code reviews, optimize performance, and mentor engineers.

Skills

AWS data services

PySpark

Snowpark

RAG PoCs

Agentic AI

SQL

Data modeling

Leadership

Mentoring

Control-M

Tools

IICS

EKS

EMR

Snowflake

DynamoDB

Glue Data Catalog

Job description

Job Title :- Senior Data Engineer / 12+yrs/ Local or near by State only

Employment Type :- Contract

Job Description :-

Lead the design and development of cloud-native data pipelines on AWS and Snowflake.

Drive Spark-on-EKS, advanced Snowflake capabilities, and Al/Agentic Al proofs of concept (PoCs).

Mentor mid-level developers and collaborate with stakeholders.

Architect end-to-end pipelines using AWS, Snowflake, IICS, and PySpark.

• Design and deploy containerized PySpark workloads on EKS.
• Build and tune EMR clusters on EC2 or EKS for large-scale processing.
• Orchestrate workflows using AWS Step Functions and Control-M.

Manage metadata using AWS Glue Data Catalog and integrate it with Snowflake Iceberg Use DynamoDB for pipeline state, checkpoints, and configuration.

Lead implementation of Snowflake capabilities, including Iceberg Tabies, Dynamic Tables, Views, Materialized Views, Tasks and Streams, Data Governance, Horizon Catalog, Semantic Views, Cortex (CoCo), Snowflake Agents, and token-usage optimization strategies.

Build RAG and Agentic Al PoCs covering vector search, BM25 hybrid retrieval, evaluation frameworks, human-in-the-loop controls, and security controls.

Develop PySpark and Snowpark pipelines for ingestion and transformation.

• Lead IICS file-based ingestion design.
• Conduct code reviews, improve performance, and mentor engineers.

Leadership and Collaboration Expectations:

Demonstrates ownership and accountability for design quality, delivery commitments, and production outcomes.

• Communicates complex technical decisions clearly to engineering teams, product partners, and stakeholders.

Collaborates across teams, constructively challenges assumptions, and drives alignment on scalable solutions.

Provides practical coaching and feedback that strengthens engineering capability and promotes knowledge sharing.

Applies sound judgment, balances speed with engineering rigor, and proactively manages risks and dependencies.

Shows curiosity and adaptability when evaluating emerging cloud, data, and Al capabilities.

Skills Required:

• Strong hands-on experience with AWS data services, including EKS, EMR, EC2, Step Functions, DynamoDB, AWS Glue Data Catalog, and S3.
• Advanced Snowflake expertise.
• Expertise in PySpark and Snowpark.
• Hands-on experience with IICS for file-based ETL ingestion.
• Experience building RAG and Agentic Al PoCs.
• Strong SQL and data modeling skills.
• Leadership and mentoring capability.
• Knowledge of Control-M orchestration.

Must-Have Skills:

• AWS: EKS, EMR, ECZ, Step Functions, DynamoDB, AWS Glue Data Catalog, S3, and IAM.
• Snowflake: Iceberg, Dynamic Tables, Views, Materialized Views, Tasks, Governance, Horizon, Cortex, and Agents.
• ICS for file-based ETL ingestion.
• Strong SQL and data modeling.
• Experience with RAG or Agentic Al PoCs.
• Awareness of reporting tools such as Power Bi or Tableau.