Data Engineer

VDart Inc; Seattle (WA); On-site; USD 96,000 - 152,000; Full time

Job summary

VDart Inc. is seeking a Data Engineer for an onsite contract in Seattle, WA. You will migrate on-prem data stores to AWS Lakehouse, refactor pipelines, and ensure data integrity across systems.

The role requires hands-on experience with SQL, Python/Java, and Spark, plus Snowflake and Iceberg. You will collaborate with stakeholders and global teams, following CI/CD practices.

Qualifications

• 3+ years hands-on software development and/or data engineering.
• 3+ years SQL development, analysis, and performance tuning.
• 3+ years Python and/or Java for data processing.
• 3+ years ETL/ELT pipeline development and troubleshooting.
• Experience with Apache Spark for distributed processing.
• Familiarity with SCD Type 2, schema evolution, and data quality controls.
• Experience with Snowflake, Iceberg, Kafka, Hadoop, and related tools.

Responsibilities

• Migrate datastore from on-prem to AWS Lakehouse platforms as part of migration factory.
• Refactor and migrate data pipelines including extraction, transformation, and scheduling.
• Ensure data integrity and consistency during migrations.
• Translate legacy SQL/Spark logic to Snowflake and Iceberg environments.
• Collaborate with stakeholders and teams to obtain migration sign-off.
• Troubleshoot pipelines, optimize performance, and ensure reliability.
• Follow SDLC/CI/CD practices and adapt to new technologies as needed.

Skills

SQL

Python

Java

ETL/ELT

Apache Spark

Snowflake

Apache Iceberg

Kafka

Hadoop

Kubernetes

Tools

Snowflake

Apache Iceberg

Kafka

Hadoop

Sybase IQ

Parquet

JSON/Avro

Job description

Role: Data Engineer

Location: Seattle, WA (Onsite)

Type: Contract

Day to Day Job Duties:

• Perform end-to-end datastore migrations from on-premises DataLake environments to AWS-hosted Lakehouse platforms as part of the migration factory team.
• Refactor and migrate existing data pipelines, including data extraction logic, transformation processes, and job scheduling.
• Execute large-scale data transfers while ensuring data integrity, completeness, reliability, and consistency between source and target platforms.
• Translate and modernize legacy SQL and Apache Spark-based processing logic for Snowflake and Apache Iceberg environments.
• Analyze existing data usage patterns, business requirements, and downstream consumption to support the development and delivery of reusable data products.
• Design and build data reconciliation and validation frameworks to verify data accuracy during and after migration.
• Collaborate with business stakeholders, application teams, data owners, and technical teams to perform validation and obtain migration sign-off.
• Act as a technical liaison between migration, data engineering, application, infrastructure, and business teams throughout the migration lifecycle.
• Troubleshoot data pipeline, data transformation, performance, and migration-related issues and implement appropriate technical solutions.
• Optimize data pipelines and processing workloads to improve performance, scalability, and operational efficiency.
• Apply core data engineering concepts including Slowly Changing Dimensions (SCD Type 2), schema evolution, partitioning, clustering, normalization and denormalization, natural and surrogate keys, and data quality controls.
• Work with structured and semi-structured data formats including JSON, Avro, and Parquet.
• Follow established Software Development Life Cycle (SDLC), source-control, testing, deployment, and Continuous Integration/Continuous Deployment (CI/CD) practices.
• Adapt to new data technologies, migration tools, engineering standards, and workflows as required by the program.
• Collaborate effectively with geographically distributed and global delivery teams.

Basic Qualifications:

• Minimum 3+ years of hands-on software development and/or data engineering experience, including coding, development, troubleshooting, and implementation of data-related solutions.
• Minimum 3+ years of experience working with SQL, including development, analysis, query troubleshooting, and performance optimization.
• Minimum 3+ years of hands-on programming experience using Python and/or Java for data processing, application development, automation, or integration.
• Minimum 3+ years of experience developing or supporting ETL/ELT data pipelines, including data extraction, transformation, loading, and pipeline troubleshooting.
• Experience developing or supporting distributed data-processing solutions using Apache Spark.
• Experience working with data engineering concepts including SCD Type 2, schema evolution, partitioning, clustering, normalization versus denormalization, natural versus surrogate keys, and data quality frameworks.
• Experience working with one or more data and integration technologies including Kafka, ANSI SQL, FTP, Apache Spark, Hadoop, Snowflake, Apache Iceberg, and Sybase IQ.
• Experience working with structured and semi-structured data formats including JSON, Avro, and Parquet.
• Experience working within established SDLC and CI/CD processes, including source control, testing, deployment, and release practices.
• Familiarity with containerized application environments and Kubernetes.
• Demonstrated ability to troubleshoot technical issues, communicate effectively with stakeholders, collaborate across global teams, and take ownership of assigned deliverables.

Nice to Have:

• Experience performing large-scale data platform or datastore migration initiatives.
• Experience migrating data workloads from on-premises environments to cloud-based platforms, particularly AWS.
• Experience working with Lakehouse architectures.
• Experience with Snowflake and Apache Iceberg-based data platforms.
• Experience working within the financial services industry.
• Experience supporting complex migration programs involving multiple business, technology, and global delivery teams.