Snowflake Data Engineer

TechDigital Group; Alpharetta (GA); Sur place; USD 90 000 - 150 000; Plein temps

Résumé du poste

An innovative firm is seeking a skilled Snowflake Data Engineer to spearhead data migration projects from DB2 to Snowflake. This role involves extensive experience with Snowflake SQL, building ETL pipelines using Python and Spark, and leveraging AWS cloud for seamless data operations. You'll be responsible for schema migrations, optimizing ETL processes, and ensuring data integrity throughout the migration process. Join a dynamic team where your expertise will drive impactful data solutions and enhance cloud operations. If you're passionate about data engineering and cloud technologies, this opportunity is tailored for you.

Qualifications

• Extensive experience with Snowflake data migration from DB2.
• Proficiency in building ETL pipelines using Python and Spark.

Responsabilités

• Migrate schema and data to Snowflake, ensuring integrity.
• Design and implement features for identity and access management.
• Optimize ETL processes and ensure archival strategy.

Connaissances

Snowflake data migration

Snowflake SQL

Data migration techniques

AWS cloud platform

Informatica

Abinitio

ETL pipelines

Python

Spark

AWS Glue

Description du poste

Job Title: Snowflake Data Engineer

Job Requirements:

1. Extensive experience in Snowflake data migration, preferably from DB2 to Snowflake.
2. Proficiency in Snowflake SQL and snowproc.
3. Strong understanding of data migration techniques.
4. Experience with AWS cloud platform.
5. Experience in Informatica/Abinitio.
6. Experience in building ETL pipelines using Python, Spark, and AWS Glue.

Responsibilities:

1. Schema Migration – Index, Constraints, and Stored Procedures.
2. Data Migration – Historical Load.
3. Cloud Operations – Snowflake.
4. Design and implement features for identity and access management.
5. Create migration/ingestion pipelines into Snowflake.
6. Implement query optimization and solve performance and scalability issues.
7. Build, monitor, and optimize ETL and ELT processes with data models.
8. Ensure archival strategy.
9. Maintain testing tracker highlighting data comparability between Netezza and Snowflake.
10. Data orchestration/development and automation of data workflows (CI/CD).
11. Testing between DB2 tables and Snowflake tables, including data types.
12. Maintain test results and validation activities in a tracker.
13. Collaborate with Business/Tech SME to write test cases for data validation.
14. Facilitate user-level testing.
15. Prepare test case scenarios for validation or performance testing across DB2 data set vs Snowflake staging tables/data mart.