Lakehouse Migration Data Engineer

Compunnel, Inc.; Dallas (TX); On-site; USD 90,000 - 120,000; Full time

Job summary

A technology solutions provider is seeking a highly motivated Engineer for a critical Data Lake migration project in Dallas, TX. The role involves end-to-end migration processes, ensuring data integrity, and collaborating with stakeholders. Candidates should possess strong data engineering skills, coding proficiency in Python or Java, and familiarity with CI/CD best practices. This is a 100% onsite position, requiring applicants to be local to the area.

Qualifications

• Minimum 3-5 years of hands-on coding experience in a team-based setup.
• Strong understanding of Software Development Life Cycle (SDLC).
• Advanced knowledge of data partitioning and clustering.

Responsibilities

• Refactor and migrate extraction logic to the Lakehouse.
• Execute migration of datasets ensuring data integrity.
• Translate and optimize SQL and Spark-based patterns for Snowflake compatibility.

Skills

Data engineering skills

Strong communication

Collaboration abilities

Troubleshooting (SQL)

Basic scripting

Proficiency in Python or Java

Understanding of data formats like JSON, Avro, Parquet

Experience with CI/CD best practices

Familiarity with data extraction tools

Ability to work across time zones

Education

Bachelor’s or Master’s degree in Computer Science or related field

Tools

Kafka

Apache Spark

Snowflake

Hadoop

Apache Iceberg

Job description

A technology solutions provider is seeking a highly motivated Engineer for a critical Data Lake migration project in Dallas, TX. The role involves end-to-end migration processes, ensuring data integrity, and collaborating with stakeholders. Candidates should possess strong data engineering skills, coding proficiency in Python or Java, and familiarity with CI/CD best practices. This is a 100% onsite position, requiring applicants to be local to the area.