Sr. Big Data Engineer(Data and ETL)

Zenotis Group; Austin (TX); Remote; USD 90,000 - 130,000; Full time

Job summary

An established industry player is seeking a Sr. Big Data Engineer to lead data engineering and ETL processes in a remote setting. This role requires extensive experience in Python programming, big data technologies, and managing AWS resources. You will be responsible for developing test automation utilities and ensuring data quality through effective validation techniques. Join a dynamic team where your expertise will contribute to innovative data solutions and drive impactful projects. If you are passionate about data engineering and thrive in a collaborative environment, this opportunity is perfect for you.

Qualifications

• 8+ years of experience in Python programming and big data technologies.
• Proficient in SQL and data reconciliation validation.

Responsibilities

• Develop test automation utilities for data validation using Python.
• Manage AWS resources like Glue, EC2, and Lambda for ETL processes.

Skills

Python Programming

Test Automation Utilities

SQL

Data Reconciliation Validation

Big Data Technologies

Tools

AWS Glue

AWS EC2

AWS Lambda

pandas

numpy

pyspark

Job description

Join to apply for the Sr. Big Data Engineer (Data and ETL) role at Zenotis Group.

Position details:

Location:

Austin

Employment Type:

Contract

Workplace Type:

Remote

Minimum Experience:

8+ years

Key Responsibilities and Requirements:

• 8+ years of hands-on experience in Python programming.
• 8+ years of experience developing test automation utilities for data reconciliation validation using Python data science libraries.
• Proficiency in pandas, numpy, pyspark, and big data technologies.
• 8+ years of experience managing resources like AWS Glue, EC2, and Lambda.
• Proficient in writing complex SQL queries.
• Understanding of the testing lifecycle.
• Experience handling ETL testing for various data sources (on-premises and cloud).

Application form: Please provide your Full Name, Email, Phone, and upload your CV/Resume (.pdf, .doc, .docx).