Senior Big Data Engineer – Spark & Cloud

ALLTECH CONSULTING SVC INC; Alpharetta (GA); On-site; USD 110,000 - 140,000; Full time

Job summary

A consultancy firm is seeking a Big Data Engineer in Alpharetta, GA to design and develop automation frameworks for ETL processing in Wealth Management. The ideal candidate should have over 7 years of data engineering experience along with expertise in Unix Shell, Python, and Spark. Strong skills in SQL and familiarity with cloud platforms like AWS or Azure are preferred. This role involves supporting existing frameworks and collaborating with teams to optimize processes for improved efficiency.

Qualifications

• 7+ years of data engineering experience required.
• Strong Unix Shell and Python scripting expertise.
• Experience with Spark and SQL essential.

Responsibilities

• Design and develop new automation framework for ETL processing.
• Support existing framework as technical point of contact.
• Collaborate on issues and new features.

Skills

Data engineering experience

Unix Shell scripting

Python knowledge

Spark expertise

SQL knowledge

HDFS knowledge

GitHub knowledge

Communication skills

Tools

Snowflake

ETL tools

Cloud platforms (Azure, AWS)

Job description

A consultancy firm is seeking a Big Data Engineer in Alpharetta, GA to design and develop automation frameworks for ETL processing in Wealth Management. The ideal candidate should have over 7 years of data engineering experience along with expertise in Unix Shell, Python, and Spark. Strong skills in SQL and familiarity with cloud platforms like AWS or Azure are preferred. This role involves supporting existing frameworks and collaborating with teams to optimize processes for improved efficiency.