Big Data Engineer

ALLTECH CONSULTING SVC INC; Alpharetta (GA); On-site; USD 110,000 - 140,000; Full time

Job summary

A consultancy firm is seeking a Big Data Engineer in Alpharetta, GA to design and develop automation frameworks for ETL processing in Wealth Management. The ideal candidate should have over 7 years of data engineering experience along with expertise in Unix Shell, Python, and Spark. Strong skills in SQL and familiarity with cloud platforms like AWS or Azure are preferred. This role involves supporting existing frameworks and collaborating with teams to optimize processes for improved efficiency.

Qualifications

• 7+ years of data engineering experience required.
• Strong Unix Shell and Python scripting expertise.
• Experience with Spark and SQL essential.

Responsibilities

• Design and develop new automation framework for ETL processing.
• Support existing framework as technical point of contact.
• Collaborate on issues and new features.

Skills

Data engineering experience

Unix Shell scripting

Python knowledge

Spark expertise

SQL knowledge

HDFS knowledge

GitHub knowledge

Communication skills

Tools

Snowflake

ETL tools

Cloud platforms (Azure, AWS)

Job description

Job Description

Position: Big Data Engineer – Wealth Management

About the Company

Since our founding in 1935, we have been committed to serving local and global communities by leading in Investment Banking, Securities, Investment Management and Wealth Management services. Our belief that capital can work to benefit all of society inspires us to put our clients first, lead with exceptional ideas, hold our business to high ethical standards, and give back to communities around the world through philanthropy and public works. We maintain a smart‑casual dress code and balance work with personal life. Diversity and inclusiveness are a critical component of our success, and we prioritize building a firm that values the unique background and identity of every employee. Teamwork, integrity and excellence are the values that enable us to achieve at the highest levels.

Our Technology Department

The global technology department is responsible for designing, developing, delivering and supporting the technical solutions behind the products and services used by the Company business. The department is comprised of ten organizations, including Sales, Banking & Corporate‑Client Technology, Investment Products & Markets Technology, Client Reporting, Core Processing, Private and International Wealth Management Technology, Technology Integration Office, Enterprise Infrastructure & Production Management, Capital Markets Application & Data Services, Deployment Planning & Release Management, and the Chief Operating Office.

Wealth Management Core Platform Services

The Core Platform Services group provides horizontal services to all Wealth Management development teams. Our mission is to provide stable and scalable infrastructure and technology solutions for the entire Wealth Management organization, acting as a centralized interface through which teams can obtain infrastructure solutions and project support. We work closely with application owners throughout the SDLC process to ensure that established products and services are leveraged and that new requirements are fulfilled.

Position Description

• Framework CoE team located at Alpharetta or New York offices.
• CoE team is responsible to define and govern the data platforms.
• Role is primarily responsible for automating existing processes and bringing new ideas and innovation.
• Candidate will code, conduct code reviews, test framework as needed, and participate in application architecture and design.
• Self‑motivated team player, committed to delivering on time, able to work with or without minimal supervision.

Responsibilities

• Design and develop new automation framework for ETL processing.
• Support existing framework and become technical point of contact for all related teams.
• Enhance existing ETL automation framework per user requirements.
• Performance tuning of Spark and Snowflake ETL jobs.
• New technology POC and suitability analysis for cloud migration.
• Process optimization with automation and new utility development.
• Work in collaboration for any issues and new features.
• Support batch and application team queries.

Required Skills

• 7+ years of data engineering experience.
• Strong Unix Shell and Python scripting knowledge.
• Strong Spark expertise.
• Strong SQL knowledge.
• Hands‑on knowledge of HDFS, Hive, Impala, Spark.
• Logical reasoning capabilities.
• Working knowledge of GitHub, DevOps, CI/CD and enterprise code management tools.
• Strong collaboration and communication skills.
• Team‑player skills, excellent written and verbal communication.
• Positive environment of shared success.
• Ability to execute and prioritize tasks, resolve issues without aid from direct manager or project sponsor.
• Good to have working experience on Snowflake and any data integration tool (e.g., Informatica Cloud).

Primary Skills

• Python, Big Data & Apache Spark.

Desired Skills

• Snowflake, Azure, AWS or any cloud platform.
• IDMC or any ETL tool.