Senior Data Engineer

TechDigital Group; Atlanta (GA); On-site; USD 100,000 - 130,000; Full time

Job summary

A leading technology company in Atlanta is seeking a Senior Data Engineer who will design and develop cutting-edge Big Data solutions. This role requires hands-on experience with technologies like Hadoop and Spark, along with programming expertise in Java, Scala, or Python. The ideal candidate will work collaboratively to optimize data pipelines and contribute to the company's innovative data strategies.

Qualifications

• Hands-on experience with Big Data and analytics technologies.
• Programming expertise in Java, Scala, or Python.
• Real-time and batch data processing skills.
• Proficiency in data modeling and schema design.
• Exposure to cloud technologies.

Responsibilities

• Design & develop batch and streaming solutions.
• Assess technologies for ingestion and storage.
• Develop software utilizing open-source technologies.
• Identify functional and system requirements.

Skills

Hadoop

Spark

Java

Scala

Python

Data modeling

ETL workflows

Agile/Scrum

Education

BS/BA degree in Computer Science

Tools

Databricks

Cloudera

Job description

Job Description: Senior Data Engineer who is passionate about data and analytics, highly motivated and can take technical ownership of data solutions with an emphasis on engineering efficiency and on‑time delivery.

The ideal candidate is exposed to the fast‑paced world of Big Data technology and has experience in building data solutions using new and emerging technologies while maintaining stability of the platform. The person will get a chance to work with diverse datasets and be on the cutting edge of transforming the way Client captures, processes, stores and visualizes data.

Role

In this role, you will:

• Design & Develop batch and streaming solutions using various Big Data technologies
• Work closely with architect to design secure and scalable
• Be responsible for assessing technologies and approaches for ingestion, transformation and storage
• Develop software utilizing open‑source technologies to interface with distributed and relational data sources
• Work closely with team members from across Client to identify functional and system requirements

All About You

The ideal candidate for this position should:

• Hands‑on experience with Big Data and analytics technologies including Hadoop, Spark, and related frameworks.
• Programming expertise in languages such as Java, Scala, or Python, with an emphasis on developing data processing and transformation logic.
• Real‑time and batch data processing skills with experience in building robust data pipelines for real‑time or near‑real‑time ingestion.
• Proficiency in data modeling and schema design for relational and non‑relational databases, supporting high data volume, and optimized query performance.
• Exposure to cloud technologies with hands‑on experience in Databricks and Cloudera, including knowledge of cloud architecture, data security, and scalability best practices.
• Experience in optimizing and managing ETL workflows using scheduling, monitoring, and debugging tools.
• Strong problem‑solving abilities with a mindset for automation and efficiency improvements.
• Excellent communication and collaboration skills to work effectively across technical and business teams, facilitating knowledge sharing and alignment on data engineering best practices.
• Proven track record of delivering software projects and willingness to roll up sleeves to get the job done.
• Have excellent oral and written communication.
• Be familiar with Agile/Scrum methodologies.
• Have the ability to manage multiple priorities simultaneously.
• Have a BS/BA degree in Computer Science, Information Systems or related field.