Senior Data Engineer: Spark ETL, AWS & ML-Driven Analytics

JPS Tech Solutions; Atlanta (GA); On-site; USD 140,000 - 170,000; Full time

Job summary

JPS Tech Solutions, based in Atlanta, GA, seeks a Sr Data Engineer to build scalable ETL pipelines with Spark, ensuring data availability and quality for security analytics across client networks. Leverage AWS for scalable solutions and apply machine learning to automate data-driven operations.

Responsibilities include deploying CI/CD pipelines for data products, monitoring data quality, and collaborating across teams to socialize results and drive action within the organization.

Qualifications

• Ability to read, write, speak and understand English.
• Strong communication and presentation skills.
• Expertise in data engineering languages and Spark for BigData.
• Proficiency in building and managing ETL pipelines.
• Quantitative analysis including model interpretation and causality awareness.
• Experience with relational databases and NoSQL platforms.

Responsibilities

• Design, develop, and maintain scalable ETL pipelines to ensure data quality and availability.
• Implement monitoring and alerting solutions to ensure pipeline reliability and performance.
• Develop and manage deployment pipelines to enable CI/CD of data solutions.
• Implement data integration to support analytics and reporting needs.
• Execute the analytics lifecycle: data exploration, grooming, model prototyping, validation.

Skills

ETL pipelines

Spark

Python

SQL

Education

Bachelor's degree in CS/Math/Stats/Physics/OR related

Masters in CS or related field

Tools

Scala

Java

Job description

JPS Tech Solutions, based in Atlanta, GA, seeks a Sr Data Engineer to build scalable ETL pipelines with Spark, ensuring data availability and quality for security analytics across client networks. Leverage AWS for scalable solutions and apply machine learning to automate data-driven operations.

Responsibilities include deploying CI/CD pipelines for data products, monitoring data quality, and collaborating across teams to socialize results and drive action within the organization.