Senior AWS Data Engineer: PySpark & Cloud Pipelines

System One, Inc.; Knoxville (TN); Hybrid; USD 140,000 - 190,000; Full time

Job summary

System One, Inc. seeks a Senior Data Engineer to modernize vulnerability management data pipelines in a hybrid role based in Knoxville, TN.

You will leverage PySpark, Python, and AWS to build scalable data processing solutions that collect, contextualize, and prioritize data for enterprise use. The ideal candidate has 10+ years in data engineering, strong Spark/PySpark expertise, and deep AWS experience across EMR, Redshift, S3, and Glue.

Qualifications

• 10+ years of experience building and optimizing large-scale data pipelines.
• Strong PySpark, Python, and AWS data processing expertise.
• Experience with big data tech (Hadoop/EMR, Spark, Hive).
• Advanced SQL with complex queries and transformations.
• Experience with REST APIs and external data integrations.
• Experience with CI/CD and infrastructure automation (GitLab, Terraform).
• Bachelor's degree in CS, IS, or related field.

Responsibilities

• Develop scalable Python applications and data processing solutions.
• Integrate APIs and external data sources.
• Write and optimize complex SQL queries.
• Process large datasets using Spark and AWS services.
• Troubleshoot performance in applications and data pipelines.
• Contribute to CI/CD automation and deployment pipelines.
• Collaborate with technical and business teams across the organization.

Skills

Python

PySpark

Spark

AWS

SQL

REST APIs

CI/CD

Terraform

GitLab

Hadoop/EMR

Hive

Data pipelines

Big data

Education

Bachelor's degree in Computer Science/Information Systems or related field

Tools

EMR

Lambda

EventBridge

Step Functions

Redshift

S3

Glue

Job description

System One, Inc. seeks a Senior Data Engineer to modernize vulnerability management data pipelines in a hybrid role based in Knoxville, TN.

You will leverage PySpark, Python, and AWS to build scalable data processing solutions that collect, contextualize, and prioritize data for enterprise use. The ideal candidate has 10+ years in data engineering, strong Spark/PySpark expertise, and deep AWS experience across EMR, Redshift, S3, and Glue.