Data Engineer II - Build Scalable, Real-Time Data Pipelines

SpyCloud, Inc.; Austin (TX); Hybrid; USD 104,000 - 135,000; Full time

Benefits offered by this job

401(k) with Employer Contribution

Health, Vision, and Dental Insurance

Health Savings Account (HSA) available

Employer Paid Life and Disability

Generous PTO and 16 paid holidays

Hybrid/Remote-friendly options

Job summary

SpyCloud, Inc. is hiring a Data Engineer II to join the Engineering team in Austin. You will evolve our data platform using Databricks and Spark, build scalable pipelines, and serve data stores such as RDS, Elasticsearch, and DynamoDB. This is a hands-on IC role focusing on scalable data initiatives and reliability.

You will collaborate with product and engineering teams to design robust solutions, maintain data quality, and drive observability across production systems.

Qualifications

• 5+ years of experience in data engineering or related field.
• Experience building and operating large-scale data architectures in AWS.
• Experience with ETL pipelines and data processing for ML/LLM workflows.
• Hands-on Python development for data-intensive systems.
• Experience with DynamoDB and Elasticsearch/OpenSearch.

Responsibilities

• Design, build, and improve scalable data systems for performance and reliability.
• Use Databricks and Spark to prepare and serve high-quality data.
• Build, optimize, and maintain real-time and batch data pipelines on AWS.
• Identify and implement cost/performance improvements for data workflows.
• Provide on-call support for data pipelines and production issues.
• Collaborate with Product, Data Science, and Engineering teams on requirements.
• Evaluate new data processing tools and observability solutions.
• Contribute to architectural decisions and code reviews.

Skills

Python

Spark

Databricks

ETL pipelines

AWS

DynamoDB

Elasticsearch/OpenSearch

Terraform

Education

Bachelor's degree in Computer Science or related field

Tools

Databricks ecosystem

AWS services

Job description

SpyCloud, Inc. is hiring a Data Engineer II to join the Engineering team in Austin. You will evolve our data platform using Databricks and Spark, build scalable pipelines, and serve data stores such as RDS, Elasticsearch, and DynamoDB. This is a hands-on IC role focusing on scalable data initiatives and reliability.

You will collaborate with product and engineering teams to design robust solutions, maintain data quality, and drive observability across production systems.