Senior Data Engineer: Build Scalable Pipelines & Platforms

Socure; United States; Remote; USD 120,000 - 150,000; Full time

Job summary

A leading identity verification firm is seeking a Senior Data Engineer to join their Data Automation team. You will be responsible for designing and building scalable data platforms and pipelines crucial to their identity verification products. The ideal candidate has over 5 years of hands-on expertise in data engineering, along with strong programming skills in Python or Scala and experience with AWS services. This position offers a chance to work in a fast-paced environment with a team devoted to solving complex problems.

Qualifications

• 5+ years of hands-on data engineering experience.
• Strong programming skills in Python or Scala.
• Deep experience with distributed data processing frameworks.

Responsibilities

• Design and build batch and streaming data pipelines.
• Own end-to-end delivery of complex data initiatives.
• Develop and evolve the data platform for large-scale processing.
• Automate data operations to improve consistency.
• Optimize cost, performance, and reliability of data workloads.

Skills

Data engineering

Programming in Python or Scala

SQL for data analytics

Apache Spark performance tuning

AWS services (EMR, Lambda, S3)

Data modeling

Software engineering practices

Collaboration with stakeholders

Tools

Apache Spark

Airflow

Kafka

AWS EMR

CI/CD tools

Job description

A leading identity verification firm is seeking a Senior Data Engineer to join their Data Automation team. You will be responsible for designing and building scalable data platforms and pipelines crucial to their identity verification products. The ideal candidate has over 5 years of hands-on expertise in data engineering, along with strong programming skills in Python or Scala and experience with AWS services. This position offers a chance to work in a fast-paced environment with a team devoted to solving complex problems.