Senior Data Engineer - Big Data Pipelines (Hybrid)

Apex Systems; Beaverton (OR); Hybrid; USD 120,000 - 144,000; Full time

Benefits offered by this job

Health insurance

401K plan

HSA

Job summary

Everforth Apex is seeking a Senior Data Engineer for an 8-month contract in Portland, OR, with a hybrid work arrangement. You will design and implement data products, contribute to architecture, and build distributed pipelines using Python/Scala, Hadoop, Spark, and Airflow.

Strong SQL, NoSQL, and cloud tooling experience are required. You will collaborate with product owners and analysts, implement testing, and drive CI/CD adoption while solving complex data-processing challenges in a fast-paced

Qualifications

• Bachelor’s degree in Computer Science or related field.
• 10+ years of experience in large-scale software development, 5+ years of big data experience.
• Programming experience in Python or Scala.
• Experience with Hadoop and Spark/Hive ecosystems.
• Experience with messaging/streaming tooling and NoSQL stores.
• Experience with SQL and analytical functions.
• Experience with Airflow for workflow orchestration.
• Experience with Git or Bitbucket for source control.
• Ability to communicate effectively in a team and across geographies.
• Familiarity with Agile/Scrum practices.

Responsibilities

• Design and implement data products and features with cross-functional teams.
• Contribute to architecture and patterns for processing large data volumes.
• Evaluate new technologies for high-volume data processing.
• Translate backlog items into engineering tasks.
• Build distributed data processing pipelines using the big data stack.
• Create utilities and libraries to enable data flows.
• Ensure quality and engineering best practices.
• Develop automated tests and participate in CI/CD.
• Troubleshoot data issues and perform root cause analysis.
• Collaborate across teams to resolve ops and performance issues.

Skills

Python

Scala

Hadoop

Spark

Hive

Airflow

Git

AWS

Databricks

NoSQL

RESTful APIs

Terraform/CloudFormation

CI/CD

Education

Bachelor's degree in Computer Science

Tools

Airflow

Git

Terraform

CloudFormation

Jenkins

CircleCI

Databricks

Job description

Everforth Apex is seeking a Senior Data Engineer for an 8-month contract in Portland, OR, with a hybrid work arrangement. You will design and implement data products, contribute to architecture, and build distributed pipelines using Python/Scala, Hadoop, Spark, and Airflow.

Strong SQL, NoSQL, and cloud tooling experience are required. You will collaborate with product owners and analysts, implement testing, and drive CI/CD adoption while solving complex data-processing challenges in a fast-paced