Senior Data Engineer

eTeam; Beaverton (OR); Hybrid; USD 137,000 - 158,000; Full time

Job summary

eTeam in Beaverton, OR is seeking an experienced Data Engineer to design and implement data products in collaboration with product owners, data analysts, and business partners using Agile/Scrum.

The role focuses on high-volume data processing with Spark, Databricks, Python, and SQL. 8–10 years in data-related roles and a BS in CS are required; AWS/Databricks certifications are a plus. Hybrid work model with Mon–Thu onsite and Fri remote.

Qualifications

• Bachelor’s degree in Computer Science or related technical discipline.
• 8-10 years in data-related roles with strong software development experience.
• Proficiency in Python or Scala; experience with Spark and SQL.
• Experience with data warehousing concepts and Snowflake; knowledge of Hadoop ecosystem.

Responsibilities

• Design and implement data products and features with Agile/Scrum teams.
• Contribute to architecture and patterns for processing large data volumes.
• Build distributed data pipelines using big data tools and languages.
• Develop utilities and libraries to improve data flow and performance.
• Collaborate with engineers to ensure quality and CI/CD best practices.
• Create unit tests and participate in integration testing.

Skills

Databricks

Spark

Python

Snowflake

SQL

AI tools

Education

Bachelor’s degree in Computer Science

Tools

Hadoop

Hive

Airflow

GitHub/Bitbucket

Terraform/CloudFormation

Jenkins/CircleCI

Job description

Pay Range: $(66.00 - 76.00)/hr on W2 all-inclusive without benefits

Hybrid: Mon – Thurs onsite, Fri remote

Job Role:

• On-site role at PHK; typical work hours, 4/1 hybrid schedule
• Degree will be required for this role
• AWS, Databricks certifications nice to have, not required
• 8-10 years’ experience minimum preferred in a data-related role

Top must-have skills include the following:

• Databricks
• Spark
• Python
• Snowflake
• SQL
• Experience with AI tools and initiatives
• Skills that would make a candidate stand out:
• Experience with/knowledge of finance domain
• Experience with data analytics and BI tools
• supporting gaps in the team, building capabilities, tools, and new data products, and improving profitability and margins
• Urgent need - HM will prioritize those who can start ASAP
• 2 rounds of interviews anticipated

Role Responsibilities:

• Contribute to Design and implement data products and features in collaboration with product owners, data analysts, and business partners using Agile / Scrum methodology
• Contribute to overall architecture, frameworks and patterns for processing and storing large data volumes
• Contribute to evaluation of new technologies/tools/frameworks centered around high-volume data processing
• Translate product backlog items into logical units of work in engineering
• Implement distributed data processing pipelines using tools and languages prevalent in the big data ecosystem
• Build utilities, user defined functions, libraries, and frameworks to better enable data flow patterns
• Work with engineering leads and other teams to ensure quality solutions are implemented, and engineering best practices are defined and followed
• Build and incorporate automated unit tests and participate in integration testing efforts
• Utilize and advance continuous integration and deployment frameworks
• Troubleshoot data issues and perform root cause analysis
• Work across teams to resolve operational & performance issues

The following qualifications and technical skills will position you well for this role:

• Bachelor’s degree in Computer Science, or related technical discipline
• 10+ years of experience in large-scale software development, 5+ years of big data experience
• Programming experience, Python or Scala preferred.
• Experience working with Hadoop and related processing frameworks such as Spark, Hive, etc.
• Experience with messaging/streaming/complex event processing tooling and frameworks
• Experience with data warehousing concepts, SQL and SQL Analytical functions
• Experience with workflow orchestration tools like Apache Airflow
• Experience with source code control tools like Github or Bitbucket
• Ability to communicate effectively, both verbally and written, with team members
• Interest in and ability to quickly pick up new languages, technologies, and frameworks
• Experience in Agile/Scrum application development

The following skills and experience are also relevant to our overall environment, and nice to have:

• Experience with Java
• Experience working in a public cloud environment, particularly AWS, databricks
• Experience with cloud warehouse tools like Snowflake
• Experience working with NoSQL data stores such as HBase, DynamoDB, etc.
• Experience building RESTful API’s to enable data consumption
• Experience with build tools such as Terraform or CloudFormation and automation tools such as Jenkins or Circle CI
• Experience with practices like Continuous Development, Continuous Integration and Automated Testing

These are the characteristics that we strive for in our own work. We would love to hear from candidates who embody the same:

• Desire to work collaboratively with your teammates to come up with the best solution to a problem
• Demonstrated experience and ability to deliver results on multiple projects in a fast-paced, agile environment
• Excellent problem-solving and interpersonal communication skills
• Strong desire to learn and share knowledge with others