Senior Data Engineer Backend (Vancouver)

Flagler Health; Vancouver (WA); On-site; USD 80,000 - 120,000; Full time

Job summary

A healthcare company located in Vancouver, Washington, is seeking a data engineer with strong expertise in Databricks and MongoDB. The role involves developing and optimizing data pipelines, debugging Spark applications, and writing efficient Python code. Proven experience in large-scale data processing and strong problem-solving skills are essential. Candidates should also be familiar with best practices in data architecture. This position offers opportunities for personal and professional growth within a dynamic team environment.

Qualifications

• Proven experience working with Databricks and Spark compute.
• Proficient in Python, including object-oriented programming and API development.
• Familiarity with NoSQL (MongoDB preferred), including querying, data modeling, and optimization.
• Strong problem-solving skills and ability to debug and optimize data processing tasks.
• Experience with large-scale data processing and distributed systems.

Responsibilities

• Develop, manage, and optimize data pipelines on the Databricks platform.
• Debug and troubleshoot Spark applications to ensure reliability and performance.
• Implement best practices for Spark compute and optimize workloads.
• Design and build APIs to support data integration and application needs.
• Develop scripts and tools to automate data processing and workflows.

Skills

Databricks Platform Expertise

Python Development

MongoDB Management

Problem-solving skills

Experience with distributed systems

Tools

Databricks

MongoDB

AWS

Azure

Git

Job description

Key Responsibilities

Databricks Platform Expertise

• Develop, manage, and optimize data pipelines on the Databricks platform.
• Debug and troubleshoot Spark applications to ensure reliability and performance.
• Implement best practices for Spark compute and optimize workloads.
• Python Development: Write clean, efficient, and reusable Python code using object-oriented programming principles.
• Design and build APIs to support data integration and application needs.
• Develop scripts and tools to automate data processing and workflows.

MongoDB Management

• Integrate, query, and manage data within MongoDB.
• Ensure efficient storage and retrieval processes tailored to application requirements.
• Optimize MongoDB performance for large-scale data handling.

Collaboration and Problem Solving

• Work closely with data scientists, analysts, and other stakeholders to understand data needs and deliver solutions.
• Proactively identify and address technical challenges related to data processing and system design.

Required Qualifications

• Proven experience working with Databricks and Spark compute.
• Proficient in Python, including object-oriented programming and API development.
• Familiarity with NoSQL (MongoDB preferred), including querying, data modeling, and optimization.
• Strong problem-solving skills and ability to debug and optimize data processing tasks.
• Experience with large-scale data processing and distributed systems.

Preferred Qualifications

• Strong understanding of data architecture, ETL processes, and data warehousing concepts.
• Knowledge of other big data technologies like Delta Lake, Hadoop, or Kafka.
• Experience with cloud platforms (e.g., AWS, Azure, or GCP).
• Familiarity with CI/CD pipelines and version control systems like Git.

Our values

This is what you can expect of your teammates at Flagler:

• Persistence + ownership of outcomes: We wear many hats and aren’t afraid to run through walls to solve hard problems.
• Personal + professional growth: We push ourselves to learn new things and embrace challenges, even if it means that we sometimes fail.
• Don’t take things personally: We value and react quickly to constructive feedback.
• Speed is our ally: In the fast-paced world of startups, we understand the value of moving swiftly. We thrive on the adrenaline of working rapidly.
• Be Right: We are highly detailed oriented and try to be right, a lot.

Note: This description retains the original job information and context while adhering to formatting guidelines.