Sr. Databricks Data Engineer

Artech L.L.C.; Portland (OR); Hybrid; USD 110,000 - 160,000; Full time

Job summary

A leading technology firm seeks a highly skilled Databricks Data Engineer in Portland, OR. Candidates should have at least 10 years of experience, particularly in the retail sector, and strong skills in Python, SQL, and various ETL tools. This role involves designing data pipelines, optimizing databases, and implementing AWS solutions. If you're passionate about driving business success through innovative data strategies, we encourage you to apply.

Qualifications

• 10+ years of total experience in data engineering.
• 6+ years in Python programming for data processing.
• 4+ years of experience with Databricks and Snowflake.
• 4+ years in SQL development and performance tuning.
• Strong understanding of retail industry data needs.

Responsibilities

• Design and maintain scalable data pipelines using Databricks.
• Optimize SQL queries and manage RDBMS/NoSQL databases.
• Develop REST APIs to integrate data sources.
• Implement AWS cloud solutions for data management.
• Collaborate with teams to develop data solutions.

Skills

Python

SQL

ETL tools

Databricks

Snowflake

AWS

Hadoop

Tools

Apache Airflow

AWS Glue

Azure Data Factory

Talend

Alteryx

Job description

Location: Portland, OR (Local or Willing to relocate)

Job Description:

We are seeking a highly skilled Databricks Data Engineer with a minimum of 10 years of total experience, including strong expertise in the retail industry. The ideal candidate will be responsible for designing, developing, and optimizing data pipelines and architectures to support advanced analytics and business intelligence initiatives. This role requires proficiency in Python, SQL, cloud platforms, and ETL tools within a retail-focused data ecosystem.

Key Responsibilities:

• Design, develop, and maintain scalable data pipelines using Databricks and Snowflake.
• Work with Python libraries such as Pandas, NumPy, PySpark, PyOdbc, PyMsSQL, Requests, Boto3, SimpleSalesforce, and JSON for efficient data processing.
• Optimize and enhance SQL queries, stored procedures, triggers, and schema designs for RDBMS (MSSQL/MySQL) and NoSQL (DynamoDB/MongoDB/Redis) databases.
• Develop and manage REST APIs to integrate various data sources and applications.
• Implement AWS cloud solutions using AWS Data Exchange, Athena, Cloud Formation, Lambda, S3, AWS Console, IAM, STS, EC2, and EMR.
• Utilize ETL tools such as Apache Airflow, AWS Glue, Azure Data Factory, Talend, and Alteryx to orchestrate and automate data workflows.
• Work with Hadoop and Hive for big data processing and analysis.
• Collaborate with cross-functional teams to understand business needs and develop efficient data solutions that drive decision-making in the retail domain.
• Ensure data quality, governance, and security across all data assets and pipelines.

Required Qualifications:

• 10+ years of total experience in data engineering and data processing.
• 6+ years of hands-on experience in Python programming, specifically for data processing and analytics.
• 4+ years of experience working with Databricks and Snowflake.
• 4+ years of expertise in SQL development, performance tuning, and RDBMS/NoSQL databases.
• 4+ years of experience in designing and managing REST APIs.
• 2+ years of working experience in AWS data services.
• 2+ years of hands-on experience with ETL tools like Apache Airflow, AWS Glue, Azure Data Factory, Talend, or Alteryx.
• 1+ year experience with Hadoop and Hive.
• Strong understanding of retail industry data needs and best practices.
• Excellent problem-solving, analytical, and communication skills.

Preferred Qualifications:

• Experience with real-time data processing and streaming technologies.
• Familiarity with machine learning and AI-driven analytics.
• Certifications in Databricks, AWS, or Snowflake.

This is an exciting opportunity to work on cutting-edge data engineering solutions in a fast-paced retail environment. If you are passionate about leveraging data to drive business success and innovation, we encourage you to apply!

Seniority level

• Seniority level

  Mid-Senior level

Employment type

• Employment type

  Contract

Job function

• Job function

  Information Technology

• Industries

  Technology, Information and Media

Referrals increase your chances of interviewing at Artech L.L.C. by 2x

Get notified about new Data Engineer jobs in Portland, Oregon Metropolitan Area.

Portland, Oregon Metropolitan Area 2 months ago

Portland, OR $110,000.00-$160,000.00 3 hours ago

Portland, Oregon Metropolitan Area 2 weeks ago

Portland, OR $140,000.00-$200,000.00 1 week ago

Portland, OR $150,000.00-$170,000.00 1 week ago

Portland, Oregon Metropolitan Area 1 month ago

Portland, Oregon Metropolitan Area $120,000.00-$195,000.00 2 weeks ago

Portland, OR $85,307.41-$110,895.50 2 months ago

Software Engineer, Data Infrastructure & Acquisition

Portland, OR $140,000.00-$200,000.00 1 week ago

Solutions Engineer - Artificial Intelligence (AI)

Software Engineer II – Product Innovation

Portland, OR $140,000.00-$200,000.00 1 week ago

Software Engineer III - Price Optimization

Portland, OR $102,000.00-$127,000.00 1 day ago

Senior Software Engineer - Data Platform

Portland, OR $115,200.00-$199,700.00 2 weeks ago

Portland, OR $115,200.00-$199,700.00 1 week ago

United States $119,800.00-$258,000.00 6 days ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.