Data Engineer – Databricks

BuzzClan LLC; Beaverton (OR); On-site; USD 120,000 - 170,000; Full time

Job summary

BuzzClan LLC seeks a Data Engineer with strong Databricks expertise to design, build, and maintain scalable enterprise data pipelines and data engineering solutions. The role demands hands-on work with Databricks, PySpark, Spark SQL, Delta Lake, Python, and SQL, plus cloud experience across Azure, AWS, or GCP, with Azure preferred.

The candidate will implement data modeling, data warehousing, and performance optimization while collaborating with business stakeholders and cross-functional teams.

Qualifications

• 6+ years of overall IT experience.
• 5+ years of Data Engineering experience.
• Strong hands-on with Databricks.
• Proficient in PySpark, Spark SQL, and Delta Lake.
• Strong programming in Python and SQL.
• Experience with cloud platforms (Azure/AWS/GCP), preferably Azure.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines and enterprise data solutions.
• Develop and optimize data processing using Databricks, PySpark, Spark SQL, and Delta Lake.
• Build reliable data pipelines using Python and SQL with performance in mind.
• Implement data integration and transformation across various enterprise sources.
• Work with cloud platforms (Azure, AWS, GCP), with Azure preferred.
• Develop and manage data workflows using orchestration tools like Azure Data Factory and/or Airflow.
• Apply data modeling and warehousing best practices; optimize performance.
• Implement data quality, scalability, reliability, and performance improvements across platforms.
• Collaborate with stakeholders, data architects, and developers to deliver enterprise data solutions.

Job description

Job Overview

We are seeking an experiencedData Engineer with strong Databricks expertiseto design, develop, and maintain scalable enterprise data pipelines and data engineering solutions. The ideal candidate will have hands‑on experience withDatabricks, PySpark, Spark SQL, Delta Lake, Python, and SQL, along with experience in cloud platforms and modern data engineering practices.

Candidates must be local to or able to work onsite in Beaverton, Oregon 3 days per week.

Key Responsibilities

• Design, develop, and maintain scalableETL/ELT data pipelinesand enterprise data solutions.
• Develop and optimize data processing solutions usingDatabricks, PySpark, Spark SQL, and Delta Lake.
• Build reliable and high‑performance data pipelines usingPython and SQL.
• Implement data integration and transformation processes across various enterprise data sources.
• Work with cloud platforms such asAzure, AWS, or GCP, with preference forAzure.
• Develop and manage data workflows using orchestration tools such asAzure Data Factory (ADF) and/or Apache Airflow.
• Apply data modeling, data warehousing, and performance optimization best practices.
• Implement data quality, scalability, reliability, and performance improvements across data platforms.
• UseGit, CI/CD, and DevOps practicesto support automated and reliable data deployments.
• Collaborate with business stakeholders, data architects, developers, and other cross‑functional teams to deliver enterprise data solutions.
• Troubleshoot complex data pipeline and processing issues and provide effective technical solutions.
• Contribute to modern Databricks capabilities such asDelta Live Tables (DLT), Unity Catalog, and Structured Streamingwhere applicable.

Qualifications

Required Skills

• 6+ years of overall IT experience, including5+ years of Data Engineering experience.
• Strong hands‑on experience withDatabricks.
• Strong expertise inPySpark, Spark SQL, and Delta Lake.
• Strong programming skills inPython and SQL.
• Experience withAzure, AWS, or GCP, preferably Azure.
• Experience withAzure Data Factory (ADF) and/or Airflow.
• Strong understanding ofdata modeling and data warehousing.
• Experience withdata pipeline and query performance optimization.
• Hands‑on experience withGit, CI/CD, and DevOps practices.
• Strong analytical, problem‑solving, collaboration, and communication skills.

Preferred Skills

• Unity Catalog
• Structured Streaming
• Experience working with large-scale enterprise data platforms.

Additional Information

All your information will be kept confidential according to EEO guidelines.