Databricks Data Engineer and Scientist

Jobgether; Oregon; Hybrid; USD 120,000 - 180,000; Full time

Benefits offered by this job

Healthcare

Paid time off

Retirement plans

Job summary

Jobgether is seeking a Databricks Data Engineer/Scientist for a fully remote US-based role. You will design and optimize enterprise-scale data pipelines in a Databricks/AWS Lakehouse, transforming complex data into analytics-ready assets for business decisions and advanced analytics.

Collaborate across data engineering, modeling, and AI-assisted development to modernize legacy systems while ensuring accuracy, performance, and scalability.

Qualifications

• Bachelor's degree in CS/engineering/math or info systems.
• 8+ years ETL and data pipeline experience.
• 3+ years Databricks notebooks, clusters, orchestration (Delta Lake, Unity Catalog).
• 3+ years Python and PySpark for ETL scripting and transformations.
• Experience with AI-assisted coding tools (Claude Code or Cursor).
• Design data models in Bronze/Silver/Gold medallion architecture.
• Hands-on AWS (Glue, S3, RDS).
• Knowledge of performance optimization (partitioning, bucketing, caching).
• Eligibility for Public Trust/Suitability; US citizenship may be required.

Responsibilities

• Design, build, deploy, and maintain end-to-end data pipelines in a Databricks/AWS Lakehouse.
• Develop and manage reliable ETL processes with logging and error handling.
• Maintain data models in Bronze/Silver/Gold medallion architecture.
• Analyze legacy SQL, decompose pipelines, and validate results for accuracy.
• Optimize pipeline performance and query execution using partitioning and caching.
• Leverage AI-assisted coding to accelerate data engineering and modeling tasks.
• Ensure data security and cross-functional collaboration for quality analytics.

Skills

Databricks

Python

PySpark

AWS

Delta Lake

Unity Catalog

AI-assisted coding tools

ETL design and optimization

Data modeling

Education

Bachelor's degree in Computer Science, Computer Engineering, Engineering, Mathematics, Physics, or Information Systems and Technology

Tools

Databricks

Delta Lake

Unity Catalog

AWS Glue

Amazon S3

Amazon RDS

Python

PySpark

Job description

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Databricks Data Engineer and Scientist based in United States. In this role, you will design, develop, and optimize enterprise-scale data pipelines within a Databricks and AWS-based Lakehouse environment. You will transform complex data into reliable, analytics-ready assets that support informed business decisions and advanced analytical initiatives. Working across data engineering, data modeling, and AI-assisted development, you will modernize legacy systems while ensuring data accuracy, performance, and scalability. You will contribute to a Bronze, Silver, and Gold medallion architecture and build robust ETL processes with comprehensive logging and error handling. The role combines hands-on technical delivery with collaboration across cross-functional teams in support of a critical federal data environment. This is a full-time, remote opportunity for an experienced professional who enjoys solving complex data challenges and improving enterprise data infrastructure.

Accountabilities

• Design, build, deploy, and maintain end-to-end data pipelines for an enterprise Lakehouse using Databricks and AWS.
• Develop and manage reliable ETL processes with comprehensive logging, error handling, troubleshooting, and issue resolution.
• Design and maintain data models within a Bronze, Silver, and Gold medallion architecture, selecting appropriate dimensional or relational modeling approaches.
• Analyze and decompose complex legacy SQL, rebuild pipelines and data models, and validate results for accuracy and performance.
• Optimize data pipeline performance, scalability, and query execution using partitioning, bucketing, caching, and other relevant techniques.
• Use AI-assisted coding tools to accelerate pipeline analysis, data engineering, debugging, and data modeling tasks.
• Support data security and compliance requirements while collaborating with cross-functional teams to deliver high-quality, analytics-ready data.

Requirements

• Bachelor's degree in Computer Science, Computer Engineering, Engineering, Mathematics, Physics, or Information Systems and Technology.
• At least 8 years of experience designing, developing, deploying, and managing ETL processes and data pipelines.
• At least 3 years of experience building, debugging, and optimizing data pipelines using Databricks notebooks, clusters, and job orchestration tools, including Delta Lake and Unity Catalog.
• At least 3 years of experience developing Python and PySpark code for ETL scripts and custom data transformations.
• Practical experience using AI-assisted coding tools, such as Claude Code or Cursor, to support data engineering and data modeling activities.
• Experience designing data models using dimensional and relational approaches within a Bronze, Silver, and Gold medallion architecture.
• Hands‑on experience with AWS services, including AWS Glue, Amazon S3, and Amazon RDS.
• Knowledge of ETL and query performance optimization techniques, including partitioning, bucketing, and caching.
• Demonstrated ability to analyze complex legacy SQL, reconstruct data pipelines and models, and compare outputs to validate accuracy and performance.
• Ability to obtain and maintain a Public Trust or other required Suitability/Fitness determination, depending on client requirements.
• U.S. citizenship is required to support the federal contract.

Benefits

• Work arrangement: Remote position based in the United States.
• Employment type: Full-time.
• Technical environment: Opportunity to work with Databricks, AWS, Lakehouse architecture, Python, PySpark, and AI-assisted development tools.
• Compensation: Salary, bonuses, and other financial benefits were not specified in the provided job description and should be confirmed during the hiring process.
• Additional benefits: Healthcare, paid time off, retirement plans, and other employee perks were not detailed in the provided description and should be confirmed with the hiring team.

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.