Databricks Data Engineer

Eliassen Group; United States; Remote; USD 150,000 - 154,000; Full time

Benefits offered by this job

Comprehensive benefits package

Remote work option

Job summary

Eliassen Group is seeking a Databricks Data Engineer to design and maintain scalable data pipelines and infrastructure for consulting services and SaaS solutions. You will work with ML engineers, software engineers, data scientists, and analysts to deliver data products and support internal data practices.

The role is full-time and remote in the United States, offering a competitive salary and comprehensive benefits. Must be willing to work on a W2 basis.

Qualifications

• Five or more years of experience in data engineering or related role.
• Expert proficiency in Python, SQL, Databricks, and Spark.
• Experience building cloud-based data pipelines in AWS.
• Knowledge of data governance, security, privacy, and retention practices.
• Strong communication, collaboration, and problem-solving skills.
• Experience working in agile environments using Jira.
• Proficiency with Git, Docker, and CI tools such as Jenkins.
• Passion for data and continuous improvement.
• Preferred: ability to work independently and proactively.
• Preferred: experience optimizing ETL workflows in Databricks.
• Preferred: knowledge of data lineage and data quality practices.
• Preferred: experience supporting data science and machine learning solutions.
• Preferred: energy or utility industry experience.

Responsibilities

• Design, develop, and maintain data pipelines, infrastructure, and systems using AWS, Python, Databricks, Spark, and PostgreSQL.
• Collaborate with cross-functional teams to translate business requirements into technical solutions.
• Provide technical guidance and mentorship to junior data engineers.
• Develop and implement data engineering standards and best practices.
• Monitor and troubleshoot data pipelines to ensure data quality, integrity, and availability.
• Research industry trends and evaluate new data technologies.
• Build and maintain data lakes to support analytics and business intelligence.
• Manage the software development life cycle and DevOps activities.
• Partner with data scientists and analysts to deliver effective data solutions.
• Develop data validation and data analysis tools.

Skills

Python

SQL

Databricks

Spark

Education

Bachelor's degree in computer science, information technology, or related field

Master's degree preferred

Tools

AWS

PostgreSQL

Git

Docker

Jenkins

Jira

Job description

Description

Remote

Our client seeks a Databricks Data Engineer to design, build, and maintain scalable, reliable, and secure data pipelines, infrastructure, and systems that support consulting services and SaaS solutions. You will partner with machine learning engineers, software engineers, data scientists, and analysts to deliver data products for clients and to support internal data practices.

This is a full-time, permanent opportunity, offering a competitive salary and comprehensive benefits package. Qualified applicants must be willing and able to work on a w2 basis.

Salary: $150,000 - $154,000/ yr. w2

Responsibilities

• Design, develop, and maintain data pipelines, infrastructure, and systems using AWS, Python, Databricks, Spark, and PostgreSQL.
• Collaborate with cross-functional teams to translate business requirements into technical solutions.
• Provide technical guidance and mentorship to junior data engineers.
• Develop and implement data engineering standards and best practices.
• Monitor and troubleshoot data pipelines to ensure data quality, integrity, and availability.
• Research industry trends and evaluate new data technologies.
• Build and maintain data lakes to support analytics and business intelligence.
• Manage the software development life cycle and DevOps activities.
• Partner with data scientists and analysts to deliver effective data solutions.
• Develop data validation and data analysis tools.

Experience Requirements

• Five or more years of experience in data engineering or a related role.
• Expert proficiency in Python, SQL, Databricks, and Spark.
• Experience building cloud-based data pipelines in AWS.
• Knowledge of data governance, security, privacy, and retention practices.
• Strong communication, collaboration, and problem-solving skills.
• Experience working in agile environments using tools such as Jira.
• Proficiency with Git, Docker, and CI tools such as Jenkins.
• Passion for data and continuous improvement.
• Preferred: Ability to work independently and proactively.
• Preferred: Experience optimizing ETL workflows in Databricks.
• Preferred: Knowledge of data lineage and data quality practices.
• Preferred: Experience supporting data science and machine learning solutions.
• Preferred: Prior experience in the energy or utility industry.

Education Requirements

• Bachelor’s degree in computer science, information technology, or a related field.
• Preferred: Master’s degree in a relevant STEM field.