Remote Databricks Data Engineer - Build Scalable Pipelines

Eliassen Group; Frankfort (KY); Remote; USD 150,000 - 154,000; Full time

Benefits offered by this job

Medical plans

Dental insurance

Vision insurance

Pre-tax accounts

401(k) with match

Sick time

Job summary

Eliassen Group is seeking a Databricks Data Engineer for a remote, full-time role. You will design, build, and maintain scalable data pipelines and data platforms in AWS and Databricks, collaborating with ML engineers, software developers, and data scientists.

The position offers a competitive salary and comprehensive benefits, with full-time W2 employment. Candidates must be willing to work from the United States on a remote basis.

Qualifications

• Five or more years of data engineering experience.
• Expert proficiency in Python, SQL, Databricks, and Spark.
• Experience building cloud data pipelines in AWS.
• Knowledge of data governance, security, privacy, and retention practices.
• Strong communication, collaboration, and problem-solving skills.
• Experience working in agile environments using tools such as Jira.
• Proficiency with Git, Docker, and CI tools such as Jenkins.
• Passion for data and continuous improvement.
• Preferred: Ability to work independently and proactively.
• Preferred: Experience optimizing ETL workflows in Databricks.
• Preferred: Knowledge of data lineage and data quality practices.
• Preferred: Experience supporting data science and machine learning solutions.
• Preferred: Prior experience in the energy or utility industry.

Responsibilities

• Design, develop, and maintain data pipelines, infrastructure, and systems using AWS, Python, Databricks, Spark, and PostgreSQL.
• Collaborate with cross-functional teams to translate business requirements into technical solutions.
• Provide technical guidance and mentorship to junior data engineers.
• Develop and implement data engineering standards and best practices.
• Monitor and troubleshoot data pipelines to ensure data quality, integrity, and availability.
• Research industry trends and evaluate new data technologies.
• Build and maintain data lakes to support analytics and business intelligence.
• Manage the software development life cycle and DevOps activities.
• Partner with data scientists and analysts to deliver effective data solutions.
• Develop data validation and data analysis tools.

Skills

Python

SQL

Databricks

Spark

AWS

Jira

Git

Docker

CI/CD

PostgreSQL

Education

Bachelor's degree in computer science, information technology, or a related field

Master’s degree in a relevant STEM field

Tools

PostgreSQL

Job description

Eliassen Group is seeking a Databricks Data Engineer for a remote, full-time role. You will design, build, and maintain scalable data pipelines and data platforms in AWS and Databricks, collaborating with ML engineers, software developers, and data scientists.

The position offers a competitive salary and comprehensive benefits, with full-time W2 employment. Candidates must be willing to work from the United States on a remote basis.