Remote Databricks Data Engineer: Build Scalable Pipelines

Eliassen Group; Phoenix (AZ); Remoto; USD 150.000 - 154.000; Tempo pieno

Vantaggi offerti da questo lavoro

Medical, dental, vision plans

401(k) with match

Sick time and paid time off

Descrizione del lavoro

Eliassen Group is seeking a Databricks Data Engineer for remote, full-time work on diverse energy-sector data projects. You will design, build, and maintain scalable data pipelines using AWS, Databricks, Spark, and PostgreSQL, collaborating with ML engineers, software developers, data scientists, and analysts.

Qualified candidates have 5+ years in data engineering, strong Python and SQL skills, and experience with cloud-based data pipelines.

Competenze

• Five or more years of experience in data engineering or a related role.
• Expert proficiency in Python, SQL, Databricks, and Spark.
• Experience building cloud-based data pipelines in AWS.
• Knowledge of data governance, security, privacy, and retention practices.
• Strong communication, collaboration, and problem-solving skills.
• Experience working in agile environments using tools such as Jira.
• Proficiency with Git, Docker, and CI tools such as Jenkins.
• Passion for data and continuous improvement.
• Preferred: Ability to work independently and proactively.
• Preferred: Experience optimizing ETL workflows in Databricks.
• Preferred: Knowledge of data lineage and data quality practices.
• Preferred: Experience supporting data science and machine learning solutions.
• Preferred: Prior experience in the energy or utility industry.

Mansioni

• Design, develop, and maintain data pipelines, infrastructure, and systems using AWS, Python, Databricks, Spark, and PostgreSQL.
• Collaborate with cross-functional teams to translate business requirements into technical solutions.
• Provide technical guidance and mentorship to junior data engineers.
• Develop and implement data engineering standards and best practices.
• Monitor and troubleshoot data pipelines to ensure data quality, integrity, and availability.
• Research industry trends and evaluate new data technologies.
• Build and maintain data lakes to support analytics and business intelligence.
• Manage the software development life cycle and DevOps activities.
• Partner with data scientists and analysts to deliver effective data solutions.
• Develop data validation and data analysis tools.

Conoscenze

Python

SQL

Databricks

Spark

AWS

Data pipelines

ETL

Jira

Git

Docker

CI tools

Data quality

Formazione

Bachelor’s degree in computer science or related field

Master’s degree in STEM (preferred)

Strumenti

Jira

Git

Docker

Jenkins

Descrizione del lavoro

Eliassen Group is seeking a Databricks Data Engineer for remote, full-time work on diverse energy-sector data projects. You will design, build, and maintain scalable data pipelines using AWS, Databricks, Spark, and PostgreSQL, collaborating with ML engineers, software developers, data scientists, and analysts.

Qualified candidates have 5+ years in data engineering, strong Python and SQL skills, and experience with cloud-based data pipelines.