Azure Databricks Engineer

Robotic Technologies Automation; Houston (TX); Vor Ort; USD 120.000 - 180.000; Vollzeit

Zusammenfassung

ROBOTIC PROCESS AUTOMATION LLC seeks a Specialist with 7 years of experience in designing and developing scalable data solutions using Azure Databricks. You will lead the design and optimization of enterprise-grade data pipelines, writing PySpark and Python code for ingestion, transformation, and analytics.

You will troubleshoot pipelines, collaborate with data architects and analysts, document processes, and mentor junior teammates to uphold best practices in data engineering.

Qualifikationen

• 7 years of experience designing and developing scalable data solutions using Azure Databricks.
• Experience building enterprise-level data pipelines and data platforms with strong focus on quality and reliability.

Aufgaben

• Lead design, development and enhancement of complex datapipelines using Azure Databricks.
• Write clean, maintainable PySpark and Python code for data ingestion, transformation and analytics.
• Troubleshoot and resolve issues in datapipelines to ensure minimal downtime and high availability.
• Collaborate with data architects, analysts and stakeholders to deliver scalable data solutions.
• Mentor and guide junior team members on Azure Databricks best practices in data engineering.
• Document data processes, architectures and workflows for knowledge sharing and compliance.

Kenntnisse

Azure Databricks

PySpark

Python

ETL

Tools

SQL

Airflow

Jobbeschreibung

Seeking a Specialist with 7 years of experience in designing and developing scalabledatasolutions using AzureDatabricks

Design develop and optimize scalabledatapipelines leveraging AzureDatabricksto support enterprise leveldataprocessing andanalyticsImplement and maintain ETL workflows ensuring high performancedataquality and reliability Integrate AzureDatabrickswith various Azuredataservices to build robust and scalabledataplatforms Monitor and optimizedataworkflows for scalability cost effectiveness and operational efficiency Collaborate with cross functional teams to aligndatasolutions with business objectives and technical requirements Stay updated with the latest advancements in AzureDatabricksand recommend best practices for continuous improvement

Roles and Responsibilities

Lead the design development and enhancement of complexdatapipelines using AzureDatabricksWrite clean maintainable and efficient PySpark and Python code to supportdataingestion transformation andanalyticsTroubleshoot and resolve issues related todatapipelines and workflows to ensure minimal downtime and high availability Collaborate withdataarchitectsanalystsand stakeholders to gather requirements and deliver scalabledatasolutions Mentor and guide junior team members on AzureDatabricksand best practices indataengineering Documentdataprocesses architectures and workflows to ensure knowledge sharing and compliance Participate actively in design discussions code reviews and agile ceremonies to foster a collaborative and highperforming team environment

ROBOTIC PROCESS AUTOMATION LLC is an equal opportunity employer inclusive of female, minority, disability and veterans, (M/F/D/V). Hiring, promotion, transfer, compensation, benefits, discipline, termination and all other employment decisions are made without regard to race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, citizenship/immigration status, veteran status or any other protected status. ROBOTIC PROCESS AUTOMATION LLC will not make any posting or employment decision that does not comply with applicable laws relating to labor and employment, equal opportunity, employment eligibility requirements or related matters. Nor will ROBOTIC PROCESS AUTOMATION LLC require in a posting or otherwise U.S. citizenship or lawful permanent residency in the U.S. as a condition of employment except as necessary to comply with law, regulation, executive order, or federal, state, or local government contract