Azure Databricks Engineer

Valiant Technologies LLC; Houston (TX); Presencial; USD 110.000 - 160.000; Jornada completa

Descripción de la vacante

VALIANT TECHNOLOGIES LLC is seeking a Specialist with 7 years of experience to design and develop scalable data solutions using Azure Databricks. You will design, develop, and optimize scalable data pipelines to support enterprise-level data processing and analytics.

You will write PySpark and Python code to support data ingestion, transformation and analytics, troubleshoot pipelines, and collaborate with data architects, analysts and stakeholders to align solutions with business objectives.

Formación

• 7 years of experience designing and developing scalable data solutions.
• Proficient in PySpark and Python for data ingestion, transformation and analytics.
• Experience building and optimizing Azure Databricks pipelines and workflows.
• Ability to collaborate with data architects, analysts and stakeholders.

Responsabilidades

• Lead the design, development and enhancement of complex datapipelines using AzureDatabricks.
• Write clean, maintainable PySpark and Python code for data ingestion, transformation and analytics.
• Troubleshoot datapipelines and workflows to ensure minimal downtime and high availability.
• Collaborate with data architects, analysts and stakeholders to gather requirements and deliver scalable data solutions.
• Mentor and guide junior team members on AzureDatabricks and best practices in data engineering.
• Document data processes, architectures and workflows to ensure knowledge sharing and compliance.

Conocimientos

Azure Databricks Experience

PySpark & Python

Data Engineering

Herramientas

Azure Databricks

PySpark

Python

ETL

Descripción del empleo

Seeking a Specialist with 7 years of experience in designing and developing scalabledatasolutions using AzureDatabricks

Design develop and optimize scalabledatapipelines leveraging AzureDatabricksto support enterprise leveldataprocessing andanalyticsImplement and maintain ETL workflows ensuring high performancedataquality and reliability Integrate AzureDatabrickswith various Azuredataservices to build robust and scalabledataplatforms Monitor and optimizedataworkflows for scalability cost effectiveness and operational efficiency Collaborate with cross functional teams to aligndatasolutions with business objectives and technical requirements Stay updated with the latest advancements in AzureDatabricksand recommend best practices for continuous improvement

Roles and Responsibilities

Lead the design development and enhancement of complexdatapipelines using AzureDatabricksWrite clean maintainable and efficient PySpark and Python code to supportdataingestion transformation andanalyticsTroubleshoot and resolve issues related todatapipelines and workflows to ensure minimal downtime and high availability Collaborate withdataarchitectsanalystsand stakeholders to gather requirements and deliver scalabledatasolutions Mentor and guide junior team members on AzureDatabricksand best practices indataengineering Documentdataprocesses architectures and workflows to ensure knowledge sharing and compliance Participate actively in design discussions code reviews and agile ceremonies to foster a collaborative and highperforming team environment

VALIANT TECHNOLOGIES LLC is an equal opportunity employer inclusive of female, minority, disability and veterans, (M/F/D/V). Hiring, promotion, transfer, compensation, benefits, discipline, termination and all other employment decisions are made without regard to race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, citizenship/immigration status, veteran status or any other protected status. VALIANT TECHNOLOGIES LLC will not make any posting or employment decision that does not comply with applicable laws relating to labor and employment, equal opportunity, employment eligibility requirements or related matters. Nor will VALIANT TECHNOLOGIES LLC require in a posting or otherwise U.S. citizenship or lawful permanent residency in the U.S. as a condition of employment except as necessary to comply with law, regulation, executive order, or federal, state, or local government contract