Databricks Engineer

Everscale Group; Ciudad de México; Presencial; USD 60,000 - 100,000; Jornada completa

Descripción de la vacante

An innovative firm is seeking a Databricks Engineer to design and maintain scalable data processing solutions. In this role, you will leverage your expertise in big data technologies and programming languages like Python, Scala, or Java to create efficient data workflows. You'll collaborate with data scientists and business stakeholders to ensure data quality and integrity, while also optimizing data processing environments for performance. This position offers a dynamic work environment where your contributions will directly impact the company's data capabilities and drive business success. Join a forward-thinking team and advance your career in data engineering!

Formación

• Bachelor's degree in Computer Science or related field required.
• Strong expertise in Databricks and big data technologies.

Responsabilidades

• Design and implement scalable data processing solutions using Databricks.
• Collaborate with teams to develop optimal data structures and architectures.

Conocimientos

Databricks Unified Analytics Platform

Python

Scala

Java

Apache Spark

Data Management

ETL Processes

Cloud Platforms (AWS, Azure, Google Cloud)

Data Warehousing

Educación

Bachelor’s degree in Computer Science

Bachelor’s degree in Information Technology

Descripción del empleo

As a Databricks Engineer, you will be responsible for designing, implementing, and maintaining scalable data processing and analytics solutions using Databricks Unified
Analytics Platform. The ideal candidate possesses a deep understanding of big data technologies, proficient coding skills, and a strong background in data engineering and
analytics.

Responsibilities:

Architecture and Design:

• Design and implement scalable and efficient data processing solutions using Databricks.
• Collaborate with architects and data scientists to develop optimal data structures and architectures.

Development:

• Write efficient and maintainable code in languages like Python, Scala, or Java.
• Develop and implement ETL processes for data integration and transformation.

Data Management:

• Manage and optimize large-scale data storage and processing environments.
• Ensure data quality and integrity throughout the data lifecycle.

Collaboration:

• Work closely with data scientists, analysts, and business stakeholders to understand data requirements.
• Collaborate with cross-functional teams to integrate data solutions into business processes.

Monitoring and Optimization:

• Monitor performance and troubleshoot issues in Databricks clusters.
• Optimize data processing workflows for improved efficiency and cost-effectiveness.

Documentation:

• Create and maintain comprehensive documentation for data processing workflows and solutions.
• Keep documentation up-to-date with changes and improvements.

Stay Current:

• Stay abreast of the latest developments in big data technologies and Databricks features.
• Evaluate and recommend new tools and technologies to enhance data processing capabilities.

Qualifications:

Educational Background:

• Bachelor’s degree in computer science, Information Technology, or a related field.

Technical Proficiency:

• Strong expertise in Databricks Unified Analytics Platform.
• Proficiency in programming languages such as Python, Scala, or Java.
• Experience with big data technologies like Apache Spark.
• Familiarity with cloud platforms, such as AWS, Azure, or Google Cloud.
• Knowledge of data warehousing and ETL processes.

Experience:

• Proven experience in designing, implementing, and maintaining data processing pipelines.
• Hands-on experience with data modeling and database design.
• Previous work with large-scale data storage and processing systems.

Communication Skills:

• Excellent communication skills to collaborate with cross-functional teams.
• Ability to convey technical concepts to non-technical stakeholders.