Senior Data Engineer (Databricks)

Elitmind Sp. z o.o.; Town of Poland (NY); On-site; USD 120,000 - 180,000; Full time

Job summary

Elitmind Sp. z o.o. seeks a Senior Data Engineer to design and maintain cloud-based data platforms, delivering scalable solutions with our team of Data Engineers.

You will build data pipelines, work across Databricks and Airflow ecosystems, and partner with stakeholders to drive data-driven decisions. Strong SQL and Python skills, plus hands-on experience with modern data architectures, are essential.

Qualifications

• 5+ years of experience with data platforms, deployment, and maintenance.
• Strong, hands-on Databricks experience is required.
• Excellent SQL and Python coding skills with best practices.
• Experience with Git-based repositories (Bitbucket, GitHub, GitLab).
• Familiarity with Spark, Hadoop, Azure Data Factory, Kafka/Event Hub, Airflow, dbt or equivalent.

Responsibilities

• Design and build cloud-based data platform frameworks and architectures.
• Integrate data from diverse structured and unstructured sources.
• Implement automation for data workflows and CI/CD processes.
• Optimize performance and resource usage of cloud data solutions.
• Maintain knowledge of new technologies in data and cloud engineering.
• Monitor, troubleshoot, and optimize data systems for performance and cost.
• Support deployment of analytics and ML solutions on cloud platforms.
• Document processes, infrastructure, and best practices for scalability.

Skills

Databricks

Data platforms

SQL

Python

CI/CD

Azure

Data services

ETL

Tools

Spark

Hadoop

Azure Data Factory

Apache Kafka

Airflow

dbt

Job description

As Senior Data Engineer, you will be responsible for developing and maintaining data platforms and delivering with our team of Data Engineers through challenges.

This key role embraces participating in the entire engineering lifecycle and ensuring successful data platform implementation. The ideal candidate will build and design data platform frameworks and data pipelines, act as a trusted advisor for Databricks and Airflow technology for clients, and enhance process effectiveness. A data engineer collaborates with the project team on project delivery and standards, with expertise in data warehouse and lakehouse development for large-scale customers. Essential skills include communication, collaboration, problem-solving, a data-driven mindset, and a passion for automation.

What makes you a great fit?

Key technologies

Databricks | Data Platforms | SQL | Python | CI/CD | Azure | Data Services | ETL

• 5+ years of experience in configuration/release management, deployment, and maintenance and familiarity with various data platforms and practices in the data area
• Strong, hands-on experience in the Databricks ecosystem is a must.
• Very good coding skills with knowledge of best practices in SQL, Python
• Experience with Git-based repositories (Bitbucket, GitHub, GitLab)
• Familiarity with tools like Spark, Hadoop, Azure Data Factory, Apache Kafka/Event Hub, Airflow, dbt, or other data platform tools.
• Conceptual and technical knowledge on CI/CD pipelines, automation practices, and technical operations monitoring
• Strong sense of ownership, flexibility, and independence

Nice to have

• Hands-on experience of automated testing frameworks (Greater Expectations, customer-made, etc.)
• Familiarity with Oracle or Apache ecosystem

Soft skills

• Analytical Thinking
  Ability to interpret data, identify trends, and provide actionable insights to drive business decisions.
• Problem-Solving Skills
  Proactively identify and resolve issues related to BI systems, ensuring smooth operations and data accuracy.
• Effective Communication
  Ability to explain technical concepts to non-technical stakeholders clearly and concisely, fostering understanding and collaboration.
• Attention to Detail
  Meticulous in data validation, report generation, and ensuring data quality across BI platforms.
• Team Collaboration
  Strong interpersonal skills to work effectively in cross-functional teams, coordinating with IT, business analysts, and other stakeholders.
• Adaptability and Continuous Learning
  Open to learning new tools and technologies in the rapidly evolving BI landscape and adjusting to changing business needs.

What will you do?

• Design and build cloud-based data platform frameworks and architectures.
• Integrate data from various structured and unstructured sources and formats.
• Implement automation for data workflows and support CI/CD processes.
• Optimize performance and resource usage of cloud data solutions.
• Maintain up-to-date knowledge of new technologies in data and cloud engineering.
• Monitoring, troubleshooting, and optimizing data systems for performance and cost.
• Supporting deployment of analytics and machine learning solutions on cloud platforms.
• Documenting processes, infrastructure, and best practices for future scalability