Senior DataOps Engineer

Andela; Teletrabalho; BRL 80.000 - 120.000; Tempo integral

Resumo da oferta

A global technology company seeks an experienced Data Engineer to build and operate a robust data platform on Azure using Databricks. The ideal candidate will have over 7 years of experience in Data Engineering/DataOps, with a strong background in cloud services and data governance. This role emphasizes collaboration, independence, and a commitment to high data quality. The position is remote, requiring at least 5 hours of overlap with UTC-5 (EST). Competitive long-term career opportunities await successful applicants.

Qualificações

• 7+ years of overall experience in Data Engineering / DataOps.
• 5+ years of professional software development experience in a Databricks environment.
• Strong experience with implementing data quality checks at scale.

Responsabilidades

• Design and develop a robust monitoring stack for jobs executed in Azure Data Factory.
• Assist in building and maintaining data catalogs.
• Monitor data completeness and accuracy.

Conhecimentos

Data Engineering

DataOps

Databricks

Data Governance

SQL

ETL processes

Formação académica

Bachelor’s or Master’s degree in Computer Science or Software Engineering

Ferramentas

Azure Data Factory

Unity Catalog

Purview

Descrição da oferta de emprego

Andela exists to connect brilliance and opportunity. Since 2014, we have been dedicated to breaking down global barriers and accelerating the future of work for both technologists and organizations around the world. For technologists, Andela offers competitive long-term career opportunities with leading organizations, access to a global community of professionals, and educational opportunities with leading technology providers. At Andela, we’re deeply passionate about creating long-lasting and transformative growth opportunities for all - and doing it in an E.P.I.C. way! We’re excited to continue building our remote-first team with incredible people like you. After applying for this role, you will join our Andela Community of brilliant technologists by passing a technical screening and live interview. As a community member, you’ll have access to a multitude of exclusive technologist roles.

Join Andela today to access this opportunity and more in our global marketplace!

Our roles are typically filled at lightning speed, so if you’re considering applying, get your application in quickly!

About the client

The client operates in the health and information services domain, with a strong focus on enabling data-driven decision-making at enterprise scale. Their mission centers on building a modern, cloud-based data platform that serves as a single source of truth for business intelligence, analytics, machine learning, and AI use cases. The culture emphasizes data quality, governance, scalability, and collaboration across engineering, analytics, and business stakeholders.

About the role

This role is focused on building and operating a robust data platform on Azure using Databricks. The engineer will play a key part in DataOps, ensuring data completeness, accuracy, and reliability across enterprise-scale pipelines. You will design monitoring frameworks, implement data quality checks, and support governance initiatives. The role is highly collaborative, working closely with data engineers, analysts, and stakeholders. It also requires strong ownership and the ability to operate independently in a fast-paced environment.

Responsibilities

• Design and develop a robust monitoring stack for jobs executed in Azure Data Factory, Databricks, MS Fabric, and other platform tools
• Assist in building and maintaining data catalogs in Unity Catalog and Purview
• Design and implement a scalable framework for data quality checks across hundreds of columns on large datasets
• Monitor data completeness and accuracy to ensure published data meets acceptable criteria
• Collaborate with data engineers, analysts, and stakeholders to deliver high-quality data solutions
• Troubleshoot and optimize big data workflows for performance and scalability
• Support governance and oversight of data pipelines and transformation activities
• Stay current with emerging technologies and best practices in big data and analytics

Must have skills

• 7+ years of overall experience in Data Engineering / DataOps
• 5+ years of professional software development experience in a Databricks environment (any cloud)
• Strong experience with DataOps practices
• Strong experience with data catalogs
• Strong experience implementing data quality checks at scale
• Strong proficiency in Data Governance processes and tools
• Experience with Purview and / or Unity Catalog
• Advanced SQL skills
• Solid understanding of data structures, algorithms, and distributed computing
• Experience with ETL processes, data cleaning, and large-scale data transformation
• Ability to work independently and manage priorities in a fast-paced environment
• Excellent problem-solving skills with an analytics-driven mindset
• Bachelor’s or Master’s degree in Computer Science, Software Engineering, or a related field, or equivalent experience

Nice to have skills

• Experience with Azure cloud services (preferred)
• Experience with AWS
• Experience with GCP
• Familiarity with version control systems such as Git
• Experience with DevOps practices

Conditions

100% Remote role, primarily aligned to UTC-5 (EST) with a minimum of 5 hours overlap required

Contractor role – 6-month contract, renewable based on budget and performance

BYOD requirement

At Andela, we know our strengths lie in our diverse community whose talents, perspectives, backgrounds and orientations we take pride in. Andela is committed to nurturing a work environment where all individuals are treated with respect and dignity. Everyone has the right to work in a professional atmosphere that promotes equal employment opportunities and prohibits discriminatory practices.

Andela provides equal employment opportunities to all employees and applicants without regard to factors including but not limited to race, color, religion, gender, sexual orientation, gender identity, national origin, age, disability, pregnancy (including breastfeeding), genetic information, HIV / AIDS or any other medical status, family or parental status, marital status, amnesty or status as a covered veteran in accordance with applicable federal, state and local laws. This commitment applies to all terms and conditions of employment, including but limited to hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation