Senior Data Engineer

Management Solutions International

Doha

Presencial

QAR 300.000 - 540.000

Jornada completa

Hace 2 días
Sé de los primeros/as/es en solicitar esta vacante

Consigue una respuesta de este empleador

Tu currículum y tu carta de presentación, adaptados exactamente a lo que buscan contratar.

Supera los filtros ATS

Descripción de la vacante

Management Solutions International is seeking a Senior Data Engineer to own the data stack for a Doha client engagement and extend the data model with new sources. You will ensure data reliability for AI workflows and lead data integration efforts with Databricks and Azure Data Factory on an Azure-hosted platform.

You will join the founding team in Doha, collaborate with the Senior Solutions Architect, and contribute to AI-driven financial technology solutions while upholding agile engineering

Formación

  • Master's degree in Computer Science, Engineering, or a related field.
  • 7+ years of significant Data Engineering experience, excluding internships.
  • Excellent verbal and written English communication skills.
  • Outstanding programming skills in PySpark, SQL, and Python.
  • Experience processing data at terabyte scale in production environments.
  • Hands-on experience with Azure Cloud services, particularly Databricks, Azure Data Factory, and Azure DevOps.
  • Familiar with document databases such as Cosmos DB and MongoDB, graph databases such as Neo4j, and ELK technologies.
  • Strong understanding of SQL and NoSQL database internals, query languages, and distributed computing fundamentals.
  • Experience building or operating entity-resolution and record-linkage solutions across messy and complex data sources.
  • Experience with fuzzy matching, blocking, and measuring match quality through precision and recall.
  • Has designed schemas or ontologies for knowledge graphs or linked-data environments, including entity-relationship modelling.
  • Has delivered data integration for external clients or partners, including data contracts, secure data transfer, and access controls.
  • Learns quickly and demonstrates a continuous-improvement mindset.
  • Is highly collaborative and team-oriented.
  • Is proactive in identifying data issues and trade-offs and presenting recommendations.
  • Is comfortable working directly with Client teams and explaining complex data issues to non-technical stakeholders.
  • Can work autonomously within a small team while maintaining close collaboration with wider technical teams.
  • Experience working in regulated or sovereign environments with data-residency requirements would be advantageous.
  • Experience with financial services, private markets, energy, or sustainability data would be valuable; however, a proven track record of delivering reliable, production-grade data engineering solutions is more important than experience in any specific domain.

Responsabilidades

  • Design, develop, and implement robust, efficient, and scalable data pipelines into the Client's data platform.
  • Ingest structured data feeds, APIs, and unstructured documents.
  • Build batch pipelines for pre-populated datasets.
  • Build on-demand enrichment pipelines triggered by user requests and business requirements.
  • Build and operate pipelines using Databricks and Azure Data Factory.
  • Ensure all pipelines include unit testing, automation, and CI/CD.
  • Develop production-grade pipelines that are tested, automated, repeatable, and scalable from the outset.

Conocimientos

PySpark
SQL
Python
Databricks
Azure Data Factory
Git
NoSQL databases
Cosmos DB
MongoDB
Neo4j
ELK
Data modeling
Entity resolution

Educación

Master's degree in Computer Science, Engineering, or related field

Herramientas

Cosmos DB
MongoDB
Neo4j
ELK
Azure Databricks
Azure DevOps

Descripción del empleo

Job description
Senior Data Engineer

The Senior Data Engineer will own the data side of our client engagement in Doha. Joining the founding team in Doha and working day-to-day with the Senior Solutions Architect, the successful candidate will bring the Client’s own data and new sector-specific sources into the platform, extend the data model and ontology to cover them, and ensure the data is reliable enough for AI workflows to reason upon.

The Senior Data Engineer will build and operate pipelines using Databricks and Azure Data Factory on an Azure-hosted platform while upholding agile engineering practices, including unit testing, automation, and continuous integration and deployment through CI/CD.

The successful candidate will be someone who takes initiative, stays curious, and has an unrelenting drive to push the boundaries of AI and data science. They will be adaptive and introspective, willing to learn, guide, lead, and follow, with a low-ego mindset where outcomes matter more than individual recognition.

This is an opportunity to play a key role in advancing AI capabilities within the financial technology sector, working with highly experienced professionals as one of the first engineers in the Doha office and helping shape how the Client delivers solutions across the region.

As part of a fast-paced, high-growth environment, the successful candidate will have a direct impact on business growth and strategic direction, with significant opportunities for personal and professional advancement while contributing to AI-driven financial solutions.

Key Responsibilities ETL Development
  • Design, develop, and implement robust, efficient, and scalable data pipelines into the Client's data platform.
  • Ingest structured data feeds, APIs, and unstructured documents.
  • Build batch pipelines for pre-populated datasets.
  • Build on-demand enrichment pipelines triggered by user requests and business requirements.
  • Build and operate pipelines using Databricks and Azure Data Factory.
  • Ensure all pipelines include unit testing, automation, and CI/CD.
  • Develop production-grade pipelines that are tested, automated, repeatable, and scalable from the outset.
Client Data Integration
  • Onboard the Client's proprietary and licensed datasets into the platform.
  • Establish data contracts, schema mapping, secure connectors, and permission-based access controls.
  • Work within agreed data residency, security, and governance requirements.
  • Ensure client data is securely integrated and managed according to agreed access and residency rules.
Data Modelling & Ontology
  • Extend the data ontology to accommodate each new data source.
  • Map new sources across entities such as organizations, people, transactions, funds, and their relationships.
  • Work with the Head of Data to determine whether new concepts should be represented as entities or attributes.
  • Register new entities, relationships, and attributes within the field catalogue to ensure exposure through the platform API.
  • Build the data layer for sector-specific ontology packs, including new source types such as patents, scientific literature, regulatory records, and project registries.
  • Develop new entity types based on the Client's priority use cases.
  • Ensure new data is consistently modelled and accessible through the platform API.
Entity Resolution
  • Extend record-linkage and entity-resolution capabilities to new data sources.
  • Ensure client and third-party records are accurately matched to the correct organizations and entities.
  • Apply techniques such as fuzzy matching and blocking where appropriate.
  • Measure matching precision and recall to validate data quality and resolution accuracy.
Data Quality & Observability
  • Define and enforce data-quality checks across every pipeline.
  • Monitor completeness, freshness, duplication, and schema conformance.
  • Implement alerting and dashboards to identify data issues proactively.
  • Ensure issues are identified and addressed before reaching client-facing workflows.
Cost & Scale Trade-offs
  • Evaluate the economic impact of every new data source.
  • Estimate acquisition, processing, storage, and refresh costs.
  • Recommend whether sources should be pre-populated through batch processing or retrieved on demand.
  • Ensure each data source has a clear cost and maintenance strategy.
Documentation & Handover
  • Develop comprehensive documentation for pipelines, data contracts, and operational runbooks.
  • Ensure solutions can be operated and maintained by the wider data team and future hires in Doha.
  • Build sustainable systems that do not depend on a single individual.
What We Are Looking For

This role requires a combination of hands-on data engineering at scale, strong data-quality practices, and the ability to work directly with Client data. We are looking for someone who:

  • Has a Master's degree, at minimum, in Computer Science, Engineering, or a related field.
  • Has 7+ years of significant Data Engineering experience, excluding internships.
  • Has excellent verbal and written English communication skills.
  • Has outstanding programming skills in PySpark, SQL, and Python.
  • Has proven experience processing data at terabyte scale in production environments.
  • Is comfortable working with Git and modern version-control practices.
  • Is familiar with document databases such as Cosmos DB and MongoDB, graph databases such as Neo4j, and ELK technologies.
  • Has hands-on experience with Azure Cloud services, particularly Databricks, Azure Data Factory, and Azure DevOps.
  • Has a strong understanding of SQL and NoSQL database internals, query languages, and distributed computing fundamentals.
  • Has experience building or operating entity-resolution and record-linkage solutions across messy and complex data sources.
  • Has experience with fuzzy matching, blocking, and measuring match quality through precision and recall.
  • Has designed schemas or ontologies for knowledge graphs or linked-data environments, including entity-relationship modelling.
  • Has delivered data integration for external clients or partners, including data contracts, secure data transfer, and access controls.
  • Learns quickly and demonstrates a continuous-improvement mindset.
  • Is highly collaborative and team-oriented.
  • Is proactive in identifying data issues and trade-offs and presenting recommendations.
  • Is comfortable working directly with Client teams and explaining complex data issues to non-technical stakeholders.
  • Can work autonomously within a small team while maintaining close collaboration with wider technical teams.
  • Experience working in regulated or sovereign environments with data-residency requirements would be advantageous.
  • Experience with financial services, private markets, energy, or sustainability data would be valuable; however, a proven track record of delivering reliable, production-grade data engineering solutions is more important than experience in any specific domain.
Consigue la evaluación confidencial y gratuita de tu currículum.

o arrastra y suelta tu archivo aquí

Similar jobs

Puestos de trabajo similares que vale la pena comparar

Senior Data Engineer
Senior Data Engineer

Alpha10x • Doha

Presencial
QAR 180.000 - 300.000
Senior Solutions Architect
Senior Solutions Architect

Alpha10x • Doha

Presencial
QAR 546.000 - 764.000
Senior Data Engineer
Senior Data Engineer

Malomatia • Doha

Presencial
QAR 250.000 - 400.000
Senior Data Engineer - AI-Driven Data Pipelines
Senior Data Engineer - AI-Driven Data Pipelines

Management Solutions International • Doha

Presencial
QAR 300.000 - 540.000
Data & AI Engineer
Data & AI Engineer

Confidential Company • Qatar

Presencial
QAR 180.000 - 320.000
Technical Lead / Solution Architect
Technical Lead / Solution Architect

Black & Grey HR • Doha

Presencial
QAR 320.000 - 520.000
Competitive salary
Benefits package
Technical Lead / Solution Architect
Technical Lead / Solution Architect

Black Grey • Qatar

Presencial
QAR 350.000 - 600.000
Business Data Architect
Business Data Architect

Power International Holding • Doha

Presencial
QAR 350.000 - 550.000
Senior Databricks Specialist/Data Architect | Marc Ellis | Doha, Qatar
Senior Databricks Specialist/Data Architect | Marc Ellis | Doha, Qatar

Marc Ellis • Doha

Presencial
QAR 437.000 - 619.000
Senior Data Platform Architect & Tech Lead
Senior Data Platform Architect & Tech Lead

Black Grey • Qatar

Presencial
QAR 350.000 - 600.000
Ubicaciones