Senior Data Engineer - AI-Driven Data Pipelines

Management Solutions International

Doha

On-site

QAR 300,000 - 540,000

Full time

2 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Management Solutions International is seeking a Senior Data Engineer to own the data stack for a Doha client engagement and extend the data model with new sources. You will ensure data reliability for AI workflows and lead data integration efforts with Databricks and Azure Data Factory on an Azure-hosted platform.

You will join the founding team in Doha, collaborate with the Senior Solutions Architect, and contribute to AI-driven financial technology solutions while upholding agile engineering

Qualifications

  • Master's degree in Computer Science, Engineering, or a related field.
  • 7+ years of significant Data Engineering experience, excluding internships.
  • Excellent verbal and written English communication skills.
  • Outstanding programming skills in PySpark, SQL, and Python.
  • Experience processing data at terabyte scale in production environments.
  • Hands-on experience with Azure Cloud services, particularly Databricks, Azure Data Factory, and Azure DevOps.
  • Familiar with document databases such as Cosmos DB and MongoDB, graph databases such as Neo4j, and ELK technologies.
  • Strong understanding of SQL and NoSQL database internals, query languages, and distributed computing fundamentals.
  • Experience building or operating entity-resolution and record-linkage solutions across messy and complex data sources.
  • Experience with fuzzy matching, blocking, and measuring match quality through precision and recall.
  • Has designed schemas or ontologies for knowledge graphs or linked-data environments, including entity-relationship modelling.
  • Has delivered data integration for external clients or partners, including data contracts, secure data transfer, and access controls.
  • Learns quickly and demonstrates a continuous-improvement mindset.
  • Is highly collaborative and team-oriented.
  • Is proactive in identifying data issues and trade-offs and presenting recommendations.
  • Is comfortable working directly with Client teams and explaining complex data issues to non-technical stakeholders.
  • Can work autonomously within a small team while maintaining close collaboration with wider technical teams.
  • Experience working in regulated or sovereign environments with data-residency requirements would be advantageous.
  • Experience with financial services, private markets, energy, or sustainability data would be valuable; however, a proven track record of delivering reliable, production-grade data engineering solutions is more important than experience in any specific domain.

Responsibilities

  • Design, develop, and implement robust, efficient, and scalable data pipelines into the Client's data platform.
  • Ingest structured data feeds, APIs, and unstructured documents.
  • Build batch pipelines for pre-populated datasets.
  • Build on-demand enrichment pipelines triggered by user requests and business requirements.
  • Build and operate pipelines using Databricks and Azure Data Factory.
  • Ensure all pipelines include unit testing, automation, and CI/CD.
  • Develop production-grade pipelines that are tested, automated, repeatable, and scalable from the outset.

Skills

PySpark
SQL
Python
Databricks
Azure Data Factory
Git
NoSQL databases
Cosmos DB
MongoDB
Neo4j
ELK
Data modeling
Entity resolution

Education

Master's degree in Computer Science, Engineering, or related field

Tools

Cosmos DB
MongoDB
Neo4j
ELK
Azure Databricks
Azure DevOps

Job description

Management Solutions International is seeking a Senior Data Engineer to own the data stack for a Doha client engagement and extend the data model with new sources. You will ensure data reliability for AI workflows and lead data integration efforts with Databricks and Azure Data Factory on an Azure-hosted platform.

You will join the founding team in Doha, collaborate with the Senior Solutions Architect, and contribute to AI-driven financial technology solutions while upholding agile engineering

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Doha Senior Data Engineer — AI Pipelines for Fintech
Doha Senior Data Engineer — AI Pipelines for Fintech

Alpha10x • Doha

On-site
QAR 180,000 - 300,000
Data & AI Engineer: Scalable Pipelines & AI-Ready Data
Data & AI Engineer: Scalable Pipelines & AI-Ready Data

Confidential Company • Qatar

On-site
QAR 180,000 - 320,000
Senior Data Architect - Enterprise Data Platform & AI
Senior Data Architect - Enterprise Data Platform & AI

Power International Holding • Doha

On-site
QAR 350,000 - 550,000
Data Engineer: Real-Time Pipelines, Cloud & ML
Data Engineer: Real-Time Pipelines, Cloud & ML

Employment • Doha

On-site
QAR 218,000 - 437,000
Senior Data Engineer
Senior Data Engineer

Management Solutions International • Doha

On-site
QAR 300,000 - 540,000
Data Engineer: Build Real-Time Pipelines & Cloud ML
Data Engineer: Build Real-Time Pipelines & Cloud ML

Artefact • Doha

On-site
QAR 120,000 - 240,000
Data & AI Engineer
Data & AI Engineer

Confidential Company • Qatar

On-site
QAR 180,000 - 320,000
Senior Data Engineer
Senior Data Engineer

Malomatia • Doha

On-site
QAR 250,000 - 400,000
Data Engineer - Real-Time ETL & Cloud Pipelines
Data Engineer - Real-Time ETL & Cloud Pipelines

Snoonu • Qatar

Remote
QAR 180,000 - 300,000
Global Vibes
Flexible Time Off
Learning budget
Senior Azure Databricks Architect
Senior Azure Databricks Architect

Malomatia • Doha

On-site
QAR 300,000 - 420,000
Top locations