Microsoft Fabric & Data Platform Development
- Design, develop, and maintain end-to-end dataengineering solutions using Microsoft Fabric.
- Build and manage Fabric Data Factory pipelinesfor batch, scheduled, and incremental data ingestion.
- Develop and maintain Fabric Lakehouse andWarehouse solutions, using OneLake for centralized data storage and management.
- Use Fabric notebooks, Spark, SQL analytics, andDataflows Gen2 to process and transform data.
- Apply Medallion Architecture (Bronze, Silver,and Gold) where appropriate for data organization, quality, and reuse.
Data Ingestion, Transformation & Modeling
- Develop robust ETL/ELT processes to ingest datafrom databases, REST APIs, files, and cloud/SaaS applications.
- Implement incremental loads, change data capture(CDC), slowly changing dimensions (SCD), partitioning, and other suitableprocessing strategies.
- Develop and optimize analytical data models,including dimensional models and star/snowflake schemas.
- Use SQL, Python, PySpark, and Spark SQL totransform, validate, and prepare data for reporting and analytics.
- Implement data quality checks, validation,reconciliation, error handling, and appropriate recovery mechanisms.
Performance, Reliability & Operations
- Monitor pipeline execution and data processing;troubleshoot failures and perform root-cause analysis.
- Optimize Spark jobs, SQL queries, pipelinedesign, and storage usage for performance, scalability, and cost efficiency.
- Implement logging, monitoring, alerting, andproduction-support practices to improve platform reliability.
- Support issue resolution and continuousimprovement across data pipelines and platform workloads.
Security, Governance & Deployment
- Apply appropriate security controls, workspaceaccess, identity and access management, and data access practices with platformand security teams.
- Follow data governance, documentation, andengineering standards; support data lineage and governance requirements whereapplicable.
- Use Git-based source control and support CI/CDand deployment processes for Fabric solutions through Azure DevOps or GitHub.
- Document data architecture, pipelines, datamodels, dependencies, operational procedures, and technical decisions.
Collaboration & Business Alignment
- Work with Power BI teams to provide curated,trusted, analytics-ready datasets and support semantic-model requirements.
- Collaborate with analysts, data scientists,application teams, and business stakeholders to understand requirements anddeliver practical solutions.
- Contribute to design reviews, estimation, sprintplanning, testing, release activities, and continuous improvement in an Agiledelivery environment.
Required Qualifications
Experience & Education
- 3+ years of experience in data engineering, withmandatory hands-on experience developing solutions in Microsoft Fabric.
- Bachelor’s or Master’s degree in ComputerScience, Information Technology, Engineering, Data Science, or a related field;equivalent professional experience may be considered.
- Experience building end-to-end data pipelinesand supporting data solutions through development, deployment, and production.
Microsoft Fabric Skills
- Hands-on experience with Fabric Data Factory /Data Pipelines, Lakehouse, Warehouse, OneLake, notebooks, Spark, SQL analytics,and Dataflows Gen2.
- Understanding of Microsoft Fabric architecture,workspaces, and end-to-end data processing patterns.
- Ability to develop, troubleshoot, and optimizeproduction-ready data pipelines in Microsoft Fabric.
- Familiarity with Fabric data agents and theirpotential use in data discovery or conversational access to data.
Core Data Engineering Skills
- Strong SQL skills, including complex queries,joins, CTEs, window functions, and query optimization.
- Strong programming knowledge in Python and/orPySpark, with practical experience in data transformation and processing.
- Solid understanding of ETL/ELT concepts, batchand incremental processing, data warehousing, and dimensional modeling.
- Experience with data quality, validation,reconciliation, and reliable pipeline design.
- Working understanding of cloud security,identity, access management, and secure handling of enterprise data.
- Experience with Git and source control, andfamiliarity with Azure DevOps or GitHub.
- Understanding of CI/CD pipelines, automateddeployment, environment management, and release practices.
- Experience with monitoring, logging,troubleshooting, and production support.
Preferred Qualifications
- Microsoft Certified: Fabric Data EngineerAssociate (DP-700) is preferred; Microsoft Certified: Fabric Analytics EngineerAssociate (DP-600) is also relevant.
- Experience migrating data solutions from AzureSynapse Analytics, Azure Data Factory, Databricks, or traditional datawarehouses to Microsoft Fabric.
- Experience with Power BI semantic models andDAX.
- Knowledge of Microsoft Purview, data governance,data lineage, and compliance practices.
- Exposure to real-time analytics, Eventstream, orReal-Time Intelligence in Microsoft Fabric.
- Experience integrating REST APIs and SaaS datasources.
- Familiarity with Microsoft Azure AI Foundry /Microsoft Foundry and its data or AI-related capabilities.
- Experience working in Agile/Scrum teams.
Preferred Candidate Profile
- Strong analytical thinking, problem-solving, andattention to data quality.
- Able to work independently while collaboratingeffectively with cross-functional teams.
- Clear communication and technical documentationskills.
- Able to translate business requirements intoscalable and maintainable data solutions.
- Takes ownership of data engineering solutionsfrom design and development through deployment and production support.
- Shows initiative in learning and applyingevolving Microsoft Fabric and modern data engineering capabilities.
Key Success Measures
- Reliable, maintainable, and well-documentedFabric pipelines delivered to agreed timelines.
- Improved data quality, reconciliation, andavailability of trusted analytics-ready datasets.
- Effective pipeline and query performance, withappropriate cost and resource optimization.
- Timely resolution of pipeline issues andadherence to security and engineering standards.
- Effective collaboration with reporting,analytics, application, and business teams.