Location: USA
Experience: 10+ Years
Employment Type: Full-Time
Job Description
We are looking for an experienced Senior Data Engineer with strong Microsoft Fabric expertise to design, develop, and optimize scalable data engineering solutions. The ideal candidate will have hands-on experience with modern data platforms, cloud technologies, data pipelines, and enterprise data architecture.
Key Responsibilities
- Design and implement scalable data pipelines using Microsoft Fabric, Data Factory, Lakehouse, and Warehouse.
- Develop data transformation workflows using PySpark, Spark SQL, Python, and SQL.
- Build and optimize Lakehouse architectures, Delta tables, and OneLake solutions.
- Implement ETL/ELT processes for structured and unstructured data.
- Integrate data from multiple sources using Azure Data Factory, APIs, and cloud-based connectors.
- Optimize data processing performance, pipeline reliability, and query execution.
- Implement data quality, governance, security, and access controls.
- Work with Power BI teams to support reporting, analytics, and business intelligence solutions.
- Collaborate with architects, analysts, and business stakeholders to deliver enterprise data solutions.
- Follow best practices for CI/CD, version control, monitoring, and deployment.
Required Skills
- 10+ years of experience in Data Engineering, Data Warehousing, or Big Data.
- Strong hands-on experience with Microsoft Fabric and its core components.
- Expertise in PySpark, Python, Spark SQL, and advanced SQL.
- Experience with Fabric Lakehouse, Warehouse, OneLake, and Data Factory.
- Strong understanding of ETL/ELT, dimensional modeling, and data warehouse concepts.
- Experience with Azure Data Factory, Azure Data Lake Storage, or other cloud data platforms.
- Knowledge of Delta Lake, data partitioning, performance tuning, and large-scale data processing.
- Experience with Git, CI/CD pipelines, and automated deployment practices.
- Strong problem-solving, communication, and stakeholder collaboration skills.
Preferred Qualifications
- Microsoft Fabric certifications or relevant Azure data certifications.
- Experience with Fabric Real-Time Intelligence, Direct Lake, and semantic models.
- Knowledge of data governance, Microsoft Purview, and enterprise security practices.
- Experience migrating legacy data platforms to Microsoft Fabric.
- Exposure to AWS, Snowflake, Databricks, or other modern data platforms.