MSR Cosmos is seeking a Data Engineer / Lead Data Engineer (hybrid) to design and deliver data foundations and end-to-end analytics solutions for a complex energy trading environment.
Responsibilities
- Lead the design and development of scalable data ingestion and transformation pipelines using standard data lake and cloud engineering frameworks
- Build solutions for structured and unstructured data in high-volume, high-velocity, high-variety environments
- Develop transformation notebooks, physical and tabular data models, data endpoints, semantic layers, and visualisations
- Create reusable data products that improve data accessibility, consistency, and business value
- Lead ETL/ELT activities across source systems, data platforms, and downstream BI solutions
- Create and publish dashboards and reports for stakeholders with accessible, actionable information
- Run trend and correlation analysis; detect data discrepancies and implement controls to protect data quality across platforms
- Define and enforce engineering standards, reusable patterns, documentation, and development practices
- Build automated CI and release pipelines and apply disciplined source-control practices
- Gather functional and non-functional requirements; translate features and Product Backlog Items into technical designs and delivery direction
- Collaborate with cross-functional teams and mentor engineers where required
Requirements
- Mandatory: demonstrable data engineering experience in energy trading, commodities trading, or an ETR/CTRM environment
- Strong hands‑on experience designing production‑grade ingestion, storage, transformation, modelling, and serving layers on Microsoft Azure
- Advanced SQL, including development and optimisation of complex queries for data transformation and analysis
- Hands‑on experience with Azure Data Factory, Azure SQL, Azure Databricks, Azure Blob Storage, and Azure Data Lake Storage
- Strong ETL/ELT and data warehousing experience, including delivery of BI solutions
- Experience with dimensional, physical, and tabular data modelling and performance optimisation
- Experience with GitHub and/or Azure DevOps, including automated build, test, and release pipelines
- Ability to gather requirements, shape technical solutions, and communicate clearly with technical and business stakeholders
- Proven ability to lead workstreams, manage priorities, and deliver effectively in a collaborative team environment
Preferred Experience
- Experience with dbt, Azure Analysis Services, Azure Table Storage, and Azure Logic Apps
- Experience with Power BI, including clear, decision‑oriented dashboards and reports
- Exposure to Power Apps and broader Microsoft Power Platform capabilities
- Familiarity with IBM DataStage or comparable enterprise integration tooling
- Basic understanding of AI engineering, including how governed data products support analytics and AI use cases
- Knowledge of data governance, lineage, metadata management, observability, and automated data‑quality controls
Technologies
- SQL
- Microsoft Azure
- Azure Data Factory
- Azure SQL
- Azure Databricks
- Azure Blob Storage
- Azure Data Lake Storage
- GitHub
- Azure DevOps
Location: Houston, TX (hybrid)
Compensation: USD 65 - 70 per hour