Data/Sr Data Engineer

Milestone Technologies; Hyderabad; Hybrid; INR 900,000 - 1,400,000; Full time

Job summary

Milestone Technologies is seeking a Data Engineer to support an EDW migration from on-prem SQL Server 2016 to the Cloud. The role focuses on data quality, errors, and monitoring of data pipes from source to target cloud database.

You will collaborate with the data pipes team and business analysts to define quality requirements and thresholds, design measurement processes, and develop dashboards and reports displaying data quality metrics.

Qualifications

• Translate data requirements for new data quality controls and thresholds into technical designs and specifications.
• Create automated processes to measure data quality, failures, and errors.
• Analyze existing systems and data pipes to identify areas for improvement or opportunities to enhance.
• Continue to help evolve and improve the data quality processes and templates.
• Profile data sources to understand existing quality issues in the ETL processes.
• Collaborate on data modeling tasks including recommending and evaluating designs based on Kimball dimensional data modeling practices.
• Aggregate and combine raw information from different data sources.
• Explore ways to enhance data quality and reliability including helping with the development of support and remediation processes.
• Identify opportunities for data acquisition.

Responsibilities

• Translate data requirements for new data quality controls and thresholds into technical designs and specifications.
• Create automated processes to measure data quality, failures, and errors.
• Analyze existing systems and data pipes to identify areas for improvement or opportunities to enhance.
• Continue to evolve and improve the data quality processes and templates.
• Profile data sources to understand existing quality issues in the ETL processes.

Job description

Milestone Technologies is looking for a Data Engineer to support a client's EDW migration from on-prem (SQL Server 2016) to the Cloud. This role will focus on the data quality and errors for the new data pipes established from the source to the new target cloud database. You will work closely with the team that is designing and building the data pipes as well as the lead business analyst who is working with internal stakeholders defining the data quality requirements and thresholds. You will be responsible for designing and building the processes for measuring data quality, data transmission failures, and data load errors. In addition to this, you will be expected to work with report developers to create dashboards and reports showcasing the data quality.

Role & responsibilities

• Translate data requirements for new data quality controls and thresholds into technical designs and specifications
• Create automated processes to measure data quality, failures, and errors
• Analyze existing systems and data pipes to identify areas for improvement or opportunities to enhance
• Continue to help evolve and improve the data quality processes and templates
• Profile data sources to understand existing quality issues in the ETL processes
• Collaborate on data modeling tasks including recommending and evaluating designs based on Kimball dimensional data modeling practices
• Aggregate and combine raw information from different data sources
• Explore ways to enhance data quality and reliability including helping with the development of support and remediation processes
• Identify opportunities for data acquisition