Senior Data Engineer

Aventior Digital; United States; Remote; USD 60,000 - 80,000; Full time

Job description

Designation: Senior Data Engineer

Preferred Experience: 7+ years

Responsibilities:

• Design, develop, and maintain data pipelines for ingesting, processing, and transforming data from various sources into actionable insights.
• Integrate data from disparate sources (databases, APIs, and files) into a unified data platform using ETL processes and data integration techniques.
• Design and implement data models, schemas, and data structures to support analytical queries, reporting, and business intelligence needs.
• Optimize database performance, query execution, and data processing workflows for efficiency, scalability, and reliability.
• Ensure data quality, integrity, and consistency through data validation, cleansing, deduplication, and error-handling mechanisms.
• Architect and implement data solutions on Azure cloud platforms, leveraging Azure services for data storage, processing, and analytics.
• Implement data security measures, encryption techniques, and access controls to protect sensitive data and ensure compliance with regulations (e.g., GDPR, HIPAA).
• Work closely with cross-functional teams, data scientists, analysts, and stakeholders to understand data requirements, provide data solutions, and communicate insights effectively.
• Document data processes, workflows, and best practices, and promote data governance standards, data lineage, and metadata management.
• Stay updated with emerging technologies, industry trends, and best practices in data engineering, cloud computing, and data analytics to drive innovation and continuous improvement.

Required Skills:

• Proficiency in writing complex SQL queries, stored procedures, and functions for data extraction, transformation, and analysis.
• Experience in database design, optimization, and management using SQL Server/Azure SQL Database.
• Knowledge of data modeling techniques, including entity-relationship diagrams, dimensional modeling, and data normalization, is needed to design efficient data structures.
• Familiarity with ETL (Extract, Transform, Load) processes and tools such as Azure Data Factory, SSIS (SQL Server Integration Services), or other data integration platforms.
• Hands-on experience with Azure cloud services, including Azure SQL Database, Azure Data Factory and Azure Storage.
• Experience handling large-scale data processing and analytics with strong analytical skills and attention to detail.
• Strong statistical knowledge.
• Excellent communication skills.

Good To Have Skills:

• Experience working with C#, .Net Framework
• Experience with other Azure cloud services and sharepoint
• Experience with Python as programming language
• Experience manipulating unstructured data with regular expressions.
• Basic understanding of machine learning concepts and algorithms for data mining, predictive modeling, and statistical analysis.
• Knowledge of data warehousing concepts, methodologies, and tools for building and maintaining data warehouses or data marts.