Data Engineer Lead

GE; United States; Remote; USD 80,000 - 120,000; Full time

Job summary

Join a forward-thinking company as an ETL Lead, where you will leverage your expertise in ETL tools and Azure services to design and manage robust data solutions. This role involves leading a talented team of data engineers, optimizing data pipelines, and ensuring data integrity and compliance. With a focus on innovation and collaboration, you will play a key role in transforming data into actionable insights, supporting business objectives, and driving continuous improvement. If you are passionate about data engineering and have a solutions-oriented mindset, this is the perfect opportunity for you to make a significant impact.

Qualifications

• Expertise in ETL tools like Informatica and Azure services.
• Strong programming skills in Python and SQL for data manipulation.
• Experience with big data technologies like Hadoop and Spark.

Responsibilities

• Design and implement ETL workflows using Informatica.
• Lead a team of data engineers to build scalable data pipelines.
• Ensure data quality and compliance with regulations.

Skills

ETL Tools (Informatica)

Azure Data Factory

Python

SQL

Data Engineering

Azure Databricks

Data Warehousing

Problem-Solving

Leadership Skills

Education

Bachelor's or Master's in Computer Science

Tools

Azure Synapse Analytics

Azure SQL Database

Power BI

Tableau

Job description

Job Description SummaryWe are looking for a skilled and experienced ETL Lead with strong experience in ETL tools like Informatica on Cloud ( IICS ) and Azure Synapse Analytics to join our collaborative IT team. You will be responsible for designing, implementing, and managing data solutions on the Microsoft Azure platform.

Role Overview:

• Design and Implement robust ETL workflows using Informatica to extract, transform and load data from diverse sources.
• Design data solutions, including data lakes, data warehouses, and real-time data processing systems, leveraging Azure services like Azure Data Lake Storage, Azure Synapse Analytics
• Lead a team of azure data engineers in designing, building, and maintaining scalable and efficient data pipelines on Azure cloud platform, utilizing services like Azure Synapse, Azure Data Factory, Azure Databricks, Azure SQL Database, and others.
• Integrate diverse data sources into Azure-based solutions. Ensure smooth and efficient ETL processes, real-time data ingestion, and data transformation. Implement data integration best practices.
• Design and implement data models and schemas to support business requirements. Ensure data accuracy, consistency, and reliability. Optimize data structures for performance and scalability.
• Implement robust data security measures, including encryption, access control, and data masking. Ensure compliance with data privacy regulations and company policies.
• Monitor and optimize data pipelines and queries for performance and efficiency. Implement caching, partitioning, and indexing strategies. Troubleshoot and resolve performance issues.
• Establish and enforce data quality standards. Implement data governance practices, metadata management, and data lineage tracking. Ensure data quality through validation and cleansing processes.
• Collaborate with business stakeholders to understand data requirements and deliver.
• Create and maintain comprehensive technical documentation, including system architecture, design documents, and deployment procedures. Ensure knowledge sharing within the team.
• Implement Lean daily management and Lean continuous improvement concepts in Application development and operations.

Required Qualifications:

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
• Expertise in ETL tools like Informatica PowerCenter, Informatica Cloud ( IICS )
• Extensive experience in data engineering, with a focus on Azure cloud platform.
• Proficiency in Azure services like Azure Data Factory, Azure Databricks, Azure SQL Data Warehouse, and Azure Stream Analytics.
• Strong programming skills in languages such as Python, SQL, or Scala for data manipulation and transformation.
• Experience with big data technologies like Hadoop, Spark, or Hive is a plus.
• Familiarity with data visualization tools like Power BI or Tableau.
• Knowledge of data warehousing concepts, data lakes, and real-time data processing.
• Excellent problem-solving skills and attention to detail,
• Strong communication and leadership skills.
• Relevant certifications (e.g., Microsoft Certified: Azure Data Engineer) are a plus.

Preferred Qualifications:

• Effective Communicator: Possess the ability to articulate complex technical concepts in a clear and concise manner, facilitating understanding across a wide range of audiences.
• Solution-Oriented Mindset: Demonstrated ability to approach challenges with a solutions-first attitude, proactively identifying and addressing potential obstacles.
• Change Advocate: Capable of spearheading and implementing change, showcasing a proactive approach to driving improvements and leading teams towards new initiatives.
• Leadership Qualities: Show a strong capacity to lead, influence, and guide teams, fostering a collaborative and productive work environment.
• Technical Translator: Excel at translating and breaking down intricate technical topics, ensuring that stakeholders at all levels have a clear understanding of the subject matter.
• Analytical Thinker: Possess an analytical and detail-oriented mindset, critically evaluating information derived from multiple sources and drawing meaningful conclusions.
• Team Player: Exhibit a strong willingness to collaborate and work cohesively with colleagues, understanding the importance of collective growth and knowledge sharing.
• Adaptability: Demonstrate resilience and adaptability in an ever-changing environment, staying updated with the latest technological advancements and best practices in the field of database administration.

Desirable Qualification

• Humble: respectful, receptive, agile, eager to learn
• Transparent: shares critical information, speaks with candor, contributes constructively
• Focused: quick learner, strategically prioritizes work, committed
• Leadership ability: strong communicator, decision-maker, collaborative
• Problem solver: analytical-minded, challenges existing processes, critical thinker