Data Engineer - Infosys Consulting - Remote

The Remote Job Journal; United Kingdom; Remote; GBP 45,000 - 70,000; Full time

Job summary

A leading company is seeking a skilled Data Engineer to design and maintain data pipelines and architectures. The role involves collaboration with data scientists and stakeholders to ensure data integrity and support data-driven initiatives. Candidates should have a strong background in SQL, cloud services, and data integration.

Benefits

Industry-leading compensation and benefits

Top training and development opportunities

Inclusive and entrepreneurial culture

Qualifications

• Proven experience as a Data Engineer or similar role in large scale data platform implementation.
• Minimum 5 years of hands-on experience with ETL tools.
• Strong experience in SQL and relational database systems.

Responsibilities

• Design and build scalable data pipelines for large data sets.
• Integrate data from different source systems into the data lake.
• Monitor data pipeline performance and troubleshoot issues.

Skills

SQL

Python

Data Integration

ETL

Data Governance

Cloud Services

Education

Bachelor’s degree in Computer Science

Tools

Apache Nifi

Talend

AWS

Azure

Google Cloud

Job description

Description

Do you want to boost your career and collaborate with expert, talented colleagues to solve and deliver against our clients' most important challenges? We are growing and are looking for people to join our team. You'll be part of an entrepreneurial, high-growth environment of 300,000 employees. Our dynamic organization allows you to work across functional business pillars, contributing your ideas, experiences, diverse thinking, and a strong mindset. Are you ready?

Job Overview:

We are looking for a skilled and experienced Data Engineer to join our data team. The Data Engineer will be responsible for designing, building, and maintaining scalable and efficient data pipelines and architectures that support various data-driven initiatives. The ideal candidate will have strong technical expertise in data processing, data storage, and data integration, and will work closely with data scientists, analysts, and other stakeholders to ensure the integrity, availability, and reliability of the data.

Key Responsibilities:

1. Design and Build Data Pipelines:

• Develop, construct, test, and maintain scalable data pipelines for ingestion, processing, and storage of large data sets from multiple sources.
• Optimize data architectures for both structured and unstructured data to support batch and real-time data processing.

Data Integration and Management:

• Integrate data from different source systems (APIs, databases, cloud services) into the data lake or data warehouse.
• Implement and manage ETL (Extract, Transform, Load) processes for efficient data movement, cleaning, and transformation.
• Ensure data quality, consistency, and integrity through proper validation and testing.

Database and Data Warehousing Solutions:

• Design and implement database solutions that support data storage, transformation, and querying.
• Build and maintain data warehouse solutions to facilitate business intelligence and analytics needs.

Cloud Infrastructure:

• Work with cloud platforms such as AWS, Azure, or Google Cloud to implement data storage, processing, and streaming architectures.
• Set up and manage cloud-based data infrastructure, ensuring scalability, security, and performance optimization.

Collaboration and Stakeholder Management:

• Collaborate with data scientists, analysts, software engineers, and business teams to understand data requirements and deliver solutions that meet their needs.
• Document data flows, schemas, and pipeline designs to ensure transparency and understanding across the team.

Data Governance and Security:

• Implement data governance standards, data security, and compliance measures to protect sensitive data.
• Ensure compliance with data privacy regulations such as GDPR, CCPA, etc.
• Define and maintain data retention policies and ensure adherence to best practices for data management.

Performance Monitoring and Optimization:

• Monitor data pipeline performance, identify bottlenecks, and troubleshoot issues as they arise.
• Continuously improve data processes to ensure they are efficient and scalable for growing data sets.

Automation and Tools Development:

• Automate repetitive tasks and processes to streamline data engineering workflows.
• Develop tools and frameworks for data access, management, and analysis, enabling the business to easily work with data.

Requirements

Skills and Qualifications:

Essential Skills:

• Bachelor’s degree in Computer Science, Engineering, or a related field.
• Proven experience as a Data Engineer, Software Engineer, or similar role in large scale data platform implementation (project team size of ≥ 10 FTEs and/or project duration 6 months or more).
• Strong experience in SQL and relational database systems (e.g., MySQL, PostgreSQL, Oracle).
• Proficiency in programming languages such as Python, Java, or Scala for data manipulation and ETL tasks.
• Minimum 5 years of hands-on experience with ETL tools like Apache Nifi, Talend, Informatica, or equivalent.
• Familiarity with big data technologies such as Hadoop, Spark, Kafka, and HBase.
• Minimum 3 years of hands-on experience with cloud-based data services (e.g., AWS S3, Azure Data Lake, Google BigQuery, Redshift).
• Knowledge of data modelling, database design, and data architecture best practices.
• Experience with version control (e.g., Git) and agile development practices.

Preferred Skills:

• Experience with NoSQL databases like MongoDB, Cassandra, or DynamoDB.
• Experience with Databricks and Dataiku.
• Familiarity with machine learning pipelines and integration with data systems.
• Knowledge of CI/CD pipelines and automation tools like Jenkins, Docker, or Kubernetes.
• Exposure to real-time data processing tools such as Apache Flink or Storm.
• Understanding of data privacy regulations (GDPR, CCPA, HIPAA) and how they impact data storage and processing.

Personal Attributes:

• Strong problem-solving skills and ability to work independently or as part of a team.
• Excellent communication skills, with the ability to explain technical concepts to non-technical stakeholders.
• Attention to detail and a focus on data accuracy and quality.
• Proactive mindset with a passion for innovation and staying up-to-date with emerging technologies.

Given that this is just a short snapshot of the role, we encourage you to apply even if you don't meet all the requirements listed above. We are looking for individuals who strive to make an impact and are eager to learn. If this sounds like you and you feel you have the skills and experience required, then please apply now.

Benefits

About your team

Join our growing Data & Analytics practice and make a difference.

In this practice, you will be utilizing the most innovative technological solutions in the modern data ecosystem. In this role, you’ll be able to see your ideas transform into breakthrough results in Data & Analytics strategy, Management & Governance, Data Integration & Engineering, Analytics & Data Science.

About Infosys Consulting

Be part of a globally renowned management consulting firm at the forefront of industry disruption and technology. We work with market-leading brands across sectors. Our culture is inclusive and entrepreneurial. Being a mid-size consultancy within Infosys gives us the global reach to partner with our clients throughout their transformation journey.

Our core values, IC-LIFE, stand for Inclusion, Equity, Diversity, Client, Leadership, Integrity, Fairness, and Excellence. To learn more about Infosys Consulting and our values, please visit our careers page.

Within Europe, we are recognized as one of the UK’s top firms by the Financial Times and Forbes due to our client innovations, cultural diversity, and dedicated training and career paths. Infosys is on Germany’s top employers list for 2023. Management Consulting Magazine named us one of the Best Firms to Work For. Furthermore, Infosys has been recognized by the Top Employers Institute for its employee standards across Europe for five consecutive years.

We offer industry-leading compensation and benefits, along with top training and development opportunities to grow your career and achieve your personal ambitions. Curious to learn more? We’d love to hear from you... Apply today!