Data Engineering Intern

Bright Network Consulting; United States; Remote; USD 90,000 - 150,000; Full time

Job summary

Bright Network Consulting is seeking a Data Engineer to design, build, and maintain scalable data pipelines and data architectures for analytics in a remote United States setting.

The ideal candidate will have strong SQL, Python or Scala skills, experience with ETL/ELT, and familiarity with cloud data services and data warehousing to support BI and reporting objectives.

Qualifications

• Strong SQL, data modelling, and processing capabilities.
• Experience building scalable data pipelines and ETL/ELT workflows.
• Proficiency in Python or Scala for data processing.
• Familiarity with cloud data services and data warehousing.
• Ability to design and optimise data architectures for BI and analytics.

Responsibilities

• Design, build, and maintain scalable data pipelines as part of analytics workflows.
• Develop and optimise ETL/ELT processes from diverse sources.
• Design data warehouses, data lakes, and lakehouses to support BI needs.
• Write efficient SQL and transform data into usable formats.
• Collaborate with analysts and data scientists to understand requirements.
• Implement data quality checks, monitoring, and error handling.

Skills

SQL proficiency

Python

PySpark

Scala

Data pipelines

ETL/ELT

Cloud platforms

Data modelling

Education

Degree in Computer Science or related field

Tools

PostgreSQL

MySQL

SQL Server

MongoDB

AWS

Azure

GCP

Snowflake

Job description

We are hiring a skilled and motivated Data Engineer on behalf of one of our clients for a remote opportunity.

The ideal candidate will have strong technical and problem-solving skills, experience in building and maintaining data pipelines, and the ability to design scalable data infrastructure that supports analytics, reporting, and business intelligence.

Job Title: Data Engineer

Location: United States

Key Responsibilities

• Design, develop, and maintain scalable data pipelines to collect, process, and transform large volumes of structured and unstructured data.
• Build and optimise ETL/ELT workflows to integrate data from multiple internal and external sources.
• Develop, test, and maintain robust data ingestion and transformation processes.
• Design and manage data warehouses, data lakes, and lakehouse architectures to support analytical requirements.
• Write efficient SQL queries and develop complex transformations to process and organise data.
• Develop data engineering solutions using Python, PySpark, Scala, or other relevant programming languages.
• Build and maintain batch processing and real-time data streaming pipelines.
• Integrate data from relational databases, APIs, cloud storage, third-party applications, and enterprise systems.
• Work with data analysts, data scientists, business intelligence developers, and software engineers to understand data requirements.
• Implement data validation, quality checks, monitoring, and error-handling mechanisms to ensure reliable data delivery.
• Optimise data pipelines and database queries to improve performance, scalability, and processing efficiency.
• Design and implement data models, including dimensional models, star schemas, and snowflake schemas.
• Manage data transformation workflows and ensure consistency across development, testing, and production environments.
• Implement workflow orchestration and scheduling to automate recurring data processing tasks.
• Monitor pipeline performance, troubleshoot failures, and resolve data processing and integration issues.
• Develop and maintain documentation for data architectures, pipelines, data models, and technical processes.
• Implement data security, access controls, encryption, and governance practices to protect sensitive information.
• Support the migration of legacy data systems to modern cloud-based data platforms.
• Implement version control, testing, and deployment practices for data engineering code and pipelines.
• Collaborate with DevOps and platform engineering teams to support automated deployment and infrastructure management.
• Ensure data availability, reliability, consistency, and accessibility for downstream analytics and reporting.
• Identify opportunities to improve data architecture, automate manual processes, and reduce infrastructure costs.
• Support real-time analytics and event-driven data processing where required.
• Troubleshoot complex data issues and perform root cause analysis to prevent recurring failures.
• Ensure data engineering solutions follow organisational standards and applicable data protection requirements.

Technical Skills

• Strong programming knowledge in Python, SQL, and preferably PySpark or Scala.
• Experience developing ETL/ELT pipelines and automated data transformation workflows.
• Knowledge of relational databases such as PostgreSQL, MySQL, SQL Server, or Oracle.
• Familiarity with NoSQL databases such as MongoDB, Cassandra, or DynamoDB.
• Experience with cloud platforms such as AWS, Microsoft Azure, or Google Cloud Platform.
• Knowledge of cloud data services such as Amazon S3, AWS Glue, Azure Data Factory, Azure Synapse Analytics, or Google Cloud Dataflow.
• Experience with data warehousing platforms such as Snowflake, Amazon Redshift, Google BigQuery, or Azure Synapse.
• Understanding of Apache Spark, Hadoop, and distributed data processing concepts.
• Familiarity with workflow orchestration tools such as Apache Airflow, Azure Data Factory, or Dagster.
• Knowledge of streaming technologies such as Apache Kafka, Amazon Kinesis, or Azure Event Hubs.
• Understanding of data modelling, schema design, partitioning, indexing, and query optimisation.
• Experience with data transformation frameworks such as dbt.
• Familiarity with Git, GitHub, GitLab, and version control practices.
• Knowledge of Docker, Kubernetes, and containerisation concepts.
• Understanding of CI/CD pipelines and automated testing for data workflows.
• Familiarity with infrastructure-as-code tools such as Terraform is an advantage.
• Knowledge of data monitoring, logging, observability, and pipeline alerting.
• Understanding of data governance, metadata management, data lineage, and data quality frameworks.

Required Skills & Qualifications

• Strong understanding of data engineering principles, database systems, and data architecture.
• Proficiency in SQL, including complex joins, CTEs, window functions, aggregations, and query optimisation.
• Hands-on experience with Python or another programming language used for data processing.
• Understanding of ETL/ELT processes, data integration, and data transformation techniques.
• Knowledge of relational databases, data warehousing, and data modelling concepts.
• Ability to build reliable, scalable, and maintainable data pipelines.
• Understanding of batch processing and distributed computing fundamentals.
• Strong troubleshooting, debugging, and analytical problem-solving abilities.
• Familiarity with data validation, testing, and quality assurance practices.
• Ability to collaborate effectively with data analysts, data scientists, developers, and other stakeholders.
• Strong documentation, communication, and organisational skills.
• Ability to work independently and manage priorities in a remote working environment.
• A degree in Computer Science, Information Technology, Data Engineering, Software Engineering, or a related field is preferred.

Preferred Qualifications

• Hands-on experience with cloud-based data engineering services across AWS, Azure, or Google Cloud.
• Experience designing modern data lake, data warehouse, and lakehouse architectures.
• Advanced knowledge of Apache Spark, PySpark, and distributed data processing.
• Experience implementing real-time streaming solutions using Kafka or similar technologies.
• Familiarity with modern data stack tools such as dbt, Airflow, Snowflake, and Databricks.
• Experience with medallion architecture and layered data processing frameworks.
• Knowledge of CDC (Change Data Capture), incremental loading, and data synchronisation techniques.
• Experience implementing data quality frameworks and automated data testing.
• Understanding of data mesh, data fabric, and modern data architecture principles.
• Familiarity with machine learning data pipelines and preparing datasets for AI and analytics applications.
• Experience with cloud security, IAM, secrets management, and secure data access.
• Knowledge of data governance, GDPR, data retention, and compliance requirements where applicable.
• Experience with CI/CD, infrastructure as code, and automated deployment of data pipelines.
• Familiarity with monitoring tools such as Grafana, Prometheus, or cloud-native observability services.
• Experience optimising cloud resource usage, query performance, and data processing costs.
• Understanding of DevOps and DataOps practices for reliable data platform delivery.
• Experience migrating legacy databases, ETL jobs, or on-premises data platforms to cloud environments.
• Familiarity with APIs, event-driven architectures, and microservices-based data integrations.
• Relevant certifications in AWS, Microsoft Azure, Google Cloud, Databricks, or Snowflake are an advantage.
• A portfolio of data engineering projects or experience building end-to-end data pipelines is a plus.