Data Engineer

TESTQ TECHNOLOGIES LTD

United States

Hybrid

USD 73,000 - 79,000

Full time

2 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

TESTQ TECHNOLOGIES LTD in West Midlands is seeking an experienced Data Engineer to design, build and maintain scalable data pipelines and ETL/ELT processes across cloud platforms. You will collaborate with data architects, analysts and data scientists to deliver datasets that meet business needs, implement CI/CD, IaC, data governance and GDPR-compliant security.

Candidates should have 4+ years' experience, strong Python/SQL, and hands-on with Azure/AWS/GCP, Databricks, Spark, and modern data

Qualifications

  • Minimum 4+ years as a data engineer or in a related field.
  • Strong Python and SQL coding skills; Java/Scala desirable.
  • Experience with major cloud data platforms (Azure, AWS, GCP).
  • Experience with Databricks, Spark, Delta Lake, and data warehousing.

Responsibilities

  • Design, build and maintain scalable data pipelines and ETL/ELT processes.
  • Analyse data requirements and design data models, warehouses, lakes and lakehouse.
  • Ingest and integrate data from databases, APIs, files and streaming platforms.
  • Develop and optimise data transformations for performance and cost.
  • Implement data quality checks, monitoring and alerting across pipelines.
  • Migrate legacy platforms to modern cloud technologies.
  • Review code and mentor junior engineers on best practices.
  • Collaborate with data architects, analysts and data scientists to meet business needs.

Skills

Python
SQL
Strong communication
Stakeholder management
Data modeling
Cloud data platforms

Education

Bachelor's degree

Tools

Azure Data Factory
Azure Synapse
Azure Fabric
AWS Glue
Redshift
GCP BigQuery
Databricks
Apache Spark
Delta Lake
Snowflake
dbt
Airflow
Terraform
Git
Azure DevOps
GitHub Actions

Job description

Location: West Midlands (Hybrid/On-site)

Job Purpose:

TESTQ Technologies is an IT services and solutions company whose offerings spans over variety of industry sectors with strong technical, domain and process expertise helping clients grow their businesses and decrease operational costs on continuous basis in an ever-changing business environment.

Job Description (Main Duties and Responsibilities):
  • Design, build and maintain scalable and reliable data pipelines and ETL/ELT processes.
  • Analyse data requirements and design data models, data warehouses, data lakes and lakehouse solutions.
  • Ingest and integrate data from multiple sources including databases, APIs, files and streaming platforms.
  • Develop and optimise data transformations for performance, cost and reliability.
  • Implement data quality checks, validation, monitoring and alerting across pipelines.
  • Work with data architects, analysts and data scientists to deliver data sets that meet business needs.
  • Automate deployments using CI/CD and infrastructure as code.
  • Implement data security, access controls and governance in line with UK GDPR and company policies.
  • Troubleshoot and resolve data and pipeline issues in production environments.
  • Document data flows, data models, pipelines and operational procedures.
  • Migrate legacy data platforms and workloads to modern cloud technologies.
  • Review code and mentor junior engineers on engineering best practice.
  • Continually identify opportunities to improve platform performance and create additional value.
Key Skills, Qualifications and Experience Needed [The candidate must demonstrate these in all stages of assessment]
  • Min 4+yrs of experience as a data engineer or in a related field.
  • Strong programming skills in Python and SQL; knowledge of Scala or Java is desirable.
  • Hands-on experience with cloud data platforms such as Azure (Data Factory, Synapse, Fabric), AWS (Glue, Redshift) or GCP (BigQuery).
  • Experience with Databricks, Apache Spark/PySpark and Delta Lake.
  • Experience with data warehousing technologies such as Snowflake, Synapse or Redshift.
  • Strong knowledge of data modelling, including dimensional modelling, Data Vault and medallion architecture.
  • Experience with orchestration and transformation tools such as Apache Airflow, Azure Data Factory or dbt.
  • Experience with streaming technologies such as Kafka or Azure Event Hubs.
  • Experience with CI/CD, Git, Azure DevOps or GitHub Actions and infrastructure as code (Terraform).
  • Knowledge of relational and NoSQL databases.
  • Understanding of data quality, data governance, metadata and data lineage.
  • Good awareness of data security and UK GDPR.
  • Proven ability to manage stakeholders and SMEs to deliver results within demanding timelines.
  • Strong communication skills, being able to understand technical detail and tailor summary to the audience in a simple and clear manner - both verbally and written.
  • Cloud data engineering certification (e.g. Microsoft, AWS or Databricks) is desirable.
Other Key skills:
  • Good analytical and Problem-solving skills
  • A thorough approach and Self starter
  • Focus on quality and delivery
  • Working together in teams.
  • Leadership and effective decision making.
  • Excellent customer service
Qualifications: Bachelor's degree or above in the UK or Equivalent.

Salary: GBP 55000 to GBP 60000 per annum

Evaluation: CV Review, Technical Test, Personal and Technical Interview and References

Job Type: Full-time, Permanent [Part time and Fixed Term option is available]

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Analyst
Data Analyst

TESTQ TECHNOLOGIES LTD • United States

Hybrid
USD 46,000 - 53,000
Data Governance Manager
Data Governance Manager

TESTQ TECHNOLOGIES LTD • United States

Hybrid
USD 64,000 - 71,000
Leicester office
Hybrid Data Engineer: ETL, Lakehouse & Cloud
Hybrid Data Engineer: ETL, Lakehouse & Cloud

TESTQ TECHNOLOGIES LTD • United States

Hybrid
USD 73,000 - 79,000
Data Engineer
Data Engineer

Compunnel Inc. • Norfolk (VA)

On-site
USD 110,000 - 150,000
Software Developer
Software Developer

TESTQ TECHNOLOGIES LTD • United States

Hybrid
USD 73,000 - 79,000
Data Engineer – AWS
Data Engineer – AWS

Tronlogix • United States

Remote
USD 120,000 - 180,000
Senior Data Engineer
Senior Data Engineer

Ceox • United States

Remote
USD 92,000 - 119,000
Data Engineer
Data Engineer

Northbound Executive Search • Austin (TX)

On-site
USD 80,000 - 120,000
Data Engineer
Data Engineer

EdgeQ Technologies Private Limited • Dallas (TX)

On-site
USD 100,000 - 160,000
Data Engineer
Data Engineer

Compunnel Inc. • Harrisburg

On-site
USD 110,000 - 150,000