Data Engineer

Different Technologies Pty Ltd.; Greater London; Hybrid; GBP 70,000 - 100,000; Full time

Benefits offered by this job

25 days annual leave

Private health insurance

Training budget

5% pension

Employee referral programme

Gym discounts

Job summary

Daintta is a growing data engineering consultancy partnering with government clients to deliver data-driven insights. You will develop pipelines for analytics, ML and business intelligence in a security-conscious environment.

The role requires 3+ years in data engineering, strong experience with Spark, Python, Kafka, SQL and Airflow, and a willingness to work hybrid in London or Cheltenham. You will contribute to client engagements and growth of the company.

Qualifications

• 3+ years of degree-level industry experience in data engineering
• Experience in client delivery across projects involving data analysis, ETL/ELT, data intelligence and data security
• Experience with cloud-based infrastructure (AWS, Azure, GCP) and data engineering tools
• Demonstrable data quality checks, data warehousing schemas and CI/CD in pipelines
• Ability to work in a security-conscious environment and understand data privacy requirements

Responsibilities

• Assess clients' technical needs and manage stakeholders
• Identify data sources and design ETL/ELT concepts
• Design on-premise, cloud, and hybrid data engineering solutions
• Model, structure and store data for analytics, ML, BI and compliance
• Ingest structured and unstructured data from diverse formats
• Ensure data security, storage compliance and data governance
• Support client engagements, pitches and company strategy

Skills

Client delivery

ETL/ELT

Data pipelines

Security clearance

CI/CD tooling

Education

Degree in data engineering or related field

Tools

Spark

Python

Kafka

SQL

Airflow

PostgreSQL

Elasticsearch

Job description

Daintta are a rapidly growing, values-driven team of specialists who work with government clients across Cyber, Telecommunications and Data. We are seeking a talented and motivated Data Engineer to join our team and contribute to our mission of protecting the UK through data-driven insights and solutions. As a Data Engineer, you will work closely with our public sector clients and project teams to develop and implement data engineering pipelines to support data science and business/mission intelligence use cases.

What's in it for you?

In addition to being rewarded fairly for your contribution to the business, you get to work in a dynamic organisation that is agile and responsive. A business that is growing fast and where you get to drive and shape the future. A place where you are respected by everyone and your voice is important. Somewhere where you can be innovative and creative. A place where you have the opportunity to learn about all aspects of business from marketing to sales, to delivery and business operations.

Key Responsibilities

• Assessing your clients' technical needs and understanding how their needs are different to wants and managing clients' stakeholders relationship appropriately
• Identifying data sources, data extraction, transformation, and loading (ETL/ELT) concepts and methods
• Designing, evaluating, and implementing on-premise, cloud-based and hybrid data engineering solutions (including providing review and guidance on testing aspects related to data engineering, identification of risks and proposing and implementing their mitigations)
• Modelling, structuring and storing data along with their data flows for uses including - but not limited to - analytics, machine learning, data mining, compliance, business intelligence, sharing with applications and organisations
• Harvesting and ingesting structured and unstructured data in different formats and standards
• Integrating, consolidating and cleansing data
• Migrating and converting data
• Applying ethical principles in handling data
• Ensuring appropriate storage of data in line with relevant legislation
• Building in security, compliance, scalability, efficiency, reliability, fidelity, flexibility and portability
• Accurately delivering high quality work to agreed timelines and taking the initiative and knowing how to 'jump straight in'
• Supporting client engagements, including pitches and presentations
• Helping to support & grow Daintta by actively inputting into the company strategy and helping to shape our future
• Representing us and our core values: Transparent, fair and daring

Skills/Knowledge

• You have 3+ years of degree level industry experience in data engineering, preferably in a consultancy or industry setting
• You have worked in client delivery across a range of projects, including data analysis, extraction, transformation, and loading, data intelligence, data security and proven experience in their technologies (e.g. Spark, cloud-based ETL services, Python, Kafka, SQL, Airflow)
• You have experience in assessing the relevant data quality issues based on data sources & uses cases, and can integrate the relevant data quality checks into data pipelines
• You have experience in designing schemas for data warehouses/data marts to support the relevant ingest and queries
• You have experience working on cloud-based infrastructure (e.g. AWS, Azure, GCP)
• You have demonstrable continuous personal development with relevant data certifications and accreditations
• You have strong interpersonal skills
• You are security cleared or capable and willing to undergo security clearance
• You have experience with using CI/CD tooling to analyse, build, test and deploy your pipelines and proven experience in their technologies
• You have experience in database technologies including writing complex queries against their (relational and non-relational) data stores (e.g. Postgres, Hadoop, Elasticsearch, Graph databases), and designing the database schemas to support those queries
• You have a good understanding of coding best practices and design patterns and experience with code and data versioning, dependency management, code quality and optimisation, error handling, logging, monitoring, validation and alerting

Location?

Hybrid, with 2-3 days working from Daintta office (London or Cheltenham) or on client site as required.

Benefits

Time Off:

• 25 days annual leave, plus bank holidays
• Up to 5 days annual training leave with a dedicated training budget
• Up to 3 days annual volunteering leave - give back to the community
• Competitive maternity, paternity, shared parental leave & compassionate leave

Health & Wellness:

• Comprehensive Private Health Insurance
• Employee Assistance Programme - 24/7 support services
• £2,000 Flex Cash Allowance, paid pro-rata over the year

Financial Benefits:

• 5% pension contribution
• Discretionary company awards and bonuses, based on performance and company targets

Professional Development

• Up to £1,000 annual training budget (access to additional training and development budget via business case)
• Up to 5 days annual training leave
• 1 professional membership paid annually
• Up to £200 of additional IT budget for new joiners

Perks

• Free breakfast every Tuesday in the London office
• Fortnightly drinks - in London
• Regular social events, quizzes, and guest workshops
• Huckletree perks - including gym and restaurant discounts
• Employee referral programme
• Monthly breakfast club in our Cheltenham office

Security Information

Due to clients' requirements, all applications are strictly subject to security clearance requirements relevant to the role.