Data Engineer

Description:

ESSENTIAL ROLES & RESPONSIBILITIES

* Identify and understand customer data-centric use cases within regulated financial services environments

* Design and implement data ingestion, processing, and transformation pipelines on Azure

* Build and maintain data pipelines for cleaning, normalisation, enrichment, and preparation

* Apply appropriate data modelling techniques and architecture patterns, with a strong focus on medallion architecture

* Orchestrate, monitor, and optimise Azure Databricks jobs and Azure Data Factory pipelines across development, UAT, and production environments

* Operationalise workflows developed by analysts and data scientists

* Support customers in adopting Azure data, analytics, and machine learning services

GOVERNANCE & REPORTING

* Maintain accurate documentation of data pipelines, schemas, transformations, and deployment processes

* Support data governance initiatives including lineage, metadata management, and access control

TECHNOLOGY STACK (AZURE)

Data Engineering & Analytics:

* Azure Databricks (development, UAT, and production)

* Azure Data Factory

* Azure Synapse Analytics (where applicable)

Databases:

* Microsoft SQL Server / Azure SQL Database (primary platforms)

* PostgreSQL (limited use)

* MySQL (limited use)

Security & Governance:

* Role-based access control (RBAC)

* Data encryption and key management

* Audit logging and monitoring

CRITICAL COMPETENCIES – TECHNICAL FIT

Essential:

* Strong SQL skills

* Programming experience with Python and/or Scala

* Hands-on experience with Azure-based data platforms

* Experience designing, building, and maintaining data pipelines

* Strong understanding of data modelling (relational and analytical), including medallion architecture

* Experience orchestrating and optimising Databricks and Data Factory workloads

* Experience using CI/CD pipelines for data and analytics solutions

* Strong awareness of security, networking best practices, GDPR, and PII handling

Desirable:

* Experience with Azure Databricks in production environments

* Familiarity with Azure Machine Learning and AI services

SHIFT & WORKING PATTERN

* Standard business hours, with participation in an on-call rota as required

* Occasional weekend engineering coverage will be required, typically limited to a small number of planned weekends per year to support business continuity, resilience testing, or disaster recovery activities