Data Engineer

Description:

We are looking for an experienced Data Engineer to support a data migration and integration programme within a relatively low maturity Azure environment.

The role will focus on designing and building robust data pipelines, migrating data from a wide range of legacy and enterprise source systems, and establishing scalable data flows into Azure Data Lake and Databricks. A key part of the role will also involve integrating data into a proprietary data transformation solution where applicable.

Key Skills & Experience

* Azure Data Engineering

* Azure Data Factory (ADF)

* Azure Data Lake

* Databricks

* Building ETL / ELT pipelines

* Large-scale data migration and integration

* Oracle and/or Oracle Fusion data sources

* SharePoint data extraction and integration

* Working with multiple and varied source systems

* Data transformation, mapping and validation

* Designing data pipelines in environments where architecture and processes are still developing

Key Responsibilities

* Design, build and maintain data migration and ingestion pipelines within Azure.

* Support the setup and development of the Azure Data Lake environment.

* Build integrations between source systems, Azure Data Lake and Databricks.

* Develop and manage pipelines using Azure Data Factory (ADF).

* Extract and migrate data from a variety of source systems, including:

* Oracle

* Oracle Fusion

* Informatica

* SharePoint, which will be an initial priority

* Additional enterprise and legacy systems as the programme develops

* Help define scalable data ingestion, transformation and migration patterns for future source systems.

* Support data quality, validation and troubleshooting throughout the migration process.

* Work closely with technical and business stakeholders to understand source data, target structures and transformation requirements.

* Help improve the overall maturity of the Azure data environment, introducing appropriate engineering standards and best practices