Data Engineer

Description:

Digital with Purpose.

DWP Digital is seeking a Data Engineer who is passionate about utilising cutting-edge technologies to deliver impactful data products.

DWP is the UK's largest department, every year we pay over £195 billion to over 20 million citizens.

DWP services cover everything from helping children & families right through to supporting relatives of lost loved ones.

We help people find jobs, and support people financially when they are unable to work due to health conditions and make ensure everybody has the money they need to live during their retirement years.

The scale of what we do is outstanding, and our purpose is unique. We'd love you to join us.

Your role:

You'll be part of our DataWorks team. We deliver proactive, innovative, and insightful data products in an Agile environment.

DataWorks is an AWS platform that provides Universal Credit data to data scientists and analysts for reporting and statistical purposes.

The team is currently working on two key remits that involve technically standardising the platform to be DWP digitally compliant and providing users with the data they need to evaluate and improve the Universal Credit service's performance.

You'll play a vital role in code development activities for the delivery of data transformation products.

You'll decide the right approach, and integrate different data sources. You'll make sure that appropriate data quality, integrity and alerting is built into all solutions.

You'll conduct proof of concept and discovery work and evaluate proposed changes that might impact data performance, models, or reports.

You will also coach and mentor more junior colleagues, helping to build data engineering knowledge and capability within the team and across government.

Essential criteria:

Experience of developing, testing, maintaining, deploying, and supporting efficient, scalable, automated ETL pipelines in an AWS cloud environment using AWS S3 Cloud Object Storage
Strong coding skills using Python, PySpark, Hive SQL and Spark SQL
Experience of working with a wide variety of structured and unstructured dataIt's also great if you have the following (if not, don't worry, you'll have lots of opportunity to learn skills like this here):

Knowledge or experience of the following tools: Hue, GitHub/Lab, Azkaban (or similar scheduling tools), CI/CD (Concourse or similar), R-Studio, Juypter-Lab, Kafka and BashDetails. Wages. Perks.

You'll join us based at one of our brilliant digital hubs in either Blackpool, Leeds, Newcastle or Sheffield.

We also have all the tools and tech we need to enable our people to work flexibly. We operate a hybrid working model with a mix of home working and some time in the office.

In return for your skills, we offer a competitive salary of up to £43,347. There is also a Digital Allowance of up to £15,000 per annum available for exceptional candidates.

You'll be eligible for a brilliant civil service pension with employer contributions worth 28.97% (worth up to £12,557 a year).

You'll get a generous leave package starting at 26 days, plus all the usual bank holidays, and the option to take extra days off on flexi leave.

We also have a broad benefits package built around your work-life balance which includes:

Flexible working, flexi hours and flexi days
Hybrid working: some time in a hub with the team, some time at home
Family-friendly policies
Time off volunteering and charitable giving
Bring your authentic self to work with 'I Can Be Me in DWP'
Discounts and savings on shopping, fun days out and more
Interest-free loans to buy a bike or a season ticket
Sports and social activities
Working in an award-winning environment and culture
Professional development, coaching, mentoring and career progression opportunitiesClick 'Apply' for more information and to start an application on Civil Service Jobs.

Please note, this role requires you to pass Security Check clearance