Data Engineer


Details:
  • Salary: £50,000 - 60,000 - Annum
  • Job Type: Permanent
  • Job Status: Full-Time
  • Salary Per: Annum
  • Location: London
  • Date: 3 weeks ago
Description:

My client is a not-for-profit organisation focused on developing the blockchain/DLT ecosystem.

Due to ongoing growth they are now looking for an experienced Data Engineer to join their team.

This role is pivotal in managing, curating, optimising, and securing datasets specifically related to cryptocurrency discussions across various platforms. The ideal candidate will be adept in web scraping, data quality assurance using AI, data integration, ensuring data security and compliance, and maintaining detailed documentation.

Blockchain experience is not needed but a solid background with Python, AWS and web scraping is a must.

Role and Responsibilities

* Data Collection: Identify relevant chat sources, groups, and forums on platforms discussing particular topics. Maintain and develop web scraping tools or APIs for periodic data extraction.

* Data Quality Assurance: Develop and implement AI-based procedures for quality control of data and data sources to eliminate inaccuracies and anomalies. Create tools for monitoring data sources for changes and updates, adapting data collection and cleaning processes accordingly.

* Data Integration: Collaborate with data scientists and analysts to integrate collected data into various projects and analysis tools. Ensure smooth data flow and integration with other data sources within the organisation.

* Data Security and Compliance: Uphold the security and privacy of collected data in compliance with relevant regulations and company policies.

* Documentation: Maintain clear and comprehensive documentation of data sources, collection methods, and workflows. Produce reports and documentation for both internal and external stakeholders as required.

* Monitoring and Reporting: Develop and maintain systems to monitor the performance and health of data collection processes.

Key skills needed;

* Bachelor’s degree in Computer Science, Data Science, or a related field.

* Demonstrable experience in a data engineering role, with a focus on web scraping and data collection.

* Proficient in programming languages such as Python, SQL.

* Strong AWS experience

* Knowledge of Data Structures and Databases

* Knowledge of data privacy laws and compliance requirements.

* Experience with Data Analytics/algorithms would be nice to have but training will be provided for the right candidate

* Excellent communication and collaboration abilities.

Desirable

* Experience with Typescript

* Experience with big data technologies and cloud services.

* Proficiency in AI and machine learning techniques for data quality assurance.

They are happy for this role to be predominately remote based with once or twice a month in the office

Report this job

By sending this message I agree to GrindJob’s Terms and Conditions and Privacy Policy.

Enter your email to get a notification when similar jobs become available.

Create a job alert for Data Engineer in London ()

By continuing, you agree to GrindJob’s T&Cs and Privacy Policy.

When applying for a job, do not provide bank account details or any other financial information.
Never make any form of payment. GrindJob is not responsible for any external website content.

Enter your email to get a notification when similar jobs become available.

Your browser does not support Cookies or JavaScript or this option is turned off in your browser settings.

How to enable Cookies and JavaScript

Your browser is out of date!

Update your browser to view this website correctly. Update my browser now

×

Please wait...
There was an error loading the page. Would you like to reload the page?