Back to Job Search

Data Engineer

  • Location: Edinburgh
  • £ Salary: Outside IR35 per day
  • Job Type:Contract

Posted over 2 years ago

  • Sector: Transformation
  • Contact: James Taylor
  • Duration: 6 months
  • Start Date: ASAP
  • Expiry Date: 27 November 2021
  • Job Ref: JN -102021-465079

Hydrogen Group are currently looking for several Data Engineers to join an exciting programme of work that is been carried out by a consultancy for a challenger bank. The role is a contract role that sits outside IR35 and is paying up to £500 per day. The role is a remote role.

The successful Data Engineer will be Working as part of a team of other Data Engineers, the successful Data Engineer will work closely with the project Data Modeller and Data Architect to build out the Enterprise Data Model in Data Warehouse in an RDS using EMR, Spark and Redshift. Additionally, building out real-time streaming capability from ingestion through to reporting, BI & data marts and presenting data for API consumption, all built in and deployed through multiple AWS environments.

Essential Key skills/knowledge/experience:

  • Minimum 3 years' working with AWS tools such as Lambda, S3, Apache Spark, AWS Data Pipeline, EMR Redshift or other RDS
  • Minimum 3 years' experience of using a mixture of tools/languages/concepts (SQL, python, Spark, data lakes, APIs, Lambda, data tiering, normalisation, data schemas)
  • There will be work with numerous other DevOps tools/languages used such as SQL, python, Spark, Lambda. It is essential that the successful candidate can work with Cloud concepts such as data lakes, data tiering, normalisation, data schemas, and real-time data e.g. DMS/Kafka/Kinesis etc
  • Working in an agile environment/ waterfall or sprint-based approach

Desirable skills/experience:

  • Ideally have some awareness of the information security/data protection/governance and change management involved in working in a bank.
  • Real-time data streaming: 1 year+ experience. The requirements of the data platform call for experience with real-time data streaming in AWS, thus experience of this will be beneficial

...