Back to Job Search

Lead DataOps Engineer (DevOps, Big Data Hadoop)

  • Location: London
  • Salary: Negotiable per day
  • Job Type:Contract

Posted 25 days ago

  • Sector: Java & Dev Ops
  • Contact: James Dryden
  • Duration: 3 months
  • Start Date: ASAP
  • Expiry Date: 03 March 2021
  • Job Ref: JN -022021-456126

I am in need of a Lead DataOps Engineer for an initial 3 month contract. This will be outside IR35 and based remotely, then in London when back in the office.

This role will be starting in March and will require you to go via SC Clearance (to be eligible you need to have been in the country at least 5 years) the client will cover the costs. I am reaching out early as the onboarding will take a while, so please respond ASAP.

DevOps leadership and engineering experience

  • 3 + years of design experience for installation of components within Hortonworks or Cloudera and to include design
  • Experience of performance tunning and performance testing Hortonworks Clusters and connected systems. AND/OR at least 1 project delivered where design of Airflow with solid HA DR were important and demonstrable components

3 + years of performance testing experience ideally using:

  • SparkBench, HiveBench, TeraGen/TeraSort/TeraValidate, · TestDFSIO,

3+ years of performance monitoring experience with:

  • Grafana, HDFS, YARN, Ambari, Linux OS,

3+ years performance tuning experience:

  • Yarn and Yarn resource allocation architecture factors affecting resource utilization of Hadoop clusters
  • Spark Job tuning/parameters
  • HIVE LLAP Queue capacity

5+ years working knowledge of :

  • Hadoop · Spark and SparkSQL · HIVE and HIVE SQL

Even more desirable skills (but not necessary) are Spark MLib, GraphX or StreamingAwareness

Please get in touch with your latest CV to find out more.