This listing was posted on iSmartRecruit.
Spark Architect
Location:
London, Gt Lon
Description:
Spark Architect / SME Context: Legacy ETL code for example Datastage is being refactored into PySpark using Prophecy low-code no-code and available converters. Converted code is causing failures/performance issues. Skills: • Spark Architecture – component understanding around Spark Data Integration (PySpark, scripting, variable setting etc.), Spark SQL, Spark Explain plans. • Spark SME – Be able to analyse Spark code failures through Spark Plans and make correcting recommendations. • Spark SME – Be able to review PySpark and Spark SQL jobs and make performance improvement recommendations. • Spark – SME Be able to understand Data Frames / Resilient Distributed Data Sets and understand any memory related problems and make corrective recommendations. • Monitoring – Be able to monitor Spark jobs using wider tools such as Grafana to see whether there are Cluster level failures. • Cloudera (CDP) – Knowledge of understanding how Cloudera Spark is set up and how the run time libraries are used by PySpark code. • Prophecy – High level understanding of Low-Code No-Code prophecy set up and its use to generate PySpark code.
Posted:
February 3 on iSmartRecruit
Visit Our Partner Website
This listing was posted on another website. Click here to open: Go to iSmartRecruit
Important Safety Tips
- Always meet the employer in person.
- Avoid sharing sensitive personal and financial information.
- Avoid employment offers that require a deposit or investment.
To learn more, visit the Safety Center or click here to report this listing.
More About this Listing: Spark Architect
Spark Architect is a Architecture & Design Architect Job located in London LND. Find other listings like Spark Architect by searching Oodle for Architecture & Design Architect Jobs.
Spark Architect is a Architecture & Design Architect Job located in London LND. Find other listings like Spark Architect by searching Oodle for Architecture & Design Architect Jobs.