Skip to main content
ExLibris
  • Subscribe by RSS
  • ExLibris Dev

    Primo Central - RCA - October 18, 2016

    Confidential Information, Disclaimer and Trade Marks

    Introduction

    This document serves as a Root Cause Analysis for the Primo Central service interruption experienced by Ex Libris customers on October 18,2016.

     

    The goal of this document is to share our findings regarding the event, specify the root cause analysis, outline actions to be taken to solve the downtime event, as well as preventive measures Ex Libris is taking to avoid similar cases in future.

    Event Timeline

    Service interruption was experienced by Ex Libris customers served by the Primo Central instance at the North America Data Center during the following hours: 

     

    October 18, 2016 from 8:54 AM until 9:30 AM Chicago time zone

     

    During the event, the service was unavailable for the environment.

    Root Cause Analysis

    Ex Libris Engineers investigated this event to determine the root cause analysis with the following results:

    An ongoing repeating process had been identified to cause an abnormal load on the system, causing a complete system down.

    The process is running on an ongoing basis and its purpose is rebalancing of the information between the multiple database components.

    We were able to identify that the job is causing a load on the system, that in specific situations may cause a complete service disruption.

    Technical Action Items and Preventive Measures

    Ex Libris has taken the following action and preventive measures to avoid such an occurrence in future:

    • Once Identified, the process had been stopped and the system returned to it's normal behavior.
    • Following the system disruption, the ongoing process had been analyzed and had been updated to run in a manner that will have no impact on the system.

    Customer Communication

    ExLibris is committed to providing customers with prompt and ongoing updates during Cloud events. Ongoing and prompt updates on service interruptions appear in the system status portal at this address: http://status.exlibrisgroup.com/

    • Was this article helpful?
    //doorbell.io feedback widged