Sentinel Performance Issues

Sentinel Too Slow

Problem: Sentinel is too slow.

Description: Sentinel is much slower than usual.

Cause: Sentinel has many tables containing thousands of rows of data. From time to time these tables need to be re-indexed for optimal database performance.

Resolution: The System Administrator should run the optimisation script, either as required or as part of a scheduled maintenance plan. 

Note: It is strongly recommended that you regularly optimise the Sentinel database. Either routinely submit the re-indexing script, or add the script as a regular database maintenance task.

Alternative resolutions: There are a number of different optimisation possibilities within Sentinel. 


Contact Lookup Taking Too Long

Problem: The contact list is taking too long to load.

Description: When adding a list of contacts to an email, SMS or SMS via Web Service action, the list of available contacts takes a long time to load.

Cause: Sentinel is searching through the whole of the Active Directory. If this is a very large directory, the search is going through all nodes of the Active Directory server.

Resolution: Update the ActiveServerDirectory setting in the Sentinel Configuration file. You need to add the node information that is relevant to users. For example, LDAP://adserver/CN=Users,DC=example,DC=com. This will limit the search to the Users node in the domain example.com.


Monitors Keep Timing Out

Problem: Sentinel monitors time out.

Description: Sentinel monitors time out and processing stops.

Cause: Sentinel is encountering bad data and cannot continue processing. There are many scenarios that may result in bad data being stored by a system.  In some scenarios these systems can recover at a later time, and then back fill the missing data. Sentinel is watching this data in near real time, so if the data is not available at the time that it runs Sentinel needs to make a choice of running and processing the live data that has now been returned, or stopping and waiting for all of the data to be backfilled before it continues. Waiting for the data will mean possible new ‘Live’ events may be missed, so by default Sentinel will mark the data period as bad and will continue processing when live data returns.

Resolution: Sentinel monitors can be configured to wait for the data to be come available again. This is achieved by pausing when bad data is detected, and rechecking a data source a defined number of times at specified intervals, waiting for it to be present. When the data is determined to be good again, Sentinel will process the newly acquired data and generate historical events until it catches back up to the live data.

There are two monitor configuration parameters available in the SentinelConfig.xml file.

  • BadDataRetryPeriodMinutes: The amount of time to wait between collection attempts.
  • MaximumBadDataRetryCount: The number of times a monitor will keep trying to collect data after bad data has been detected. 

This combination will enable you to set how long to wait for the data to return. If this time period is exceeded, Sentinel will default back to its previous workflow and will skip each processing interval until live data returns.

 

Comments are closed