Rechecking site state to avoid false positives

Published: Alex Shashenko 2023-03-06 all articles | Glossary | FAQ

Possible outcomes of new state detectionHostTracker monitoring system checks sites according to given policy in monitoring configuration. The detected status, UP or DOWN, could be local or temporal, and, therefore, is the subject for verification and confirmation before the next action, an alert or uptime statistics update, takes place. 

Examples of errors such as downtime due to new deployment and configuration propagation (DNS unavailability), local site blacklistting, incompatible environment of checking agent to server technologies, are verified with rechecks from locations of configured geo location.

HostTracker supports the following recheck strategies:

  1. Vote of majority defines if new status is valid. On new status (UP/DOWN), the system allocates up to 7 new locations from specified geo region of monitoring to confirm it. Returned results from these locations are grouped by status and UP/DOWN state is decided by the larger group of agents. Example: if New York, Boston, Chicago returned status UP, DOWN, UP as result of recheck of possible DOWN from Miami, this DOWN is unconfirmed and discarded. Miami location will be penalized and will not be taken for checks of the monitored site. Alternativelly, if Miami originally returned UP, New York an Chicago confirms this UP, and it is treated as new status, the DOWN-UP transition is registered. 
  2. Specified number of locations confirm DOWN status. In this case, you can setup in configuration what minimal number of rechecks could confirm Down(Up would still require majority vote). In previous example, if this settings is set to 1, the fact that Boston returned DOWN would be confirmation of DOWN from Miami. 
  3. No recheck, obtained new status is always confirmed. This scenario assumes complete trust on server, and, moreover, monitoring location environments. It make sense for very specific demands of monitoring but should be considered carefully if you want to avoid false positives. 

Obviously, proposed list of strategies is not full and subject for further extensions. If your monitoring task have specific demand, do not hesitate to write to our support or chat and we could consider the strategies in next releases.

 

Chief Communications and Technology Officer of HostTracker. Alex has been part of the team since the company's early days. His work focuses on business reporting, analysis of database statistics, and system administration. Alex also takes care of communication with the development team and clients.