Webmonit

Retention and storage

History is the product: every result and every sample is kept until its retention drops it, whole months at a time. The interface shows what each machine's months weigh (Settings › Data maintenance) and everything here is a setting — nothing is forced.

What it weighs

With the agent sampling every second, expect on the order of 12 MB per day for a simple machine, plus about 0.2 MB per day per busy hosting account on a shared server (the per-account detail is most of the weight on such machines — it is also the point). Results from outside checks weigh a few GB a month for hundreds of sites checked every few minutes.

As a rule of thumb: a fleet of a dozen machines, two of them busy shared hosts, writes around 300 MB a day of raw per-second data — nothing is averaged away to get there: every value is stored exactly, only its repetition is not. Charts over days and weeks read per-minute rows kept beside the samples, so a month opens in a fraction of a second.

What to keep

  • retention.metrics — the raw per-second samples. 30 days is comfortable; 7 days is a sane floor.
  • retention.results — the checks' answers. 30 days by default; each month weighs a few GB for hundreds of sites.
  • retention.history — the incidents, the alerts sent, the changes and the records' snapshots. Years, for little: a month at least, since they are what a dispute is settled with.
  • Availability roll-ups are tiny (bytes per site per day) and kept for years: long-term statistics survive the raw data they came from.

A retention saved in Settings › Data maintenance takes precedence over the one in the configuration file — the page shows the value in force. A month is dropped whole once its last day is past the retention, so the disk holds up to one month more than the setting.

Dropping data on purpose

Retention is the rule; the Data maintenance page is the exception, by hand: flush one machine's period (a spike an agent wrote while it was being fixed), or drop a whole month. Both are deliberate, audited, and give the disk back immediately.

Installations upgrading from an earlier version can also compact their metric months into the current storage shape from the same page — two to three times smaller, nothing lost. It runs in the background and can be asked again: what is already compact is left alone.