Immutable Backup - Disk Monitoring

[matthew.evans]

Hello All,

I hope this is the right place to ask this, but when using an immutable backup repository how has everyone else approached monitoring disk health?
I have a Dell R740 with a MD1400 for storage, the iDrac is disconnected and SSH is off on the host. So its not accessible.

While we use monitoring over SNMP for other repos obviously there isn't any external access to the iDrac for checking disk health. Someone has to routinely go on site and check this repo has no failed drives or issues.

In the spirit of further improving this process I was curious if anyone had approached this differently and what issues have you faced?

Thanks
Matt

[Gostev]

Hi, Matthew. This may make you smile but someone came up with mounting a web cam in front of the server to monitor the HDD lights through it, and since then it is becoming the gold standard. Thanks!

[matthew.evans]

Funnily enough that was a suggestion my colleague came up with

[jorgedlcruz]

Hello Matthew,
I would keep everything like that, but have a monitoring agent installed (telegraf, opentelemetry, you name it), for example telegraf has IPMI monitor meaning that is the only host which access the iDrac, collects the data, and telegraf send traffic out to your favourite DB (InfluxDB, SQL, PostgreSQL). So you need to open one port only OUT with the Db destination port, example InfluxDB is 8086.

More advanced could be having all the IPMI devices of the Datacenter into a separate switch, no access to Internet whatsoever, and have a RPi, or some small Linux device with 2 ETH, querying all of those in one range, and sending the metrics with the other eth.

Plenty of options.