Your KPIs Aren't Telling You Enough
Is GitHub down if the website loads but GitHub Actions doesn’t work? Technically, no. A lot of GitHub is still available. You can browse, review things, and push a commit. But if your release process depends on Actions, a big piece of GitHub is absolutely down for you. The same thing happens with clusters. The login nodes might be available while storage is having problems. Most of the compute nodes might be healthy while one GPU partition is offline. The owners can truthfully say that the cluster is up, while a researcher can say that they can’t do their work. ...