Nutanix Task Monitoring — MS Teams

Nutanix clusters do a very good job at monitoring themselves and sending alerts if something goes awry. But I wanted to monitor some long-running operations a bit more closely (LCM in particular) and decided to look into API calls and sending output to Microsoft Teams.

The script does the following:

• connects to specified cluster
• gathers recent tasks
• calculates overall disk percentage used — of particular concern during LCM updates
• sends message to Microsoft Teams channel via their API
• sleeps for 60 seconds and repeats

If you have a Nutanix cluster with disk used above 60% you could run into space issues depending on how long the LCM updates take to complete. This was the real driver for this script, so I could monitor update cycles from my mobile device without needing a VPN connection and access to Prism (only metadata is sent to Teams).

Please feel free to use this code as you see fit, and be gentle. I am NOT a skilled coder, I merely know how to cobble things together until they start working.

Enjoy!

Teams Message Example

The code:

#
# query Nutanix clusters for tasks - send to Teams for monitoring progress
#
Param(
    [Parameter(Mandatory=$true)]$cluster
)
# you will need to setup your Teams channel for API access
$teamsurl = 'https://URL'
function SendTeams { 
 # use the Nutanix icon from Twitter just to be fancy - no big deal if it does not work
 Param(
  [Parameter(Mandatory=$true)][string]$message,
  [Parameter(Mandatory=$true)][string]$cluster
 )
 $body = ConvertTo-Json -Depth 8 @{
  "@type" = "MessageCard"
  "@context" = "<http://schema.org/extensions"
  "title" = "$cluster tasks"
  "summary"  = "$message"
  "sections" = @(
   @{
    "activityText" = $message
    "activityImage" = "https://pbs.twimg.com/profile_images/1280565180792004608/wzUxxRu0_400x400.jpg"
   }
  )
 }
 $rv = Invoke-RestMethod -uri $teamsurl -Method Post -body $body -ContentType 'application/json'
}
$cred = get-credential -message "Nutanix credentials"
$username = $cred.UserName
$password = $cred.GetNetworkCredential().Password
$ntnxcred = "${username}:${password}"
$bytes = [System.Text.Encoding]::ASCII.GetBytes($ntnxcred)
$base64 = [System.Convert]::ToBase64String($bytes)
$basicAuthValue = "Basic $base64"
$ntnxheaders = @{ Authorization = $basicAuthValue }
# disable certificate checking
if ("TrustAllCertsPolicy" -as [type]) {} else {
    Add-Type "using System.Net;using System.Security.Cryptography.X509Certificates;public class TrustAllCertsPolicy : ICertificatePolicy {public bool CheckValidationResult(ServicePoint srvPoint, X509Certificate certificate, WebRequest request, int certificateProblem) {return true;}}"
    [System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
}
# force TLSv1.2
[Net.ServicePointManager]::SecurityProtocol = "tls12"
function get-ntnx-disk ($cluster) { # retrieve disk usage for specified cluster
 $uri='https://' + $cluster + ':9440/PrismGateway/services/rest/v1/storage_pools/'
 try {
  $stats = Invoke-RestMethod -uri $uri -contenttype "application/json" -Headers $ntnxheaders -method GET -TimeoutSec 30
  return [math]::round(($stats.entities.usageStats.'storage.usage_bytes'/1TB) / ($stats.entities.usageStats.'storage.capacity_bytes'/1TB) * 100, 2)
 } catch {
  return -1
 }
}
# connect to requested cluster and obtain 100 recent tasks
$uri='https://' + $cluster + ':9440/PrismGateway/services/rest/v2.0/tasks/list'
$postparams = @{ count="100"; include_completed="true"; include_subtasks_info="true" }
$postparams = $postparams | convertto-json
while ($true ) { # continue looping until killed
 $diskusage = get-ntnx-disk $cluster
 try {
  $tasks = Invoke-RestMethod -uri $uri -Headers $ntnxheaders -method POST -TimeoutSec 30 -contenttype "application/json" -body $postparams
  $tasks = $tasks.entities    # Nutanix response from REST API
 } catch {
  write-error "problem connecting to $cluster"
 }
 # now change time to local and deal with milliseconds
 $formatted = $tasks | select-object operation_type,percentage_complete,progress_status,@{N='updated';E={([datetimeoffset]::FromUnixTimeMilliseconds($_.last_updated_time_usecs/1000).localdatetime)}}
# and limit to tasks updated in last minute
 $msg = $formatted | where-object {$_.updated -gt (get-date).addminutes(-1)}
 $msg # send to screen
 $string = $msg | format-list | out-string
 if ( $string.length -gt 0 ) {
  $string = "$string`ndiskUsage : $diskusage %"
  SendTeams -message $string -cluster $cluster
 }
 start-sleep 60
}