There is a very good article written by Microway on this utility. Take a look at nvidia-smi: Control Your GPUs
What is nvidia-smi?
nvidia-smi is a command line utility, based on top of the NVIDIA Management Library (NVML), intended to aid in the management and monitoring of NVIDIA GPU devices.
Installation
Do take a look at NVIDIA CUDA Installation Guide for Linux for more information
Query GPU Status
$ nvidia-smi -L
Query overall GPU usage with 1-second update intervals
$ nvidia-smi dmon
Query System/GPU Topology and NVLink
$ nvidia-smi topo --matrix
$ nvidia-smi nvlink --status
Query Details of GPU Cards
$ nvidia-smi -i 0 -q