The NVIDIA GPU Cloud Web Interface offers the simplest way to transfer data between DGX Cloud and local systems. For more advanced data transfer needs, particularly when working with lab directories, the NVIDIA GPU Cloud Command Line Interface (NGC CLI) provides a more robust solution. The NGC CLI can be run from:
lmm.its.albany.edudgx-head01.its.albany.eduYour personal machine (requires installation)
This guide demonstrates connecting through lmm.its.albany.edu. For instructions on running NGC CLI from your personal machine, see How-to: NVIDIA DGX Cloud. This guide also explores different ways of downloading the data, so feel free to pick the one that better suits your needs. Options range from the simplest (A) to the more elaborate (B & C).
Index
Prerequisites
Active VPN connection (if accessing from off-campus) - please refer to: VPN (Virtual Private Network)
SSH client (Terminal for macOS, PuTTY for Windows) - please refer to How-to: Connect via SSH
Downloading Workspaces
Workspaces can be dowloaded from both the web interface and the command line. From the command line, you can either use ngc workspace download or use ngc workspace mount to mount the volume and move the data back and forth.
Option A - Download Workspace from NGC Web Interface
The simplest and most straightforward one, recommended when you have a small workspace (under 10GB) and a small number of objects. Just open your workspace and click the Download button in the top right corner of the page.
Option B - Download Workspace using NGC CLI Download Command
Recommended when you have a larger workspace (above 10GB) and a small number of objects.
Step 01 - Connect to LLM Server
Connect to lmm.its.albany.edu via SSH using your NetID:
macOS: Use Terminal command
ssh <NetID>@lmm.its.albany.eduWindows: Use PuTTY
Step 02 - Navigate to Your Lab Directory
Navigate to your lab directory.
cd /network/rit/lab/<your_lab>
Then use the ngc workspace download command to download the workspace to this directory. You can either download the workspace as a folder with all the subdirectories and files or you can download it as a zip file when using the --zip flag.
# To download as a folder ngc workspace download <your_workspace> # To download as a zip file ngc workspace download --zip <your_workspace>
Option C - Download Workspace using NGC CLI Mount Command
Recommended when you have a larger workspace (above 10GB) and a larger number of objects as this option allows you to resume the synchronization in case something goes wrong. The following video provides a step-by-step walkthrough of this tutorial.
Step 01 - Connect to LLM Server
Connect to lmm.its.albany.edu via SSH using your NetID:
macOS: Use Terminal command
ssh <NetID>@lmm.its.albany.eduWindows: Use PuTTY
Step 02 - Navigate to Your Lab Directory and Mount Workspace
Navigate to your lab directory (required for proper NGC workspace mounting) and from there create a directory to mount your workspace to.
cd /network/rit/lab/<your_lab> mkdir ngc-mount
Then use the ngc workspace mount command to mount the workspace to this new directory. The --mode RW flag enables both read and write access, allowing data transfer in both directions.
ngc workspace mount <your_workspace> ./ngc-mount --mode RW
Step 03 - Transferring Your Data
Basic File Access
Access your NGC remote workspace through the
ngc-mountdirectory.Transfer files as needed between local and cloud storage.
The ngc-mount directory is only a mount point to your cloud workspace - files in this directory are not physically stored on your lab directory. To permanently store files on your lab directory, you must copy them from the ngc-mount directory to your lab parent directory or another local directory.
Syncing Directories Using rsync
For efficient directory synchronization, rsync is recommended. Here's how to sync your cloud workspace to a local backup:
Create a backup directory:
mkdir ngc-backupSync the directories with progress tracking:
rsync -r ngc-mount/ ngc-backup/ --progress
The --progress flag allows you to monitor the transfer in real-time through the terminal.
If you are not familiar with rsync there is a very useful tutorial available here.
Step 04 - Cleanup
Remember to unmount the workspace when finished.
ngc workspace unmount ngc-mount
Lastly, make sure to verify your transfer:
The
ngc-mountfolder will appear empty after unmounting.Your
ngc-backupfolder should contain all synced files.
If you have any questions on how to use ngc mount or ngc unmount, please refer to the official NVIDIA documentation on this topic.
Downloading Datasets
Datasets can be dowloaded from both the web interface and the command line using ngc dataset download.
Option A - Download Dataset from NGC Web Interface
The simplest and most straightforward one, recommended when you have a small dataset (under 10GB). Just open your dataset and click the Download button in the top right corner of the page.
Option B - Download Dataset using NGC CLI Download Command
Recommended when you have a larger dataset (above 10GB).
Step 01 - Connect to LLM Server
Connect to lmm.its.albany.edu via SSH using your NetID:
macOS: Use Terminal command
ssh <NetID>@lmm.its.albany.eduWindows: Use PuTTY
Step 02 - Navigate to Your Lab Directory
Navigate to your lab directory.
cd /network/rit/lab/<your_lab>
Then use the ngc dataset download command to download the dataset to this directory. Please note that unlike the workspace that you provided the name, for the dataset you have to provide its ID, which is conveniently located in the bottom left corner of the dataset page in the NGC web interface. You can either download the dataset as a folder with all the subdirectories and files or you can download it as a zip file when using the --zip flag.
# To download as a folder ngc dataset download <your_dataset_id> # To download as a zip file ngc dataset download --zip <your_dataset_id>