Moving DGX Cloud Data to Local Lab Directory

The NVIDIA GPU Cloud Command Line Interface (NGC CLI) provides the simplest way to transfer data between DGX Cloud and local systems. You can run NGC CLI from:

lmm.its.albany.edu
dgx-head01.its.albany.edu
Your personal machine (requires installation)

This guide demonstrates connecting through lmm.its.albany.edu. For instructions on running NGC CLI from your personal machine, see How-to: NVIDIA DGX Cloud.

The following video provides a step-by-step walkthrough of this tutorial.

Moving DGX Cloid Data to Local Labs.mp4

Prerequisites

Active VPN connection (if accessing from off-campus) - please refer to: VPN (Virtual Private Network)
SSH client (Terminal for macOS, PuTTY for Windows) - please refer to How-to: Connect via SSH

Step 01 - Connect to LLM Server

Connect to lmm.its.albany.edu via SSH using your NetID:

macOS: Use Terminal command ssh <NetID>@lmm.its.albany.edu
Windows: Use PuTTY

Step 02 - Navigate to Your Lab Directory and Mount Workspace

Navigate to your lab directory (required for proper NGC workspace mounting) and from there create a directory to mount your workspace to.

cd /network/rit/lab/<your_lab>
mkdir ngc-mount

Then use the ngc workspace mount command to mount the workspace to this new directory. The --mode RW flag enables both read and write access, allowing data transfer in both directions.

ngc workspace mount <your_workspace> ./ngc-mount --mode RW

Step 03 - Transferring Your Data

Basic File Access

Access your NGC remote workspace through the ngc-mount directory.
Transfer files as needed between local and cloud storage.

The ngc-mount directory is only a mount point to your cloud workspace - files in this directory are not physically stored on your lab directory. To permanently store files on your lab directory, you must copy them from the ngc-mount directory to your lab parent directory or another local directory.

Syncing Directories Using rsync

For efficient directory synchronization, rsync is recommended. Here's how to sync your cloud workspace to a local backup:

Create a backup directory: mkdir ngc-backup
Sync the directories with progress tracking: rsync -r ngc-mount/ ngc-backup/ --progress

The --progress flag allows you to monitor the transfer in real-time through the terminal.

If you are not familiar with rsync there is a very useful tutorial available here.

Step 04 - Cleanup

Remember to unmount the workspace when finished.

ngc workspace unmount ngc-mount

Lastly, make sure to verify your transfer:

The ngc-mount folder will appear empty after unmounting.
Your ngc-backup folder should contain all synced files.

If you have any questions on how to use ngc mount or ngc unmount, please refer to the official NVIDIA documentation on this topic.