The NVIDIA GPU Cloud Web Interface offers the simplest way to transfer data between DGX Cloud and local systems. For more advanced data transfer needs, particularly when working with lab directories, the NVIDIA GPU Cloud Command Line Interface (NGC CLI) provides a more robust solution. The NGC CLI can be run from:
lmm.its.albany.edu
dgx-head01.its.albany.edu
Your personal machine (requires installation)
This guide demonstrates connecting through |
This guide explores multiple approaches to downloading data (workspaces and datasets), allowing you to choose the method that best fits your requirements. The options are presented in order of complexity, from the straightforward approach (A) to more sophisticated solutions (B & C).
Active VPN connection (if accessing from off-campus) - please refer to: VPN (Virtual Private Network)
SSH client (Terminal for macOS, PuTTY for Windows) - please refer to How-to: Connect via SSH
Workspaces can be dowloaded from both the web interface and the command line. From the command line, you can either use ngc workspace download or use ngc workspace mount to mount the volume and move the data back and forth.
The simplest and most straightforward one, recommended when you have a small workspace (under 10GB) and a small number of objects. Just open your workspace and click the Download button in the top right corner of the page.
Recommended when you have a larger workspace (above 10GB) and a small number of objects.
Connect to lmm.its.albany.edu via SSH using your NetID:
macOS: Use Terminal command ssh <NetID>@lmm.its.albany.edu
Windows: Use PuTTY
Navigate to your lab directory.
cd /network/rit/lab/<your_lab> |
Then use the ngc workspace download command to download the workspace to this directory. You can either download the workspace as a folder with all the subdirectories and files or you can download it as a zip file when using the --zip flag.
# To download as a folder ngc workspace download <your_workspace> # To download as a zip file ngc workspace download --zip <your_workspace> |
Recommended when you have a larger workspace (above 10GB) and a larger number of objects as this option allows you to resume the synchronization in case something goes wrong. The following video provides a step-by-step walkthrough of this tutorial.
Moving DGX Cloid Data to Local Labs.mp4Connect to lmm.its.albany.edu via SSH using your NetID:
macOS: Use Terminal command ssh <NetID>@lmm.its.albany.edu
Windows: Use PuTTY
Navigate to your lab directory (required for proper NGC workspace mounting) and from there create a directory to mount your workspace to.
cd /network/rit/lab/<your_lab> mkdir ngc-mount |
Then use the ngc workspace mount command to mount the workspace to this new directory. The --mode RW flag enables both read and write access, allowing data transfer in both directions.
ngc workspace mount <your_workspace> ./ngc-mount --mode RW |
Access your NGC remote workspace through the ngc-mount directory.
Transfer files as needed between local and cloud storage.
The |
For efficient directory synchronization, rsync is recommended. Here's how to sync your cloud workspace to a local backup:
Create a backup directory: mkdir ngc-backup
Sync the directories with progress tracking: rsync -r ngc-mount/ ngc-backup/ --progress
The --progress flag allows you to monitor the transfer in real-time through the terminal.
If you are not familiar with |
Remember to unmount the workspace when finished.
ngc workspace unmount ngc-mount |
Lastly, make sure to verify your transfer:
The ngc-mount folder will appear empty after unmounting.
Your ngc-backup folder should contain all synced files.
If you have any questions on how to use ngc mount or ngc unmount, please refer to the official NVIDIA documentation on this topic.
Datasets can be dowloaded from both the web interface and the command line using ngc dataset download.
The simplest and most straightforward one, recommended when you have a small dataset (under 10GB). Just open your dataset and click the Download button in the top right corner of the page.
Recommended when you have a larger dataset (above 10GB).
Connect to lmm.its.albany.edu via SSH using your NetID:
macOS: Use Terminal command ssh <NetID>@lmm.its.albany.edu
Windows: Use PuTTY
Navigate to your lab directory.
cd /network/rit/lab/<your_lab> |
Then use the ngc dataset download command to download the dataset to this directory. Please note that unlike the workspace that you provided the name, for the dataset you have to provide its ID, which is conveniently located in the bottom left corner of the dataset page in the NGC web interface. You can either download the dataset as a folder with all the subdirectories and files or you can download it as a zip file when using the --zip flag.
# To download as a folder ngc dataset download <your_dataset_id> # To download as a zip file ngc dataset download --zip <your_dataset_id> |