/
DGX Cloud (H100) Data Synchronization Guide

DGX Cloud (H100) Data Synchronization Guide

Owned by Vieira Sobrinho, Jose
Last updated: Mar 13, 2025
2 min read

Overview

The DGX Cloud is a subscription-based service that is renewed on a yearly basis. With each renewal, we typically receive NVIDIA's most up-to-date GPUs, which in turn typically require users to migrate between environments, as demonstrated in early 2025 when we transitioned from the A100 to the H100 cluster. Additionally, there's always the possibility that the subscription may not be renewed at the end of its term. For these reasons, it is critical that all users regularly synchronize their data from the cloud environment to local storage to prevent data loss.

This guide explains how to use the provided synchronization script to ensure your data is safely backed up.

Quick Start

1. Download and edit the script

  • The script is available here:

  • Choose which folder inside your local lab you want to use for synchronization

    • Example: If your lab folder is dgx_vieirasobrinho_lab

    • Create a folder called ngc on the top level of your lab folder

    • The local path will be /network/rit/dgx/dgx_vieirasobrinho_lab/ngc

  • You can do the same on the DGX Cloud side, but we advise you to sync everything in your home folder (~/ or /home/[your_netID])

  • Edit the script replacing the placeholders accordingly:

    LOCAL_PATH="/network/rit/dgx/dgx_[your_lab_folder]/ngc" REMOTE_PATH="/home/[your_NetID]" REMOTE_USER="[your_NetID]"

  • Example configuration:

  • LOCAL_PATH="/network/rit/dgx/dgx_vieirasobrinho_lab/ngc" REMOTE_PATH="/home/jv535825" REMOTE_USER="jv535825"

2. Save the changes

Move the script to your lab directory (it's helpful to have your lab folder mapped to your computer)

3. SSH to the DGX local head node

ssh [your_NetID]@dgx-head01.its.albany.edu

4. Add execute permissions

This only needs to be done once:

chmod +x sync-script.sh

5. Run the script

./sync-script.sh

6. Choose the sync direction

Verify that the paths are correct when prompted:

Directional Directory Sync Script This script synchronizes files between local and remote directories LOCAL: /network/rit/dgx/dgx_vieirasobrinho_lab/ngc REMOTE: jv535825@ngc:/home/jv535825 Please select sync direction: 1) DOWNLOAD - Sync from REMOTE → LOCAL (pull remote changes) 2) UPLOAD - Sync from LOCAL → REMOTE (push local changes) q) Quit

Sync Options

Option 1: DOWNLOAD

  • Use this to copy data FROM the cloud TO local storage

  • Run this regularly to ensure your work is backed up locally

Option 2: UPLOAD

  • Use this to copy data FROM local storage TO the cloud

  • Run this when you need to push local changes to the cloud environment - e.g., you are getting started with the DGX (H100) Cloud and need to copy your datasets and codebase to that environment

Final Steps

Preview and confirm changes

After selecting a direction, the script will:

  • Check if the source and destination directories exist

  • Show a preview of files that will be transferred

  • Ask for confirmation before proceeding

Important: Review the preview carefully before confirming. This shows which files will be transferred and which might be deleted.

Video Demo

Best Practices

  1. Regular backups: Set a recurring reminder to run the DOWNLOAD sync at least weekly

  2. Before migrations: Always perform a full DOWNLOAD sync before any scheduled environment migrations

  3. Verify transfers: After synchronization, browse both directories to ensure all critical files were transferred