/
Text to Video with Wan2.1 on NVIDIA DGX

Text to Video with Wan2.1 on NVIDIA DGX

Wan2.1 is a comprehensive and open suite of video foundation models that pushes the boundaries of video generation. In this tutorial, we will be using Wan2.1 Text-to-Video to create a 720P video based on a prompt.

Buckle up, because this is going to be an awesome demo!

Step 1 - Hugging Face Authentication

Create a new Hugging Face token with Read access to contents of all public gated repos you can access permission. The token will have a prefix hf_. Keep it securely, as it will be used in the next steps.

Step 2 - Launching the Job

Below is a template SLURM script to launch the job. Since JupyterLab will be very helpful for this demo, we built the SLURM script based on the script available in this other tutorial: UAlbany Supercomputing Resources Guide . As you can see, we have included code that downloads the model and installs the required dependencies

#!/bin/bash #SBATCH --job-name=wan #SBATCH --output=%j.out #SBATCH --time=8:00:00 #SBATCH --gpus=1 #SBATCH --container-image='docker://nvcr.io#nvidia/pytorch:25.03-py3' #SBATCH --container-mounts=/network/rit/dgx/dgx_YOUR_LAB_NAME:/mnt/dgx_lab,/network/rit/lab/YOUR_LAB_NAME:/mnt/lab #SBATCH --container-writable # SLURM Parameters SLURM_NODE_NAME="$SLURMD_NODENAME" SLURM_JOB_ID="$SLURM_JOB_ID" # Generate a random port number between 8000 and 8999 JUPYTER_HTTP_PORT=$((RANDOM % 1000 + 8000)) # Build the Jupyter URL JUPYTER_HTTP_URL="http://${SLURM_NODE_NAME}.its.albany.edu:${JUPYTER_HTTP_PORT}" # Generate a random password (alphanumeric, 6 characters) JUPYTER_PASSWORD=$(tr -dc A-Za-z0-9 </dev/urandom | head -c 6) # Prepare Demo Folder DEMO_DIR="/mnt/dgx_lab/wan_demo" mkdir -p ${DEMO_DIR}/Wan2.1 # Clone model repository git clone https://github.com/Wan-Video/Wan2.1.git ${DEMO_DIR}/Wan2.1 # Install dependencies pip install -r ${DEMO_DIR}/Wan2.1/requirements.txt # Set Hugging Face Token (replace YOUR_HUGGING_FACE_TOKEN with your actual token) export HF_TOKEN="YOUR_HUGGING_FACE_TOKEN" # Hugging Face login huggingface-cli login --token $HF_TOKEN # Dowload model huggingface-cli download Wan-AI/Wan2.1-T2V-14B --local-dir ${DEMO_DIR}/Wan2.1/Wan2.1-T2V-14B # Print session details echo -e "\n================================================================================\n" echo -e "SLURM Job ID ${SLURM_JOB_ID} running on ${SLURM_NODE_NAME}\n" echo -e "Your JupyterLab session is available at: ${JUPYTER_HTTP_URL}\n" echo -e "Your password is: ${JUPYTER_PASSWORD}\n" echo -e "Please copy and paste the link into your browser and use the password to log in." echo -e "\n================================================================================\n" # Start JupyterLab session jupyter lab --allow-root --no-browser --NotebookApp.token="${JUPYTER_PASSWORD}" --NotebookApp.allow_origin='*' --NotebookApp.log_level='CRITICAL' --notebook-dir=/mnt --port=$JUPYTER_HTTP_PORT

You can copy the contents from this snippet and save it as wan.slurm or you can simply download it from here: .

  1. Connect to the DGX On-Prem cluster over SSH (ssh your_netid@dgx-head01.its.albany.edu). If off-campus, ensure VPN is connected before logging in.

  2. Download and edit the above SLURM script for your use:

    • Replace YOUR_LAB_NAME with your lab's actual name.

    • Replace YOUR_HUGGING_FACE_TOKEN with the Hugging Face token (including hf_ prefix).

  3. Submit the job to SLURM (sbatch wan.slurm). SLURM will assign and queue your job according to resource availability.

  4. Monitor job status (squeue -u YOUR_NETID).

  5. Watch job progress by checking the output file (tail -f YOUR_JOB_ID.out). Look for an entry similar to the following, as you will use the information on the next step.

================================================================================ SLURM Job ID 23625 running on dgx06 Your JupyterLab session is available at: http://dgx06.its.albany.edu:8832 Your password is: 5WYuDr Please copy and paste the link into your browser and use the password to log in. ================================================================================

Step 3 - Running Inference

Open JupyterLab in your browser using the URL and password provided.

Make sure you're inside the DEMO_DIR (in this case /mnt/dgx_lab/wan_demo).

From there, run an inference like in the example below. Feel free to play around with the prompt – this is just an example!

python generate.py --task t2v-14B --size 1280*720 --ckpt_dir ./Wan2.1-T2V-14B --prompt "A fluffy orange tabby cat with distinctive black stripes sits on a sunny windowsill, slowly turning its head to watch raindrops racing down the glass. Soft morning light filters through the window, creating warm golden highlights on the cat's fur. The cat's green eyes follow the droplets with curiosity, occasionally tilting its head. In the background, blurred houseplants and a cozy living room create a peaceful domestic scene. The cat's tail gently sways as it remains captivated by the rain. Shot with shallow depth of field, cinematic lighting, 4K quality."

It should take an hour or so to generate the video, so just sit back and wait! In the meantime, keep an eye on the logs to monitor progress.

[2025-09-25 14:43:57,884] INFO: offload_model is not specified, set to True. [2025-09-25 14:43:57,884] INFO: Generation job args: Namespace(task='t2v-14B', size='1280*720', frame_num=81, ckpt_dir='./Wan2.1-T2V-14B', offload_model=True, ulysses_size=1, ring_size=1, t5_fsdp=False, t5_cpu=False, dit_fsdp=False, save_file=None, src_video=None, src_mask=None, src_ref_images=None, prompt="A fluffy orange tabby cat with distinctive black stripes sits on a sunny windowsill, slowly turning its head to watch raindrops racing down the glass. Soft morning light filters through the window, creating warm golden highlights on the cat's fur. The cat's green eyes follow the droplets with curiosity, occasionally tilting its head. In the background, blurred houseplants and a cozy living room create a peaceful domestic scene. The cat's tail gently sways as it remains captivated by the rain. Shot with shallow depth of field, cinematic lighting, 4K quality.", use_prompt_extend=False, prompt_extend_method='local_qwen', prompt_extend_model=None, prompt_extend_target_lang='zh', base_seed=9045968746138841141, image=None, first_frame=None, last_frame=None, sample_solver='unipc', sample_steps=50, sample_shift=5.0, sample_guide_scale=5.0) [2025-09-25 14:43:57,884] INFO: Generation model config: {'__name__': 'Config: Wan T2V 14B', 't5_model': 'umt5_xxl', 't5_dtype': torch.bfloat16, 'text_len': 512, 'param_dtype': torch.bfloat16, 'num_train_timesteps': 1000, 'sample_fps': 16, 'sample_neg_prompt': '色调艳丽，过曝，静态，细节模糊不清，字幕，风格，作品，画作，画面，静止，整体发灰，最差质量，低质量，JPEG压缩残留，丑陋的，残缺的，多余的手指，画得不好的手部，画得不好的脸部，畸形的，毁容的，形态畸形的肢体，手指融合，静止不动的画面，杂乱的背景，三条腿，背景人很多，倒着走', 't5_checkpoint': 'models_t5_umt5-xxl-enc-bf16.pth', 't5_tokenizer': 'google/umt5-xxl', 'vae_checkpoint': 'Wan2.1_VAE.pth', 'vae_stride': (4, 8, 8), 'patch_size': (1, 2, 2), 'dim': 5120, 'ffn_dim': 13824, 'freq_dim': 256, 'num_heads': 40, 'num_layers': 40, 'window_size': (-1, -1), 'qk_norm': True, 'cross_attn_norm': True, 'eps': 1e-06} [2025-09-25 14:43:57,885] INFO: Input prompt: A fluffy orange tabby cat with distinctive black stripes sits on a sunny windowsill, slowly turning its head to watch raindrops racing down the glass. Soft morning light filters through the window, creating warm golden highlights on the cat's fur. The cat's green eyes follow the droplets with curiosity, occasionally tilting its head. In the background, blurred houseplants and a cozy living room create a peaceful domestic scene. The cat's tail gently sways as it remains captivated by the rain. Shot with shallow depth of field, cinematic lighting, 4K quality. [2025-09-25 14:43:57,885] INFO: Creating WanT2V pipeline. [2025-09-25 14:45:07,681] INFO: loading ./Wan2.1-T2V-14B/models_t5_umt5-xxl-enc-bf16.pth [2025-09-25 14:45:17,862] INFO: loading ./Wan2.1-T2V-14B/Wan2.1_VAE.pth [2025-09-25 14:45:18,296] INFO: Creating WanModel from ./Wan2.1-T2V-14B Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:00<00:00, 16.90it/s] [2025-09-25 14:45:36,214] INFO: Generating video ... 10%|███████████▋ | 5/50 [05:30<49:32, 66.06s/it]

While the video is being generated, you can see how the model squeezes every TFLOP from the GPU – not a single Watt is left on the table.

Every 2.0s: nvidia-smi dgx21: Thu Sep 25 15:09:33 2025 Thu Sep 25 15:09:33 2025 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 570.158.01 Driver Version: 570.158.01 CUDA Version: 12.8 | |-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 NVIDIA A100-SXM4-80GB Off | 00000000:87:00.0 Off | 0 | | N/A 68C P0 423W / 400W | 73977MiB / 81920MiB | 100% Default | | | | Disabled | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | 0 N/A N/A 2908155 C python 73968MiB | +-----------------------------------------------------------------------------------------+

When generation finishes, look for an .mp4 file in your demo folder. Download the file and play it on your computer. You should expect to see something like the following.

 

There's plenty to explore with these models, like Image-to-Video conversion for instance. Explore the official documentation for more details and parameters: GitHub - Wan-Video/Wan2.1: Wan: Open and Advanced Large-Scale Video Generative Models. This tutorial provides a solid foundation to get started with Text-to-Video on DGX systems. Feel free to experiment with different parameters and models to tailor the experience to your needs.