Storage full can occur when users generate many files, transfer files, or perform other storage-intensive tasks. This document provides guidance to help you troubleshoot this.
Check disk usage
When encountering a storage full, the first step is to check your container’s disk usage. You can use the df -h command to display a summary of disk usage.
Example output:
root@9b8e325167b2:/# df -h
Filesystem                    Size  Used Avail Use% Mounted on
overlay                        20G   16M   20G   1% /
tmpfs                          64M     0   64M   0% /dev
tmpfs                         252G     0  252G   0% /sys/fs/cgroup
shm                            24G     0   24G   0% /dev/shm
/dev/sda2                     457G   12G  423G   3% /usr/bin/nvidia-smi
tmpfs                         252G   12K  252G   1% /proc/driver/nvidia
tmpfs                         252G  4.0K  252G   1% /etc/nvidia/nvidia-application-profiles-rc.d
tmpfs                          51G  4.4M   51G   1% /run/nvidia-persistenced/socket
tmpfs                         252G     0  252G   0% /proc/asound
tmpfs                         252G     0  252G   0% /proc/acpi
tmpfs                         252G     0  252G   0% /proc/scsi
tmpfs                         252G     0  252G   0% /sys/firmware
tmpfs                         252G     0  252G   0% /sys/devices/virtual/powercap
Key areas to check
Container Disk Usage: The primary storage area for your container is mounted on the overlay filesystem. This indicates the container’s root directory.
Filesystem                    Size  Used Avail Use% Mounted on
overlay                        20G   16M   20G   1% /
du -sh . to check the space usage of the current directory.
By default, the volume disk or network volume is mounted at /workspace, You can check the usage with the following example::
root@9b8e325167b2:/# cd workspace/
root@9b8e325167b2:/workspace# du -sh .
194M    .
/workspace, you can run the following command:
root@9b8e325167b2:/# find /workspace -type f -exec du -h {} + | sort -rh | head -n 10
96M     /workspace/f.txt
96M     /workspace/e.txt
1.0K    /workspace/c.txt
512     /workspace/b.txt
512     /workspace/a.txt
Removing files and directories
Once you’ve identified large files or directories that are no longer needed, you can remove them to free up space.
This will permanently delete the file, folder. Use with caution.
# To delete a specific file, use the rm command:
rm /path/to/file
# To remove an entire directory and its contents, use the rm -r command:
rm -r /path/to/directory
JupyterLab hidden trash bin
If you deleted files through JupyterLab’s UI but storage isn’t freed, they may be in a hidden trash directory.
Quick check and fix
Check if trash directories exist:
Check for trash directories
{ ls -lah $HOME/.local/share/Trash/ 2>/dev/null && echo "✓ Found: $HOME/.local/share/Trash/"; ls -lah /workspace/.Trash* 2>/dev/null && echo "✓ Found: /workspace/.Trash*"; } || echo "✗ No trash directories found"
[ -d "$HOME/.local/share/Trash" ] && rm -rf $HOME/.local/share/Trash/* && echo "✓ Cleared home trash" || echo "✗ No home trash to clear"
ls -d /workspace/.Trash* 2>/dev/null && rm -rf /workspace/.Trash* && echo "✓ Cleared workspace trash" || echo "✗ No workspace trash to clear"
Additional storage
If your Pod needs more than 20GB of storage, consider using a network volume. For more information, see Network volumes, or refer to this blog post.