Running DVC on Windows

Different issues can arise when running DVC on Microsoft Windows, mainly involving system performance. Some, for example, have to do with NTFS file system characteristics and Windows built-in security mechanisms. Below are some workarounds that can help avoid these potential problems:

POSIX-like command line experience

The regular Command Prompt (cmd) in Windows will most likely not help you use DVC effectively, or follow the examples in our docs. Please avoid it. There's no perfect solution, but here are some ideas:

• The full Cmder console emulator combines several useful tools like ConEmu, and Git for Windows (Git Bash) among other shell options.
• Anaconda Prompt is another recommendation.
• Consider enabling and using WSL (Windows Terminal also recommended) which supports the most CLI features (e.g. \ line continuation). But it has major I/O performance issues and is unable to access GPUs, among other limitations.
• Install an actual Linux distro (e.g. Ubuntu) on a virtual machine, or in a HD partition (dual boot).

Enable symbolic links

This is done automatically by the DVC's Windows installer, but you may want to do it manually after any other installation methods like (choco, conda, pip).

Symlinks are one of the possible file link types that DVC can use for optimization purposes. They are available on Windows, but the Create symbolic links user privilege is needed. It's granted to the Administrators group by default, so running dvc in a terminal as an admin is a good option for occasional use. For regular users, it can be granted using the Local Security Policy console.

Whitelist in Windows Security

Windows 10 includes the Windows Security antivirus. If user wants to avoid antivirus scans on specific folders or files to improve the performance, then whitelist them in Windows Security as per this guide. For example, we can whitelist DVC binary files on Windows to speed up the processes.

Enable long folder/file paths

dvc pull may fail in some cases when the folder path is longer than 260 characters. In the Windows API, the maximum allowed length for a path is 260 characters. If required, the user can explicitly enable long paths by following this guide.

Fix or disable Search Indexing

Search Indexing can also slow down file I/O operations on Windows. Try fixing or disabling this feature if you don't need it.

Disable short-file name generation

With NTFS, users may want to disable 8dot3 as per this article to disable the short-file name generation. It is important to do so for better performance when the user has over 300K files in a single directory.

Avoid directories with large number of files

The performance of NTFS degrades while handling large volumes of files in a directory, as explained in this issue.

Enabling paging with less

By default, DVC tries to use Less as pager for the output of dvc dag. Windows doesn't have the less command available however. Fortunately, there is a easy way of installing it via Chocolatey (please install the tool first):

$ choco install less

less can be installed in other ways, just make sure it's available in the command line environment where you run dvc. (This usually means adding the directory where less is installed to the PATH environment variable.)