dvs

Overview

Git is built for code, not data. Large datasets bloat repositories, and sensitive ones often shouldn’t be committed at all, so data ends up on shared drives with names like final_v2_fixed.csv and no reliable link to the code that used it. dvs keeps that link without putting data in Git. Adding a file stores its contents in shared storage, compressed and identified by a fingerprint of the contents, and writes a small metadata file that’s committed alongside the code. Collaborators pull the code as usual and use dvs to retrieve the exact version of each file the metadata points to. A quick status check shows whether each file is current, missing, or changed since it was last added. Every addition is recorded in an append-only audit log of who added which version and when. Identical files are stored only once, and stored versions are made read-only so they can’t be altered after the fact. dvs runs from the command line or R, works with or without Git, and processes large batches of files in parallel.