Add support for sparse checkouts (#1369)

* Add support for sparse checkouts

* sparse-checkout: optionally turn off cone mode

While it _is_ true that cone mode is the default nowadays (mainly for
performance reasons: code mode is much faster than non-cone mode), there
_are_ legitimate use cases where non-cone mode is really useful.

Let's add a flag to optionally disable cone mode.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

* Verify minimum Git version for sparse checkout

The `git sparse-checkout` command is available only since Git version
v2.25.0. The `actions/checkout` Action actually supports older Git
versions than that; As of time of writing, the minimum version is
v2.18.0.

Instead of raising this minimum version even for users who do not
require a sparse checkout, only check for this minimum version
specifically when a sparse checkout was asked for.

Suggested-by: Tingluo Huang <tingluohuang@github.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

* Support sparse checkout/LFS better

Instead of fetching all the LFS objects present in the current revision
in a sparse checkout, whether they are needed inside the sparse cone or
not, let's instead only pull the ones that are actually needed.

To do that, let's avoid running that preemptive `git lfs fetch` call in
case of a sparse checkout.

An alternative that was considered during the development of this patch
(and ultimately rejected) was to use `git lfs pull --include <path>...`,
but it turned out to be too inflexible because it requires exact paths,
not the patterns that are available via the sparse checkout definition,
and that risks running into command-line length limitations.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

---------

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Co-authored-by: Daniel <daniel.fernandez@feverup.com>

__test__/git-auth-helper.test.ts

@@ -727,6 +727,8 @@ async function setup(testName: string): Promise<void> {
     branchDelete: jest.fn(),
     branchExists: jest.fn(),
     branchList: jest.fn(),
+    sparseCheckout: jest.fn(),
+    sparseCheckoutNonConeMode: jest.fn(),
     checkout: jest.fn(),
     checkoutDetach: jest.fn(),
     config: jest.fn(
@@ -800,6 +802,8 @@ async function setup(testName: string): Promise<void> {
     authToken: 'some auth token',
     clean: true,
     commit: '',
+    sparseCheckout: [],
+    sparseCheckoutConeMode: true,
     fetchDepth: 1,
     lfs: false,
     submodules: false,

__test__/git-command-manager.test.ts

@@ -39,7 +39,12 @@ describe('git-auth-helper tests', () => {
     jest.spyOn(exec, 'exec').mockImplementation(mockExec)
     const workingDirectory = 'test'
     const lfs = false
-    git = await commandManager.createCommandManager(workingDirectory, lfs)
+    const doSparseCheckout = false
+    git = await commandManager.createCommandManager(
+      workingDirectory,
+      lfs,
+      doSparseCheckout
+    )
 
     let branches = await git.branchList(false)
 
@@ -70,7 +75,12 @@ describe('git-auth-helper tests', () => {
     jest.spyOn(exec, 'exec').mockImplementation(mockExec)
     const workingDirectory = 'test'
     const lfs = false
-    git = await commandManager.createCommandManager(workingDirectory, lfs)
+    const doSparseCheckout = false
+    git = await commandManager.createCommandManager(
+      workingDirectory,
+      lfs,
+      doSparseCheckout
+    )
 
     let branches = await git.branchList(false)
 

__test__/git-directory-helper.test.ts

@@ -462,6 +462,8 @@ async function setup(testName: string): Promise<void> {
     branchList: jest.fn(async () => {
       return []
     }),
+    sparseCheckout: jest.fn(),
+    sparseCheckoutNonConeMode: jest.fn(),
     checkout: jest.fn(),
     checkoutDetach: jest.fn(),
     config: jest.fn(),

__test__/input-helper.test.ts

@@ -79,6 +79,8 @@ describe('input-helper tests', () => {
     expect(settings.clean).toBe(true)
     expect(settings.commit).toBeTruthy()
     expect(settings.commit).toBe('1234567890123456789012345678901234567890')
+    expect(settings.sparseCheckout).toBe(undefined)
+    expect(settings.sparseCheckoutConeMode).toBe(true)
     expect(settings.fetchDepth).toBe(1)
     expect(settings.lfs).toBe(false)
     expect(settings.ref).toBe('refs/heads/some-ref')

__test__/verify-sparse-checkout-non-cone-mode.sh

@@ -0,0 +1,51 @@
+#!/bin/sh
+
+# Verify .git folder
+if [ ! -d "./sparse-checkout-non-cone-mode/.git" ]; then
+  echo "Expected ./sparse-checkout-non-cone-mode/.git folder to exist"
+  exit 1
+fi
+
+# Verify sparse-checkout (non-cone-mode)
+cd sparse-checkout-non-cone-mode
+
+ENABLED=$(git config --local --get-all core.sparseCheckout)
+
+if [ "$?" != "0" ]; then
+    echo "Failed to verify that sparse-checkout is enabled"
+    exit 1
+fi
+
+# Check that sparse-checkout is enabled
+if [ "$ENABLED" != "true" ]; then
+  echo "Expected sparse-checkout to be enabled (is: $ENABLED)"
+  exit 1
+fi
+
+SPARSE_CHECKOUT_FILE=$(git rev-parse --git-path info/sparse-checkout)
+
+if [ "$?" != "0" ]; then
+    echo "Failed to validate sparse-checkout"
+    exit 1
+fi
+
+# Check that sparse-checkout list is not empty
+if [ ! -f "$SPARSE_CHECKOUT_FILE" ]; then
+  echo "Expected sparse-checkout file to exist"
+  exit 1
+fi
+
+# Check that all folders from sparse-checkout exists
+for pattern in $(cat "$SPARSE_CHECKOUT_FILE")
+do
+  if [ ! -d "${pattern#/}" ]; then
+    echo "Expected directory '${pattern#/}' to exist"
+    exit 1
+  fi
+done
+
+# Verify that the root directory is not checked out
+if [ -f README.md ]; then
+  echo "Expected top-level files not to exist"
+  exit 1
+fi

__test__/verify-sparse-checkout.sh

@@ -0,0 +1,63 @@
+#!/bin/sh
+
+# Verify .git folder
+if [ ! -d "./sparse-checkout/.git" ]; then
+  echo "Expected ./sparse-checkout/.git folder to exist"
+  exit 1
+fi
+
+# Verify sparse-checkout
+cd sparse-checkout
+
+SPARSE=$(git sparse-checkout list)
+
+if [ "$?" != "0" ]; then
+    echo "Failed to validate sparse-checkout"
+    exit 1
+fi
+
+# Check that sparse-checkout list is not empty
+if [ -z "$SPARSE" ]; then
+  echo "Expected sparse-checkout list to not be empty"
+  exit 1
+fi
+
+# Check that all folders of the sparse checkout exist
+for pattern in $SPARSE
+do
+  if [ ! -d "$pattern" ]; then
+    echo "Expected directory '$pattern' to exist"
+    exit 1
+  fi
+done
+
+checkSparse () {
+  if [ ! -d "./$1" ]; then
+    echo "Expected directory '$1' to exist"
+    exit 1
+  fi
+
+  for file in $(git ls-tree -r --name-only HEAD $1)
+  do
+    if [ ! -f "$file" ]; then
+      echo "Expected file '$file' to exist"
+      exit 1
+    fi
+  done
+}
+
+# Check that all folders and their children have been checked out
+checkSparse __test__
+checkSparse .github
+checkSparse dist
+
+# Check that only sparse-checkout folders have been checked out
+for pattern in $(git ls-tree --name-only HEAD)
+do
+  if [ -d "$pattern" ]; then
+    if [[ "$pattern" != "__test__" && "$pattern" != ".github" && "$pattern" != "dist" ]]; then
+      echo "Expected directory '$pattern' to not exist"
+      exit 1
+    fi
+  fi
+done