Skip to content

Fix wp-env can't checkout git repositories of a specific tag. - #31271

Merged
noahtallen merged 4 commits into
WordPress:trunkfrom
yansern:fix/wp-env-cant-checkout-tags
Apr 29, 2021
Merged

noahtallen merged 4 commits into
WordPress:trunkfrom
yansern:fix/wp-env-cant-checkout-tags

Conversation

@yansern

@yansern yansern commented Apr 28, 2021 •

Copy link
Copy Markdown
Contributor

Description

Whenever I run npm run wp-env start, I'm getting the error GitError: error: pathspec 'tt1-blocks@0.4.5' did not match any file(s) known to git.

Here's the full log.

GitError: error: pathspec 'tt1-blocks@0.4.5' did not match any file(s) known to git

    at GitExecutorChain.onFatalException (/Users/yansern/A8C/gutenberg/packages/env/node_modules/simple-git/src/lib/runners/git-executor-chain.js:65:87)
    at GitExecutorChain.<anonymous> (/Users/yansern/A8C/gutenberg/packages/env/node_modules/simple-git/src/lib/runners/git-executor-chain.js:56:28)
    at Generator.throw (<anonymous>)
    at rejected (/Users/yansern/A8C/gutenberg/packages/env/node_modules/simple-git/src/lib/runners/git-executor-chain.js:6:65)
    at processTicksAndRejections (internal/process/task_queues.js:93:5) {
  task: {
    commands: [ 'checkout', 'tt1-blocks@0.4.5' ],
    format: 'utf-8',
    parser: [Function: parser]
  }
}

Why it failed?

It appears that using the git clone option for depth=1 --no-single-branch only pulls the tip of all branches, not all tags.

The tt1-blocks@0.4.5 is a tag of WordPress/theme-experiments.

How has this been tested?

Test wp-env with newly cloned repo

  1. Start with a clean slate by running rm -fr ~/.wp-env/*.
  2. Run npm run wp-env start.
  3. Once all done, run cat ~/.wp-env/*/theme-experiments/tt1-blocks/style.css | grep Version.
  4. You should see Version: 0.4.5.

image

Test wp-env with previously cloned repo

  1. Go to the repo folder, e.g. cd ~/.wp-env/*/theme-experiments.
  2. Checkout the previous version git checkout tt1-blocks@0.4.3.
  3. Delete the new version git tag -d tt1-blocks@0.4.5.
  4. Go back to the gutenberg folder and run npm run wp-env start.
  5. Once all done, run cat ~/.wp-env/*/theme-experiments/tt1-blocks/style.css | grep Version.
  6. You should see Version: 0.4.5 as well.

Screenshots

image
image

Types of changes

Bug fix

Checklist:

  • My code is tested.
  • My code follows the WordPress code style.
  • My code follows the accessibility standards.
  • I've tested my changes with keyboard and screen readers.
  • My code has proper inline documentation.
  • I've included developer documentation if appropriate.
  • I've updated all React Native files affected by any refactorings/renamings in this PR (please manually search all *.native.js files for terms that need renaming or removal).

@yansern
yansern requested a review from noahtallen as a code owner April 28, 2021 07:44
@yansern

yansern commented Apr 28, 2021

Copy link
Copy Markdown
Contributor Author

Found relevant discussion on Slack https://wordpress.slack.com/archives/C02QB2JS7/p1618478772435400. cc @ockham :)

ockham
ockham previously approved these changes Apr 28, 2021

@ockham ockham left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @yansern! This looks like a good fix 👍

Comment thread packages/env/lib/download-sources.js Resolved
@ockham

ockham commented Apr 29, 2021

Copy link
Copy Markdown
Contributor

Related to #31271 (comment): I was able to repro now. The key is to make wp-env believe that the theme version has changed:

  • Go to your ~/.wp-env/<hash>/theme-experiments folder.
  • git checkout tt1-blocks@0.4.3
  • git tag -d tt1-blocks@0.4.5
  • Go to your Gutenberg source folder.
  • In your .wp-env.json, change the TT1 Blocks version to 0.4.3:
diff --git a/.wp-env.json b/.wp-env.json
index 260787b97f..2849ba4450 100644
--- a/.wp-env.json
+++ b/.wp-env.json
@@ -1,9 +1,7 @@
 {
        "core": "WordPress/WordPress",
-       "plugins": [
-               "."
-       ],
-       "themes": [ "WordPress/theme-experiments/tt1-blocks#tt1-blocks@0.4.5" ],
+       "plugins": [ "." ],
+       "themes": [ "WordPress/theme-experiments/tt1-blocks#tt1-blocks@0.4.3" ],
        "env": {
                "tests": {
                        "mappings": {
  • Run npm run wp-env start. This will take a while, as it'll re-download things, since the .wp-env.json hash changed.
  • Once it's up, stop it again: npm run wp-env stop.
  • Now undo the changes to your .wp-env.json (e.g. git checkout -- .wp-env.json)
  • Try starting it again: npm run wp-env start. This time it should fail with the error message seen in the PR desc.

@ockham

ockham commented Apr 29, 2021

Copy link
Copy Markdown
Contributor

If (after #31271 (comment)) I then switch to this branch, npm run wp-env start works 👍

So it still seems good for me. @noahtallen can you provide instructions how to repro the scenario you were worried about here?

@ockham

ockham commented Apr 29, 2021

Copy link
Copy Markdown
Contributor

Oh, actually, it seems like this doesn't really switch to TT1 Blocks 0.4.5 though 🤔

image

@ockham
ockham dismissed their stale review April 29, 2021 13:03

Theme update doesn't seem to work entirely

@ockham

ockham commented Apr 29, 2021

Copy link
Copy Markdown
Contributor

The truth is, I'm not really sure how wp-env is supposed to handle this behind the scenes. I thought that it would switch my local .wp-env/<hash>/theme-experiments to the tag specified in .wp-env.json, but apparently it doesn't? Though I'm still not sure why it's no longer throwing the error it threw before. It's all very confusing 😬

Maybe @noahtallen can shed some light on this?

@noahtallen

Copy link
Copy Markdown
Member

Related to #31271 (comment): I was able to repro now

Thanks for those steps, I'll try to troubleshoot based on that! The existing code is supposed to handle fetching and checking out a new ref if the source has already been cloned or not. The code proposed here (theoretically) only handles cloning and checking out a ref if the source does not exist yet.

@yansern

yansern commented Apr 29, 2021

Copy link
Copy Markdown
Contributor Author

Perhaps restoring the lines that were removed would then make it work on both new repositories and existing repositories.

Comment thread packages/env/lib/download-sources.js Resolved
@noahtallen

noahtallen commented Apr 29, 2021 •

Copy link
Copy Markdown
Member

I'm not really sure how wp-env is supposed to handle this behind the scenes. I thought that it would switch my local .wp-env//theme-experiments to the tag specified in .wp-env.json, but apparently it doesn't?

And to clarify, this is exactly how it's supposed to work 😅 (e.g. you're right: it's confusing that it's not working)

@noahtallen

Copy link
Copy Markdown
Member

Oh, actually, it seems like this doesn't really switch to TT1 Blocks 0.4.5 though

I think this makes sense to me. The current fix doesn't attempt to check out the new tag anymore, because it's gated behind ! isRepo, but you've already verified that the repo does exist. So it stays on the old version without switching.

@noahtallen

Copy link
Copy Markdown
Member

Oh... And this situation also crashes on trunk because trunk tries to checkout the ref 4.5... But in the repro steps, we deleted the 4.5 tag locally, so it doesn't exist. (which is exactly what the git error is complaining about). It doesn't crash on this branch because of my previous comment (e.g. it never tries to check anything out)

@noahtallen

Copy link
Copy Markdown
Member

I was able to replicate locally in the git repo. The fetch command doesn't seem to always make the tag available for checkout locally:

Does not work:

# Make tag unavailable locally:
$ git checkout tt1-blocks@0.4.3 && git tag -d tt1-blocks@0.4.5

# Try to fetch it again from the origin:
$ git fetch origin tt1-blocks@0.4.5
From https://github.com/WordPress/theme-experiments
 * tag               tt1-blocks@0.4.5 -> FETCH_HEAD

$ git checkout tags/tt1-blocks@0.4.5
error: pathspec 'tags/tt1-blocks@0.4.5' did not match any file(s) known to git

Does work:

# Make tag unavailable locally:
$ git checkout tt1-blocks@0.4.3 && git tag -d tt1-blocks@0.4.5

# Try to fetch it again from the origin:
$ git fetch origin tt1-blocks@0.4.5:tt1-blocks@0.4.5
From https://github.com/WordPress/theme-experiments
 * [new tag]         tt1-blocks@0.4.5 -> tt1-blocks@0.4.5
 * 
$ git checkout tags/tt1-blocks@0.4.5                

Previous HEAD position was 7aaf36d Merge pull request #188 from WordPress/tt1-blocks/version-bump-0.4.3
HEAD is now at d290b42 Merge pull request #243 from WordPress/tt1-blocks/version-bump-4-4-5

@noahtallen

noahtallen commented Apr 29, 2021 •

Copy link
Copy Markdown
Member

This patch seems to fix the problem for me:

diff --git a/packages/env/lib/download-sources.js b/packages/env/lib/download-sources.js
index d87cd8e821..73e338ddf6 100644
--- a/packages/env/lib/download-sources.js
+++ b/packages/env/lib/download-sources.js
@@ -128,7 +128,9 @@ async function downloadGitSource( source, { onProgress, spinner, debug } ) {
        }
 
        log( 'Fetching the specified ref.' );
-       await git.fetch( 'origin', source.ref );
+       await git.fetch( 'origin', `${ source.ref }:${ source.ref }`, {
+               '-u': true,
+       } );
 
        log( 'Checking out the specified ref.' );
        await git.checkout( source.ref );

edit: I don't think this is perfect. You aren't supposed to use -u for existing branches. Additionally, I'm not sure exactly what the colon does -- is it creating a branch?

@noahtallen

noahtallen commented Apr 29, 2021 •

Copy link
Copy Markdown
Member

After tinkering a bit, maybe we should just be less clever about it. I think our problems are partially caused by trying to update the repo if it already exists. This gets challenging. For example, if we just want to pull the trunk branch, we would need to detect and run git pull. But that won't work for other specs like tags. Relatedly, this seems to uncover an existing bug too. If we do git fetch origin trunk and then git checkout trunk, it doesn't actually get the new commits into the existing branch. So sources like that are currently not updating correctly, I think.

I'm not quite sure exactly what to do. My intuition right now is to try to clear the local git state somehow so that when we fetch and checkout from the remote, nothing conflicts locally 🤔

@yansern

yansern commented Apr 29, 2021

Copy link
Copy Markdown
Contributor Author

Okay got it. Just a simple git fetch --tags will do. Works on both new and existing repo without the 0.4.5 tag.

2021-04-29_23-19-34

@yansern

yansern commented Apr 29, 2021

Copy link
Copy Markdown
Contributor Author

I've updated the test instructions on how to test on both new and existing repo. :) Thanks @ockham @noahtallen for testing it out and also.. the enthusiasm!!!

@noahtallen noahtallen left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This tests fantastically for me! It solves the crash which happens in the scenario @ockham described. Will merge this once tests pass 👍

As a follow up, I plan to fix the problem with pulling / fetching branches.

Comment thread packages/env/lib/download-sources.js Resolved
@noahtallen
noahtallen merged commit 1cacd5e into WordPress:trunk Apr 29, 2021
@github-actions github-actions Bot added this to the Gutenberg 10.6 milestone Apr 29, 2021
@noahtallen

Copy link
Copy Markdown
Member

Follow-up PR: #31362

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants