Deploy preview won't build without clearing the cache

Support
,
1

Hi :wave: :relaxed:,

We use the deploy previews, but the builds are failing without clearing the cache first (using the web UI).

Logs

Fortunately it’s a public repo, so I can share with you an example of how the logs look like:
https://app.netlify.com/sites/theatrejs-playground/deploys/623ddac44ac1220008157c12

The bug (probably affects all the public repos using yarn 2 or above with workspaces)

I raised an issue on GitHub where I described where I think the problem is in the build script: link

The fix for the bug

Since I have already seen the problem, I also submitted a PR with a fix: link

Our quickfix that doesn’t actually work

We wanted to avoid manually re-deploying the previews (with the cache being cleared beforehand), so we wanted to use a quickfix: we deleted the /opt/build/cache directory after the build script has run successfully (inspired by this answer on the forum), which unfortunately does not seem to work (the problem is the same, see an example here). My suspicion is that maybe deleting the cache folder only prevents caching after a successful build and the new builds will use the last successfully saved cache, but I’m not sure…

Questions

I only have two questions:

  1. Can you help us to find a quick fix that works?
  2. Could you have a look at the PR and merge it if it solves the problem? I’m happy to work on it more if you give me feedback.

That’s all, I hope this issue can be resolved in the near future! :relaxed:

2

hi there, sorry we have been slow to reply, but i’m wondering if you saw this guide at all?

3

Hi,

thanks for the response! Actually I haven’t read the guide before but already went through all the debugging steps in it. I found the bug when I was debugging the cached dependencies locally in the Docker image.

I also created an issue about the bug (link) on GitHub (and submitted a fix in a PR), but here’s the gist of it:

In yarn 1.x only private repos can have workspaces, but this limitation was removed in yarn 2:

Worktrees used to be required to be private (ie list "private": true in their package.json). This requirement got removed with the 2.0 release in order to help standalone projects to progressively adopt workspaces (for example by listing their documentation website as a separate workspace). (source: docs)

The problem is in the run-build-functions.sh file:

  # YARN_IGNORE_PATH will ignore the presence of a local yarn executable (i.e. yarn 2) and default
  # to using the global one (which, for now, is always yarn 1.x). See https://yarnpkg.com/configuration/yarnrc#ignorePath
  # we can actually use this command for npm workspaces as well
  workspace_output="$(YARN_IGNORE_PATH=1 yarn workspaces --json info 2>/dev/null)"
  workspace_exit_code=$?

As you can see it tries to output information about the workspaces with yarn 1.x. in the command substitution and redirects STDERR to /dev/null. After that it only checks if the process exited with 0 or an error code, but ignores the error message, which would tell us, that ‘workspaces can only be enabled in private projects’. Here’s an example for the problem from the theatre-js/theatre repo:

  • when the errors are hidden:
 $ YARN_IGNORE_PATH=1 yarn workspaces --json info 2>/dev/null
{"type":"info","data":"Visit \u001b[1mhttps://yarnpkg.com/en/docs/cli/workspaces\u001b[22m for documentation about this command."}
  • when the errors are visible:
$ YARN_IGNORE_PATH=1 yarn workspaces --json info
{"type":"error","data":"Workspaces can only be enabled in private projects."}
{"type":"info","data":"Visit \u001b[1mhttps://yarnpkg.com/en/docs/cli/workspaces\u001b[22m for documentation about this command."}

I hope I gave you all the information I needed to, but feel free to ask more questions if that’s not the case! :relaxed:

5

Thanks so much for filing a PR! We don’t currently proactively monitor incoming PRs on that repo, but we have gotten this to the appropriate teams to review, so we will wait for them for feedback. It looks super thoughtful and well written, but we do have to test any changes like that super thoroughly since fixing your issue may break 2 million other customers’ builds…

RE: your quick fix, your suspicion is correct - we only update cache on successful builds, and our update will also show success in the logs (whereas I see your attempts to remove the directory led to some “missing directory” type errors while we tried to save).

The better quick fix would be if we had a straight up “disable build cache for this site” feature, which we don’t, yet, but we have added your voice to an open feature request on the topic!

One trick you can use, but which I can’t easily guide you through since the steps are likely somewhat complicated, is changing your site to use Build hooks | Netlify Docs.

For those, you can add a ?clear_cache=true query param to clear the cache before the build starts. But those hooks are per-branch and not as flexible as our default config using the GitHub app and are essentially needing to be set up on a branch-by-branch basis, rather than configuring one setting for the whole site that you can further configure in our UI - but that might unblock you in terms of “we just need our main production branch to build reliably” - you could probably most easily use this workaround by doing a “manual” linkage via the CLI: Get started with Netlify CLI | Netlify Docs which would give you a hook like the ones in the build hooks doc, to which you could add the query param string to use at GitHub in your webhook config for the repo.

I know that isn’t a very satisfying fix especially since you’ve written a better one, but that was the best we could come up with after 5 of my team all worked together on this, so maybe it’ll be better than nothing for now.

6

Thanks for the response and looking at my PR. Unfortunately build hooks would not fix our problems (automatic deploy previews does not seem to be possible if we always have to set up the hooks on every new branch), so we can only wait if we want to keep using Netlify. I do understand that testing takes time on your scale though.

7

Yeah, you’re right about build hooks not working as a workaround, if you build multiple branches. As far as I could tell you build only the production branch on this site (but, using build hooks will miss PR’s targeting it / the deploy previews we would create, as you mention, and I see you use those.)

1 Like
8

Yes, we’d like to have Deploy Previews enabled for all of the PR-s (had to switch this feature off because of the but). But thanks for letting me know about the build hooks anyways, maybe we’ll have to use it in the future for something else! :relaxed:

1 Like