Docs say Deploy Previews say preview will show result of merge, but it does not

Summary:

The docs say that a deploy preview will show you what the site will look like if the proposed changes were merged. But, the deploy preview does not change when changes happen on the production branch, even if the deploy preview is re-triggered manually or with additional commits on the PR branch. It makes me think the deploy preview is not showing the result of a merge at all, as the docs claim.

Details

The definitions page says

Deploy Preview : a deploy generated from a pull request or merge request, building the site as it would be if the proposed changes were merged

but this doesn’t seem to be true. In fact, a support agent told me:

The deploy preview shows what’s built when we build the deploy preview. When a PR is created, that’s the deploy preview. The merge will be the production build. Definitions can be found here.

The problem I had went like this

  1. create a commit in the production branch that causes builds to fail (for example, a syntax error in netlify.toml)
  2. create a PR (PR #1) building off of the production branch - modify some other code, not the netlify.toml that’s broken
  3. notice the deploy preview fails on PR #1, due to the error in netlify.toml
  4. create a separate PR (#2) fixing the netlify.toml error
  5. Merge PR #2 into the auto deploy branch. The production branch can now successfully deploy.
  6. Re-run the deploy preview for PR #1 - see the same failure as before, as if netlify.toml hasn’t been fixed in the production branch
  7. Add another commit to PR #1, which does not fix the netlify.toml but does something else
  • expectation: the code in the PR has changed, and the production branch has changed to be able to build, so the deploy preview should be able to run successfully this time
  • actual behavior: the deploy preview fails again, with the error message about your netlify.toml

The only way to get a successful deploy preview once you’re in this state is to merge the production branch into your PR #1 branch.

I’d expect the deploy preview to show me a preview of what will happen after a merge, not simply whatever is on the PR branch exactly, ignoring what’s on the production branch.

Hi, @clottman. Note, each deploy will build the site repo as it appears for the SHA1 commit reference specified for that deploy.

Can you link us to the deploy preview which isn’t working or send us the deploy id (the end of the URL for the deploy details page)? I don’t need the site name, just the deploy id will do.

Using that deploy id, I will be able to confirm exactly what commit ref was used for the build and deploy. This will hopefully clarify how deploy previews function at Netlify. If some error is occurring, knowing the deploy id where it happened will help us to find the issue.

I got my site to build, after addressing the broken build in a separate PR and then merging the production branch into my first PR, as I described. I’m interested now in the discrepancy in the docs between how the feature is described and how it actually works.

You say

each deploy will build the site repo as it appears for the SHA1 commit reference specified for that deploy.

But the definitions page says

Deploy Preview : a deploy generated from a pull request or merge request, building the site as it would be if the proposed changes were merged

Before I’ve merged a PR, there is no commit representing the merge that would appear in my repository. The docs say the PR is as if the change has been merged, but that’s not what happens, and in fact is not what you’re saying happens either.

hey @clottman, those are good questions! We are gonna do a little investigating into this, and let you know what we find. :smiley: More soon.

Hey @clottman,
I think there are a few confusing things here. The first is around your step 6:

Re-run the deploy preview for PR #1 - see the same failure as before, as if netlify.toml hasn’t been fixed in the production branch

If you use the retry deploy button in the UI, that will build the exact same commit that you pushed to create that specific Deploy Preview, even if there were changes to the production branch in the meantime. The PR is opened against a branch at a particular moment, and frozen at that commit. So you’re correct that you would need to rebase your PR branch against the production branch in order to pull in changes from the production branch.

Our deploy previews build what’s included in your PR- they don’t rebase against master before building. Does that help?

I also tried pushing a new commit to the PR I was trying to get a deploy preview for.

At this point I understand - exactly what is on the PR branch is what will build as the deploy preview. But, as I’ve pointed out a few times now, that is not what the deploy preview documentation says will happen. It says the deploy preview will show the result of merging the PR with the production branch.

All the support responses I’ve got so far are insisting that isn’t the case, and my experimental results back that up - which is fine, but it means the docs are incorrect.

That’s true! It took us some time to get here, but I get now that the issue is primarily about the docs. If you have a suggestion, I’d be happy to file an issue with our docs team. Maybe instead of

…building the site as it would be if the proposed changes were merged

I could suggest:

…building the site with the commits in your PR, before they have actually been merged

Does that seem like it’s on the right track?

Yep! That’s exactly what I’m getting at. Thanks! :slight_smile:

1 Like

Will do, thanks for raising and the discussion

The docs are still incorrect. Claiming that the site will be deployed as if the proposed changes were merged.

Hey there, @kiowa

Thank you so much for bumping this conversation back up. I have shared your notes with our Docs team.