Netlify creating random changes to Hugo site on deploy (missing content)

Support
, ,
1

Netlify site name: https://michaelmccarthy.netlify.app
Site GitHub repo: GitHub - mccarthy-m-g/michaelmccarthy: A place for myself
Hugo theme repo: GitHub - hugo-apero/hugo-apero: Apéro is a Hugo theme for personal websites. A Hugo theme you'll want to hang out with . This is the source for the theme files to install.

The problem

My Hugo site has been displaying a strange bug lately where random changes are being created to site content on deploy. Specifically, some content is not appearing on a page where it should. The behaviour is random in that if I manually trigger a deploy afterwards using “retry with latest branch commit” or “clear cache and retry with latest branch commit”, the content that did not appear on the first deploy will appear, but different content will disappear. I’ve found if I manually trigger a deploy enough times, eventually all the content that is supposed to be there will be.

This behaviour does not happen locally, so something must be happening on Netlify’s side. My local Hugo version and the one being used by Netlify are the same.

Any support is greatly appreciated, as I’m not sure how to debug this.

Additional details

In the Hugo theme I’m using, the content that is (dis)appearing is controlled by a boolean in the YAML for the page in question. The accompany partial (with the relevant lines highlighted) is here.

My guess is that something is occurring on Netlify’s side that randomly sets this boolean to false for some pages when ranging over them, resulting in the HTML code with the missing content not being included in the deploy.

I’ve downloaded two successive deploys where this bug has occurred, which support this hypothesis. Unfortunately I’m a new user here so I can’t upload the files, but here’s what’s happening in the HTML code:

On the first deploy (deploy-63827b35538dbe0009a3cd09), the following HTML code is absent on line 288 of project/index.html:

<div class="ph0 ph2-m ph3-l">

    <div class="aspect-ratio aspect-ratio--1x1 contain dim"style="background: url('https://michaelmccarthy.tidytales.ca/project/tidytales/featured-hex.png') center center no-repeat"><a href="https://michaelmccarthy.tidytales.ca/project/tidytales/" class="w-100 absolute top-0 bottom-0"></a></div>
  </div>

On the second deploy (deploy-6382b68948c6557349f791b3), the HTML code above is present where it should be, but the following is absent on line 192 of project/index.html (even though this code was present in the first deploy):

<div class="ph0 ph2-m ph3-l">

    <div class="aspect-ratio aspect-ratio--1x1 contain dim"style="background: url('https://michaelmccarthy.tidytales.ca/project/psyc-615-lab/featured-hex.png') center center no-repeat"><a href="https://michaelmccarthy.tidytales.ca/project/psyc-615-lab/" class="w-100 absolute top-0 bottom-0"></a></div>
  </div>

For these deploys, this is the missing content in question that is controlled by the YAML boolean. To reiterate, nothing has changed between these two deploys; all I did was manually redeploy. The behaviour is random. If I manually redeploy enough times, eventually I land on a deploy where no HTML code is missing (and thus the content appears as it should).

Deploy logs

Deploy log 1 (deploy-63827b35538dbe0009a3cd09)

12:46:47 PM: build-image version: 4c0c1cadee6a31c9bb8d824514030009c4c05c6a (focal)
12:46:47 PM: build-image tag: v4.15.0
12:46:47 PM: buildbot version: 8e21f3a6f94b4bf664f0a5ce69b751fdfc9791bd
12:46:47 PM: Fetching cached dependencies
12:46:47 PM: Starting to download cache of 294.9MB
12:46:49 PM: Finished downloading cache in 2.269531075s
12:46:49 PM: Starting to extract cache
12:46:50 PM: Finished extracting cache in 1.132754319s
12:46:50 PM: Finished fetching cache in 3.485783767s
12:46:50 PM: Starting to prepare the repo for build
12:46:51 PM: Preparing Git Reference refs/heads/main
12:46:51 PM: Parsing package.json dependencies
12:46:52 PM: Section completed: initializing
12:46:52 PM: Starting build script
12:46:52 PM: Installing dependencies
12:46:52 PM: Python version set to 2.7
12:46:52 PM: Started restoring cached node version
12:46:53 PM: Finished restoring cached node version
12:46:53 PM: v16.18.1 is already installed.
12:46:54 PM: Now using node v16.18.1 (npm v8.19.2)
12:46:54 PM: Enabling node corepack
12:46:54 PM: Started restoring cached build plugins
12:46:54 PM: Finished restoring cached build plugins
12:46:54 PM: Attempting ruby version 2.7.2, read from environment
12:46:54 PM: Using ruby version 2.7.2
12:46:55 PM: Using PHP version 8.0
12:46:55 PM: Installing Hugo 0.89.2
12:46:55 PM: hugo v0.89.2-63E3A5EB+extended linux/amd64 BuildDate=2021-11-08T15:22:24Z VendorInfo=gohugoio
12:46:55 PM: Started restoring cached go cache
12:46:55 PM: Finished restoring cached go cache
12:46:55 PM: Installing Go version 1.16.5 (requested 1.16.5)
12:46:59 PM: unset GOOS;
12:46:59 PM: unset GOARCH;
12:46:59 PM: export GOROOT='/opt/buildhome/.gimme/versions/go1.16.5.linux.amd64';
12:46:59 PM: export PATH="/opt/buildhome/.gimme/versions/go1.16.5.linux.amd64/bin:${PATH}";
12:46:59 PM: go version >&2;
12:46:59 PM: export GIMME_ENV="/opt/buildhome/.gimme/env/go1.16.5.linux.amd64.env"
12:46:59 PM: go version go1.16.5 linux/amd64
12:47:00 PM: Detected 0 framework(s)
12:47:00 PM: Installing missing commands
12:47:00 PM: Verify run directory
12:47:01 PM: ​
12:47:01 PM: ────────────────────────────────────────────────────────────────
12:47:01 PM:   Netlify Build                                                 
12:47:01 PM: ────────────────────────────────────────────────────────────────
12:47:01 PM: ​
12:47:01 PM: ❯ Version
12:47:01 PM:   @netlify/build 28.4.0
12:47:01 PM: ​
12:47:01 PM: ❯ Flags
12:47:01 PM:   baseRelDir: true
12:47:01 PM:   buildId: 63827b35538dbe0009a3cd07
12:47:01 PM:   deployId: 63827b35538dbe0009a3cd09
12:47:01 PM: Creating deploy upload records
12:47:01 PM: ​
12:47:01 PM: ❯ Current directory
12:47:01 PM:   /opt/build/repo
12:47:01 PM: ​
12:47:01 PM: ❯ Config file
12:47:01 PM:   /opt/build/repo/netlify.toml
12:47:01 PM: ​
12:47:01 PM: ❯ Context
12:47:01 PM:   production
12:47:01 PM: ​
12:47:01 PM: ────────────────────────────────────────────────────────────────
12:47:01 PM:   1. build.command from netlify.toml                            
12:47:01 PM: ────────────────────────────────────────────────────────────────
12:47:01 PM: ​
12:47:01 PM: $ hugo
12:47:01 PM: Start building sites …
12:47:01 PM: hugo v0.89.2-63E3A5EB+extended linux/amd64 BuildDate=2021-11-08T15:22:24Z VendorInfo=gohugoio
12:47:01 PM:                    | EN
12:47:01 PM: -------------------+-----
12:47:01 PM:   Pages            | 68
12:47:01 PM:   Paginator pages  |  0
12:47:01 PM:   Non-page files   | 24
12:47:01 PM:   Static files     | 98
12:47:01 PM:   Processed images |  0
12:47:01 PM:   Aliases          |  3
12:47:01 PM:   Sitemaps         |  1
12:47:01 PM:   Cleaned          |  0
12:47:01 PM: Total in 231 ms
12:47:01 PM: ​
12:47:01 PM: (build.command completed in 264ms)
12:47:01 PM: ​
12:47:01 PM: ────────────────────────────────────────────────────────────────
12:47:01 PM:   2. Deploy site                                                
12:47:03 PM: Starting post processing
12:47:01 PM: ────────────────────────────────────────────────────────────────
12:47:01 PM: ​
12:47:01 PM: Starting to deploy site from 'public'
12:47:01 PM: Creating deploy tree 
12:47:02 PM: 45 new files to upload
12:47:03 PM: Post processing - HTML
12:47:02 PM: 0 new functions to upload
12:47:02 PM: Section completed: deploying
12:47:02 PM: Site deploy was successfully initiated
12:47:02 PM: ​
12:47:02 PM: (Deploy site completed in 1.1s)
12:47:02 PM: ​
12:47:02 PM: ────────────────────────────────────────────────────────────────
12:47:02 PM:   Netlify Build Complete                                        
12:47:02 PM: ────────────────────────────────────────────────────────────────
12:47:02 PM: ​
12:47:02 PM: (Netlify Build completed in 1.4s)
12:47:03 PM: Caching artifacts
12:47:03 PM: Started saving build plugins
12:47:03 PM: Finished saving build plugins
12:47:03 PM: Started saving pip cache
12:47:03 PM: Finished saving pip cache
12:47:03 PM: Started saving emacs cask dependencies
12:47:03 PM: Finished saving emacs cask dependencies
12:47:03 PM: Started saving maven dependencies
12:47:03 PM: Finished saving maven dependencies
12:47:03 PM: Started saving boot dependencies
12:47:03 PM: Finished saving boot dependencies
12:47:03 PM: Started saving rust rustup cache
12:47:03 PM: Finished saving rust rustup cache
12:47:03 PM: Started saving go dependencies
12:47:03 PM: Finished saving go dependencies
12:47:03 PM: Build script success
12:47:03 PM: Section completed: building
12:47:04 PM: Uploading Cache of size 294.7MB
12:47:05 PM: Post processing - header rules
12:47:05 PM: Post processing - redirect rules
12:47:05 PM: Post processing done
12:47:05 PM: Section completed: postprocessing
12:47:06 PM: Section completed: cleanup
12:47:06 PM: Finished processing build request in 19.397010138s
12:47:08 PM: Site is live ✨

Deploy log 2 (deploy-6382b68948c6557349f791b3)

4:59:54 PM: build-image version: 4c0c1cadee6a31c9bb8d824514030009c4c05c6a (focal)
4:59:54 PM: build-image tag: v4.15.0
4:59:54 PM: buildbot version: 8e21f3a6f94b4bf664f0a5ce69b751fdfc9791bd
4:59:54 PM: Building without cache
4:59:54 PM: Starting to prepare the repo for build
4:59:54 PM: No cached dependencies found. Cloning fresh repo
4:59:54 PM: git clone https://github.com/mccarthy-m-g/michaelmccarthy
4:59:56 PM: Preparing Git Reference refs/heads/main
4:59:57 PM: Parsing package.json dependencies
4:59:57 PM: Section completed: initializing
4:59:58 PM: Starting build script
4:59:58 PM: Installing dependencies
4:59:58 PM: Python version set to 2.7
4:59:58 PM: v16.18.1 is already installed.
4:59:58 PM: Now using node v16.18.1 (npm v8.19.2)
4:59:58 PM: Enabling node corepack
4:59:58 PM: Started restoring cached build plugins
4:59:58 PM: Finished restoring cached build plugins
4:59:58 PM: Attempting ruby version 2.7.2, read from environment
4:59:59 PM: Using ruby version 2.7.2
4:59:59 PM: Using PHP version 8.0
4:59:59 PM: Installing Hugo 0.89.2
5:00:00 PM: hugo v0.89.2-63E3A5EB+extended linux/amd64 BuildDate=2021-11-08T15:22:24Z VendorInfo=gohugoio
5:00:00 PM: Started restoring cached go cache
5:00:00 PM: Finished restoring cached go cache
5:00:00 PM: Installing Go version 1.16.5 (requested 1.16.5)
5:00:04 PM: unset GOOS;
5:00:04 PM: unset GOARCH;
5:00:04 PM: export GOROOT='/opt/buildhome/.gimme/versions/go1.16.5.linux.amd64';
5:00:04 PM: export PATH="/opt/buildhome/.gimme/versions/go1.16.5.linux.amd64/bin:${PATH}";
5:00:04 PM: go version >&2;
5:00:04 PM: export GIMME_ENV="/opt/buildhome/.gimme/env/go1.16.5.linux.amd64.env"
5:00:04 PM: go version go1.16.5 linux/amd64
5:00:04 PM: Detected 0 framework(s)
5:00:04 PM: Installing missing commands
5:00:04 PM: Verify run directory
5:00:06 PM: ​
5:00:06 PM: ────────────────────────────────────────────────────────────────
5:00:06 PM:   Netlify Build                                                 
5:00:06 PM: ────────────────────────────────────────────────────────────────
5:00:06 PM: ​
5:00:06 PM: ❯ Version
5:00:06 PM:   @netlify/build 28.4.0
5:00:06 PM: ​
5:00:06 PM: ❯ Flags
5:00:06 PM:   baseRelDir: true
5:00:06 PM:   buildId: 6382b68948c6557349f791b1
5:00:06 PM:   deployId: 6382b68948c6557349f791b3
5:00:06 PM: ​
5:00:06 PM: Creating deploy upload records
5:00:06 PM: ❯ Current directory
5:00:06 PM:   /opt/build/repo
5:00:06 PM: ​
5:00:06 PM: ❯ Config file
5:00:06 PM:   /opt/build/repo/netlify.toml
5:00:06 PM: ​
5:00:06 PM: ❯ Context
5:00:06 PM:   production
5:00:06 PM: ​
5:00:06 PM: ────────────────────────────────────────────────────────────────
5:00:06 PM:   1. build.command from netlify.toml                            
5:00:06 PM: ────────────────────────────────────────────────────────────────
5:00:06 PM: ​
5:00:06 PM: $ hugo
5:00:07 PM: Starting post processing
5:00:06 PM: Start building sites …
5:00:06 PM: hugo v0.89.2-63E3A5EB+extended linux/amd64 BuildDate=2021-11-08T15:22:24Z VendorInfo=gohugoio
5:00:06 PM:                    | EN
5:00:07 PM: Post processing - HTML
5:00:06 PM: -------------------+-----
5:00:06 PM:   Pages            | 68
5:00:06 PM:   Paginator pages  |  0
5:00:06 PM:   Non-page files   | 24
5:00:06 PM:   Static files     | 98
5:00:06 PM:   Processed images |  0
5:00:06 PM:   Aliases          |  3
5:00:06 PM:   Sitemaps         |  1
5:00:06 PM:   Cleaned          |  0
5:00:06 PM: Total in 241 ms
5:00:06 PM: ​
5:00:06 PM: (build.command completed in 271ms)
5:00:06 PM: ​
5:00:06 PM: ────────────────────────────────────────────────────────────────
5:00:06 PM:   2. Deploy site                                                
5:00:06 PM: ────────────────────────────────────────────────────────────────
5:00:06 PM: ​
5:00:06 PM: Starting to deploy site from 'public'
5:00:06 PM: Creating deploy tree 
5:00:06 PM: 4 new files to upload
5:00:06 PM: 0 new functions to upload
5:00:06 PM: Section completed: deploying
5:00:06 PM: Site deploy was successfully initiated
5:00:06 PM: ​
5:00:06 PM: (Deploy site completed in 549ms)
5:00:06 PM: ​
5:00:06 PM: ────────────────────────────────────────────────────────────────
5:00:06 PM:   Netlify Build Complete                                        
5:00:06 PM: ────────────────────────────────────────────────────────────────
5:00:06 PM: ​
5:00:06 PM: (Netlify Build completed in 840ms)
5:00:07 PM: Caching artifacts
5:00:08 PM: Post processing - header rules
5:00:07 PM: Started saving build plugins
5:00:07 PM: Finished saving build plugins
5:00:08 PM: Post processing - redirect rules
5:00:07 PM: Started saving pip cache
5:00:07 PM: Finished saving pip cache
5:00:07 PM: Started saving emacs cask dependencies
5:00:08 PM: Post processing done
5:00:07 PM: Finished saving emacs cask dependencies
5:00:07 PM: Started saving maven dependencies
5:00:07 PM: Finished saving maven dependencies
5:00:08 PM: Section completed: postprocessing
5:00:07 PM: Started saving boot dependencies
5:00:07 PM: Finished saving boot dependencies
5:00:07 PM: Started saving rust rustup cache
5:00:07 PM: Finished saving rust rustup cache
5:00:07 PM: Started saving go dependencies
5:00:07 PM: Finished saving go dependencies
5:00:08 PM: Build script success
5:00:08 PM: Section completed: building
5:00:09 PM: Uploading Cache of size 232.8MB
5:00:11 PM: Section completed: cleanup
5:00:11 PM: Finished processing build request in 16.642046768s
5:00:11 PM: Site is live ✨

Workarounds

As an aside, I know I can rewrite the partial in question to just include the HTML code without any If statement. I likely will as a stopgap measure, but would still like support to solve the bug at hand.

2

Appreciate the detailed post @mccarthy-m-g, however I’m really not sure why this would be something that Netlify would be causing. There’s no reason for Netlify to flip a boolean value for you. You might think this could be a bug, but a bug that’s flipping a boolean value on a particular Hugo site, seems too specific to be occurring.

I’m willing to try this on my end though, to see if I can replicate it. After answering some other pending cases, I’ll take a look a your repo. Additionally, can you try using a different Hugo version? Maybe it’s a bug in Hugo? It’s also worth trying this on a Ubuntu machine. Maybe, it’s a bug specifically in Hugo on Ubuntu?

3

Thank you, Hrishikesh!

So I did a little more testing: It may not be flipping a boolean value, but the alternative is that If statement associated with the boolean is being evaluated as false randomly. This seems more likely, but still shouldn’t be happening.

I know you can’t evaluate third-party code here, but just for clarity (and for others following this issue), here’s the code in question. I’ve added the comment "THIS IF STATEMENT IS BEING EVALUATED FALSE RANDOMLY " before the relevant lines. The HTML and Hugo code within each If statement is just controlling whether or not a certain portion of content appears for a given listing.

Edit after a little more testing: The if statements are evaluating as false because $root.Params.show_post_thumbnail is coming back empty for random pages.

{{ $page := . }}                           <!--save current page-->
{{ $section := $page.CurrentSection }}     <!--save branch section-->
{{ $root := .Scratch.Get "$root" }}        <!--save root section-->

<article class="w-100 w-50-m w-third-l mb4 pb4 bb bw1 bw0-ns pb0-ns">
  <!--if show_post_thumbnail is TRUE in root-->
  <!-- THIS IF STATEMENT IS BEING EVALUATED FALSE RANDOMLY -->
  {{ if $root.Params.show_post_thumbnail }} 
  <div class="ph0 ph2-m ph3-l">
    <!--featured image for the leaf bundle-->
    {{- $images := $page.Resources.ByType "image" -}}
    {{ $leaf := $images.GetMatch "*feature*" }}
    {{- $hex := $images.GetMatch "*hex*" -}}
    {{- $thumbnail := cond (ne $hex nil) $hex $leaf -}}
    <!--sidebar image for the branch bundle-->
    {{ $branch := ($section.Resources.ByType "image").GetMatch "*feature*" }}
    {{- $thumbnail := cond (ne $thumbnail nil) $thumbnail $branch -}}
    <div class="aspect-ratio aspect-ratio--1x1 {{ if (ne $hex nil) }}contain{{ else }}cover{{ end }} dim"{{ with $thumbnail }}style="background: url('{{ .RelPermalink | absURL }}') center center no-repeat"{{ end }}><a href="{{ .Permalink }}" class="w-100 absolute top-0 bottom-0"></a></div>
  </div>
  {{ end }}
  <div class="ph0 pa2-m pa3-l">
    <header>
      <h1 class="f3 f4-l mv2 lh-title fw4"><a href="{{ .RelPermalink }}" class="db">{{ .Title | markdownify }}</a></h1>
      {{ range first 1 (.GetTerms "series") }}
        <h2 class="f5"><i class="fas fa-stream fa-fw pr1"></i><small class="f5 ttu tracked">Series: </small><a class="link i dim" href="{{ $section.RelPermalink }}">{{ .LinkTitle }}</a></h2>
      {{ end }}
    </header>
      {{ $summary := cond (ne .Params.excerpt nil) .Params.excerpt .Summary }}
      {{ with $summary }}<p class="f6 lh-copy">{{ . | markdownify | emojify }}</p>{{ end }}
    <!--ensures section Params are respected-->
    <footer>
      <!-- THIS IF STATEMENT IS BEING EVALUATED FALSE RANDOMLY -->
      {{ if $root.Params.show_author_byline }}<p class="f7 db mv1">{{ if .Params.author }}By {{ .Params.author }}{{ end }}{{ with .Params.categories }} in{{ range . }} <a href="{{ "categories/" | absURL }}{{ . | urlize }}">{{ . }}</a> {{ end }}{{ end }}</p>{{ end }}
      <!-- THIS IF STATEMENT IS BEING EVALUATED FALSE RANDOMLY -->
      {{ if $root.Params.show_post_date }}<p class="f7 db mv1">{{ .PublishDate.Format "January 2, 2006" }}</p>{{ end }}
      <!-- THIS IF STATEMENT IS BEING EVALUATED FALSE RANDOMLY -->
      {{ if and $root.Params.show_button_links .Params.links }}
          <p class="db pt1">
          {{ partial "shared/btn-links.html" . }}
          </p>
      {{ end }}
    </footer>
  </div>
</article>

I don’t think the Hugo version is the issue–other users of this Hugo theme reported having the same issues on Netlify with the most recent version of Hugo.

I’ve set up a new GitHub repo to try to troubleshoot this issue with Ubuntu: GitHub - mccarthy-m-g/hugo-apero-troubleshooting: Temporary repo for troubleshooting a bug with Hugo Apero and Netlify. In the repo, I set up a GitHub Actions workflow using Ubuntu 20.04 (same version as Netlify) to build and deploy the site to GitHub pages: Projects with Impact | Michael McCarthy. I also set up a Netlify build and deploy from the same repo: Projects with Impact | Michael McCarthy. Commits to the main branch will cause a build and deploy to GitHub pages and Netlify. I’ve set the appropriate base URL for each site too, so that shouldn’t be a cause for concern.

If you compare the two deploy pages linked above, you can see content is missing from the “Tidy Tales” and “embedr” listing on the Netlify site, but not the GitHub pages site. Since the behaviour is random, I plan to redeploy both sites a number of times to rule out getting lucky with the GitHub pages site; but I will wait to do that after you have had a chance to look. If it turns out the behaviour is only happening on the Netlify deploy we can probably rule out Ubuntu as the cause.

4

Thank you for investigating this further. I’m not sure what could be happening, or what even to look for.

While I can see the problem happening on Netlify, I still can’t think of what part of Netlify could be causing this issue. As far as Netlify is concerned, it’s not doing anything special. Downloading the Hugo binary available on GitHub, running it, publishing the built files. So the site is still built by hugo.

Have you tried asking on Hugo forums to at least try to make sense what part could be responsible to cause such a problem?