Please read! End of support for Trusty build image: Everything you need to know

hello! the link that you have posted foes not work for me sadly. i have followed the instructions of this form as well as clicked the links in the email telling me to change the image. the last time i changed my website was in 2018 if that helps any. thanks

I ran into the same issue as @StevenTammen. Switching to the 16.04 image worked.

This part of the log in particular looks a bit weird:

7:15:17 PM: Required ruby-2.3.6 is not installed - installing.
7:15:17 PM: Searching for binary rubies, this might take some time.
7:15:17 PM: Found remote file https://rvm_io.global.ssl.fastly.net/binaries/ubuntu/20.04/x86_64/ruby-2.3.6.tar.bz2
7:15:17 PM: Checking requirements for ubuntu.
7:15:18 PM: Missing required packages: libssl1.0-dev
7:15:18 PM: RVM autolibs is now configured with mode '2' =>
7:15:18 PM:   'Allow RVM to use package manager if found, fail if dependencies are missing. This is default.',
7:15:18 PM: please run `rvm autolibs enable` to let RVM do its job or run and read `rvm autolibs [help]`
7:15:18 PM: or visit https://rvm.io/rvm/autolibs for more information.
7:15:18 PM: Found undesired packages: libssl-dev
7:15:18 PM: RVM autolibs is now configured with mode '2' =>
7:15:18 PM:   'Allow RVM to use package manager if found, fail if dependencies are missing. This is default.',
7:15:18 PM: please run `rvm autolibs enable` to let RVM do its job or run and read `rvm autolibs [help]`
7:15:18 PM: or visit https://rvm.io/rvm/autolibs for more information.
7:15:18 PM: Requirements installation failed with status: 1.

It seems to be erroring out because the OpenSSL 1.1.x headers are installed and it wants OpenSSL 1.0.x. It’s also trying to install Ruby 2.3.6, which is significantly older than the version that shipped in Ubuntu 20.04.

On a hunch, I tried updating my build config to add the following:

RUBY_VERSION = "2.7.0"

… which is the version of Ruby that shipped in 20.04. That seemed to get things building for me.

It seems a bit weird to have to configure a Ruby version when nothing in my site depends on Ruby. Perhaps the build script should default to a newer version of Ruby when using the 20.04 image?

1 Like

I can replicate. Adding ruby 2.7.0 to my build configuration also fixes the build errors for me and lets me run Focal. Huzzah.

[build]
publish = "public"
command = "hugo"

[context.production.environment]
RUBY_VERSION = "2.7.0"
HUGO_VERSION = "0.52"
HUGO_ENV = "production"
HUGO_ENABLEGITINFO = "true"

I’m just like you in wondering why in the world a ruby version dependency even matters for building a site that builds using a Hugo binary, but :man_shrugging:.

My experience now as a full-time software developer on tighter production schedules has led me to be much more accepting of not questioning things when they happen to work unexpectedly :rofl:.

1 Like

I switch to Ubuntu Focal 20.04 (beta) for my build image
(NODE VERSION : 12.18.1 / YARN VERSION : 1.22.4)
and when i try to deply again, i have this :
what am i doing wrong ?

6:17:25 PM: Build ready to start
6:17:27 PM: build-image version: 3802cb5fb68688f5cbd80f2e312aa0ce78813b3a
6:17:27 PM: build-image tag: v4.0.4
6:17:27 PM: buildbot version: 487f6d01b44481b4d57253f3d6071f198697110f
6:17:27 PM: Fetching cached dependencies
6:17:27 PM: Failed to fetch cache, continuing with build
6:17:27 PM: Starting to prepare the repo for build
6:17:27 PM: No cached dependencies found. Cloning fresh repo
6:17:27 PM: git clone https://github.com/yvesvinckier/rode-island-2
6:17:28 PM: Preparing Git Reference refs/heads/master
6:17:29 PM: Parsing package.json dependencies
6:17:29 PM: Different publish path detected, going to use the one specified in the Netlify configuration file: 'public' versus 'public/' in the Netlify UI
6:17:30 PM: Starting build script
6:17:30 PM: Installing dependencies
6:17:30 PM: Python version set to 2.7
6:17:31 PM: Downloading and installing node v12.18.1...
6:17:31 PM: Downloading https://nodejs.org/dist/v12.18.1/node-v12.18.1-linux-x64.tar.xz...
6:17:31 PM: Computing checksum with sha256sum
6:17:31 PM: Checksums matched!
6:17:35 PM: Now using node v12.18.1 (npm v6.14.5)
6:17:35 PM: Started restoring cached build plugins
6:17:35 PM: Finished restoring cached build plugins
6:17:35 PM: Attempting ruby version 2.3.6, read from environment
6:17:36 PM: Required ruby-2.3.6 is not installed - installing.
6:17:37 PM: Searching for binary rubies, this might take some time.
6:17:37 PM: Found remote file https://rvm_io.global.ssl.fastly.net/binaries/ubuntu/20.04/x86_64/ruby-2.3.6.tar.bz2
6:17:37 PM: Checking requirements for ubuntu.
6:17:38 PM: Missing required packages: libssl1.0-dev
6:17:38 PM: RVM autolibs is now configured with mode '2' =>
6:17:38 PM:   'Allow RVM to use package manager if found, fail if dependencies are missing. This is default.',
6:17:38 PM: please run `rvm autolibs enable` to let RVM do its job or run and read `rvm autolibs [help]`
6:17:38 PM: or visit https://rvm.io/rvm/autolibs for more information.
6:17:38 PM: Found undesired packages: libssl-dev
6:17:38 PM: RVM autolibs is now configured with mode '2' =>
6:17:38 PM:   'Allow RVM to use package manager if found, fail if dependencies are missing. This is default.',
6:17:38 PM: please run `rvm autolibs enable` to let RVM do its job or run and read `rvm autolibs [help]`
6:17:38 PM: or visit https://rvm.io/rvm/autolibs for more information.
6:17:38 PM: Requirements installation failed with status: 1.
6:17:38 PM: ruby-2.3.6 - #gemset created /opt/buildhome/.rvm/gems/ruby-2.3.6
6:17:38 PM: Required ruby-2.3.6 is not installed - installing.
6:17:38 PM: Searching for binary rubies, this might take some time.
6:17:39 PM: Found remote file https://rvm_io.global.ssl.fastly.net/binaries/ubuntu/20.04/x86_64/ruby-2.3.6.tar.bz2
6:17:39 PM: Checking requirements for ubuntu.
6:17:39 PM: Missing required packages: libssl1.0-dev
6:17:39 PM: RVM autolibs is now configured with mode '2' =>
6:17:39 PM:   'Allow RVM to use package manager if found, fail if dependencies are missing. This is default.',
6:17:39 PM: please run `rvm autolibs enable` to let RVM do its job or run and read `rvm autolibs [help]`
6:17:39 PM: or visit https://rvm.io/rvm/autolibs for more information.
6:17:39 PM: Found undesired packages: libssl-dev
6:17:39 PM: RVM autolibs is now configured with mode '2' =>
6:17:39 PM:   'Allow RVM to use package manager if found, fail if dependencies are missing. This is default.',
6:17:39 PM: please run `rvm autolibs enable` to let RVM do its job or run and read `rvm autolibs [help]`
6:17:39 PM: or visit https://rvm.io/rvm/autolibs for more information.
6:17:39 PM: Requirements installation failed with status: 1.
6:17:40 PM: ruby-2.3.6 - #importing gemsetfile /opt/buildhome/.rvm/gemsets/default.gems evaluated to empty gem list
6:17:40 PM: ruby-2.3.6 - #generating default wrappers.................
6:17:40 PM: Error running 'run_gem_wrappers regenerate',
6:17:40 PM: please read /opt/buildhome/.rvm/log/1626020260_ruby-2.3.6/gemset.wrappers.default.log
6:17:40 PM: Using /opt/buildhome/.rvm/gems/ruby-2.3.6
6:17:40 PM: Warning! Executable 'ruby' missing, something went wrong with this ruby installation!
6:17:40 PM: Warning! Executable 'gem' missing, something went wrong with this ruby installation!
6:17:40 PM: Warning! Executable 'irb' missing, something went wrong with this ruby installation!
6:17:40 PM: Using ruby version 2.3.6
6:17:40 PM: /opt/buildhome/.rvm/scripts/override_gem: line 19: gem: command not found
6:17:40 PM: Error installing bundler
6:17:40 PM: Build was terminated: Build script returned non-zero exit code: 1
6:17:40 PM: Creating deploy upload records
6:17:40 PM: Failing build: Failed to build site
6:17:40 PM: Failed during stage 'building site': Build script returned non-zero exit code: 1
6:17:40 PM: Finished processing build request in 13.321461287s

I switch to Ubuntu Xenial 16.04 (default) for my build image
(NODE VERSION : 12.18.1 / YARN VERSION : 1.22.4)
and it work perfectly fine… I don’t know why it failled when i switch to Ubuntu Focal 20.04 (beta)

This post seems to indicate a better fix than manually setting RUBY_VERSION in netlify.toml:

After deleting the explicit Ruby version pin, I was able to get my site building on the 20.04 image by following these steps:

  1. Under “Site settings > Build & Deploy > Continuous Deployment > Build image selection”, click “Edit Settings” and switch the image to “Ubuntu Focal 20.04 (beta)”.
  2. Under “Site settings > Build & Deploy > Continuous Deployment > Build settings”, click “Editing Settings” and then “Link to a different repository”. Choose the same repo and branch as before.

This reset all the hidden tool version pinnings to the defaults for the 20.04 image, and cut about 20 seconds off the site build time.

Maybe it would make sense to reset the pins when switching images? It would have saved a lot of debugging time.

1 Like

@jamesh - can confirm that that also worked for me, thanks. Looks like this is a bug on Netlify’s side, then.

2 Likes

Hi @anishkny,

Yes, it’s possible to do it in bulk, but you’d have to use Netlify API for that. Do note that, we don’t recommend this as there are chances your builds might break and also the API calls would be rate limited for excessive requests. But if you want to do it, you could do it like this (considering you’ve Node.js and npm installed):

Copy the following code in a file named index.js:

const fetch = require('node-fetch')
const NetlifyAPI = require('netlify')
const AccessToken = 'Access Token Here'
const buildImage = '"xenial"' // xenial for default, focal for latest
const sitesToChange = ['netlify-subdomain1', 'netlify-subdomain2'] // leave blank to change all websites using Trusty

function manageChange(site) {
  new Promise(resolve => {
    fetch(`https://api.netlify.com/api/v1/sites/${site.id}`, {
      method: 'PUT',
      body: `{"build_image": ${buildImage}}`,
      headers: {
        Authorization: `Bearer ${AccessToken}`,
        'Content-Type': 'application/json'
      }
    }).then(response => {
        if (response.ok) {
          return response.json()
        } else {
          throw response.statusText
        }
      }).then(() => {
        console.log(`Build image for ${site.name} is now set to ${buildImage}`)
        resolve(site)
      }).catch(error => {
        console.log(error)
      })
  })
}

new NetlifyAPI(AccessToken).listSites().then(sites => {
  sites.forEach(site => {
    if (sitesToChange.length > 0) {
      if (sitesToChange.includes(site.name)) {
        manageChange(site)
      }
    } else if (site.build_image == 'production') {
      manageChange(site)
    }
  })
})

Run npm i netlify node-fetch.

There are two ways you could use this:

  1. Copy each website’s name in the sitesToChange array. If that array has even 1 website, only that would be changed. Rest would be left untouched.

  2. If you wish to change the build image of all websites in your account using Trusty, you should leave that array empty.

You could set your build image to xenial or focal by changing the buildImage constant. Do not remove the " ".

Finally, you need to set the AccessToken constant. You can get its value by going here: Netlify App. Generate a new access token and use its value there. That access token would allow anyone to use API to act on your behalf. So, keep it safe or delete it once done.

Once all this is done, just run node index.js and the script will do its job. It might not be the most optimum code, but it works.

Do let us know if you run into any errors.

2 Likes

@jamesh - thank you! the post you shared solved my build issue too

Thanks for sharing your experiences! We’re working on a fix for the issue with Ruby v2.3.6 failing to install.

I’ve also added a few notes to the last section of the migration guide (at the top of this thread) about the fact that versions can be set by default at site creation, and how to reset the defaults by re-linking the site repo.

So why do we set versions for languages a site may not even use? Because it’s not always possible to know for sure that a language isn’t being used, and our priority is to keep builds running as they always have, even if we change a default behind the scenes.

2 Likes

I think that’s perfectly logical. With this being said, I’m guessing most folks will be perfectly fine bumping the default versions if they aren’t using the languages in their builds. (At least I am, N=1).

I believe the observations @jamesh made are a bit more than this though. If (old) default versions are set behind-the-scenes, and the new build image (which now has much more up-to-date defaults) has to install the old versions when building, then does that not add a good bit of time to every build? He noted he shaved a full 20 seconds off his build. If his sites are anything like mine, that’s a huge part of the build time (perhaps even most of it)! Also, if there is ever an issue installing an old version (as with this Ruby issue), then that is actually a point of failure that would not otherwise be there, right?

Scaled across many customers, I think this could end up affecting build server load by a not insignificant amount. I don’t know what the ultimate solution is (old dependencies are always painful like this), but I think the net impact is large enough it might be worth looking into.

I guess what I’m trying to say is that I see a good case for people bumping the defaults when updating build image even if their build isn’t erroring out.

What do you think?

1 Like

It’s more that this is hidden behaviour. I’d expect anything that I didn’t pin to use the default version of the tool installed into the Docker image. If I wanted something newer, I could manually pin it or wholesale switch to a newer image.

Maybe when you only had the Ubuntu 14.04 based image available for use, it might have made sense to continuously upgrade the default tool versions and downgrade for older customers. If you are providing a choice of images, perhaps it would be better to keep each image stable.

At a very minimum, it would probably be good to undo the pins when switching build images, the same as happens when switching repositories.

1 Like

Hi, my workplace has a site on Trusty, I’m trying to migrate it to Focal. The build itself runs okay using some of the workarounds above, however, the production site crashes on the client-side due to a “Backbone is not defined” error. This has never occurred before, and the site still builds and runs perfectly on a local server. It uses Node v12.16.1 and Backbone v1.1.2. Any ideas as why this Backbone error would be appearing?

Hi @brodysmith,

Could we have a website to check this?

I am reading the email and all of these threads and am 100% lost. I have skills in coding but what you are discussing here and are expecting us to understand is past them. I never set anything up on LInux for my site. I don’t have a clue where to find the respective log you want us to check, nor even if I found it what the action to move forward without interruption is. Why did a very simple platform suddenly require a dual-PhD for use? I’m not trying to be a jerk, I have spent 2 hours trying to understand what this is all about and am 0% closer to resolution.

Hi @SCloudcroft,

If you can share what your problem is, we can help.

1 Like

@SCloudcroft I went ahead and recorded a YouTube tutorial when I re-bumped my versions just now. Perhaps this might help you and any other folks who are a little lost:

Let us know if you still have questions after this.

2 Likes

@StevenTammen, what an excellent resource! Thank you so much for taking the time to record that video and share it with the Forums :slight_smile:

Hey, thanks Steven. The icon on my profile is a cat ~I’m female. I got to 2:53 Build Image Deployment and my options on my site diverge from yours. I do not have that on my screen anywhere. I don’t have my site on GitHub. Is that why?

Hi @SCloudcroft,

If you mean you’ve sites that are manually deployed (not connected to any Git repo), you won’t have those options.