Python support needs to improve

Features
1

Hi, first of all, I want to say that I am a fan of Netlify’s product, so this is a friendly rant. I’m writing to describe a failure, and to encourage the business to try to support my needs as a potential customer.

I am a jack-of-all-trades consultant and have used Netlify to deliver several small proof-of-concept websites to clients. I’d like to think that one of these will turn into a paying Netlify customer sometime soon, but who knows.

Until now I have used Netlify in a very basic way: generating HTML from data, checking it in, and pushing. My current client would like to be able to make changes to the input data, and I thought that by configuring a build script he could simply make commits directly in GitHub and see the results. Sadly, I have run aground with Netlify’s poor support for recent python versions.

The short story is, I am using python3.8. This does not seem to be supported; the docs say 3.5 and 3.7 only. I thought I’d take a stab at downgrading my needs to 3.7, so I tried to “make my own sausage” following How Our Build Bots Build Sites | Netlify. When I run the xenial image in docker, I instead see versions 2.7, 3.4, 3.5, 3.6.

At this point, I’m about to give up. I have also previously passed on Netlify for lack of python lambda functions.

In summary, I want to like the product but it is falling short. To win my customers, you need to support ALL of the major python releases, and you need to provide step-by-step documentation for running docker images locally so that I can debug version and pip install dependency problems.

Again, I hope this feedback is constructive. I know that it’s always harder than it looks from the outside, and I admire the product you have built. Keep going!

Thank you,
-George

2

Hi @gwk, your feedback is absolutely constructive and welcome!

Since the build-image we use in our build environment is open source, you can definitely create a PR to add support for 3.8. Actually, there’s an feature request open here and someone created a PR here: Add support for PYTHON_VERSION env variable. by duckinator · Pull Request #363 · netlify/build-image · GitHub. Feel free to chime in on the issue and/or PR.

3

Thanks Dennis. I’d be willing to look into this but what I don’t understand is whether python versions that are not present in the xenial image are actually supported at all.

I first ran docker pull netlify/build:xenial. How do I tell what version of the build-image git repo this corresponds to? It seems somewhat behind.

When I run the xenial image in docker using start-image.sh, I see the following python versions in /usr/bin:

  • Python 2.7.6
  • Python 3.4.3
  • Python 3.5.6
  • Python 3.6.7

Contrast this with what is written in included_software.md:

  • 2.7 (default)
  • 3.5
  • 3.7

So, when somebody specifies 3.7 in runtime.txt, (or PYTHON_VERSION) in the future, is it causing that version to get installed at build time? or is that version hiding somewhere that is not in the default path on the xenial image? Or is 3.7 not actually working?

Looking at the dockerfile, I see that apt-get installs python, python3, and python3.7. I presume this is what included_software.md is trying to reflect, but again, it does not seem to match what I see in /usr/bin.

My guess is that netlify/build:xenial is behind the state of the git branch xenial that I’m looking at. A little guidance here would be very helpful. Thanks!

4

Hiya @gwk and sorry for the suboptimal experience! As you may infer, not many folks use python in our build network (compared with Ruby, and Go - but all other languages are at least an order of magnitude less popular than Node.js!)

We do not do any auto-installation of python as we do with Ruby or Node - what you see is what you get.

It is quite possible that we have some work in progress in the branch that isn’t live in our build network; I’ve put an ask out to our team to confirm/deny.

Re: the different advice in included_software - we put the major versions rather than the minor versions to allow upgrades in case 3.6.7 has some horrible security problem prompting the release of 3.6.8 which we’d want to put in place without breaking scripts.

Now - the entirely missing 3.7 feels like something odd, so I did check with the team and they don’t think there is any major desync between what’s live and what is declared. While I can’t speak to what’s in /usr/bin, we install the pythons in /opt/buildhome, and here is what is there in production, right now:

2:51:05 PM: Executing user command: ls -l /opt/buildhome
2:51:06 PM: total 12
2:51:06 PM: drwxr-xr-x 7 buildbot nogroup 4096 Nov 22 20:19 python2.7
2:51:06 PM: lrwxrwxrwx 1 buildbot nogroup 24 Nov 22 20:19 python2.7.11 -> /opt/buildhome/python2.7
2:51:06 PM: drwxr-xr-x 6 buildbot nogroup 4096 Nov 22 20:19 python3.5
2:51:06 PM: lrwxrwxrwx 1 buildbot nogroup 24 Nov 22 20:19 python3.5.6 -> /opt/buildhome/python3.5
2:51:06 PM: drwxr-xr-x 6 buildbot nogroup 4096 Nov 22 20:19 python3.7
2:51:06 PM: lrwxrwxrwx 1 buildbot nogroup 24 Nov 22 20:19 python3.7.2 -> /opt/buildhome/python3.7

…so that seems to match up ok with our docs :slight_smile:

5

I figured out where the misunderstanding is:

$ docker run --rm --tty --interactive netlify/build:xenial /bin/bash
buildbot@9db34dc599d9:/$ ls /opt/buildhome/
python2.7 python2.7.11 python3.5 python3.5.6 python3.7 python3.7.2
buildbot@9db34dc599d9:/$ exit

$ docker run --rm --tty --interactive netlify/build /bin/bash
buildbot@a6087d395f0f:/$ ls /opt/buildhome/
python2.7 python2.7.5 python3.4 python3.4.0 python3.5 python3.5.5 python3.6 python3.6.4

I thin the instructions in “Want to try to make your own sausage?” here are incorrect: https://www.netlify.com/blog/2016/10/18/how-our-build-bots-build-sites/.

The reason is that despite pulling netlify/build:xenial, when I run your ./test-tools/start-image.sh, it runs netlify/build which is not the same thing.

My guess is that ./test-tools/start-image.sh is out of date in that it just presumes the legacy build image. It’s worth asking how come nobody noticed this; are engineers not actually using start-image.sh, or not using xenial, or not using the combination?

8

Hi @gwk! Thanks for uncovering this unclear spot in our docs. I use the start-image.sh script regularly, so I looked into what you described. It looks like we weren’t clear enough in our docs on how to use the test script. As it turns out, start-image.sh currently only works properly if you’ve built the image locally already using docker build -t netlify/build . in the root of the build-image repo. The reason for this is, as you pointed out, that Docker is not using xenial by default - instead, it uses the latest tag if no tag is specified (as in start-image.sh), which works fine if you’ve built the image locally to that tag, but does not work correctly if you haven’t. As you can see at the Docker Hub page for netlify/build, we haven’t pushed an image to the latest tag in over a year. This can be rectified immediately by modifying start-image.sh to use netlify/build:xenial instead of just netlify/build. Furthermore, here’s what I’m planning to do to improve the situation:

  • I’m going to fix the build-image’s Jenkinsfile so we always push the current release of the xenial branch to latest.
  • I’m going to allow passing a tag to start-image.sh either through an environment variable or another CLI argument and have the script start that Docker tag. It will default to xenial or latest (but latest will now be kept up-to-date).
  • I’m going to add two scripts, start-xenial.sh and start-trusty.sh, that pull the specific Docker images netlify/build:xenial and netlify/build:trusty from Docker Hub and run them using start-image.sh. These will allow easy testing of the current images without having to build them locally.
  • I’m going to document all this in the build-image’s readme file so it’s easier for people to figure out how to use the test tools.

I’m also looking into the issues you’ve brought up with our Python support. I’ve already double-checked that Python 3.7 is installed and available in the current netlify/build:xenial image, and I’m going to look into the feasibility and image size impact of including Python 3.8.

1 Like
9

@benaiah, thank you for the detailed response. This is all very encouraging. A few small points:

  • I got this far by following the “make your own sausage” instructions (see original post); I strongly suggest you update those docs.
  • Regarding Python support, in general I think you should aim to support 3.5, 3.6, 3.7, and 3.8. If you can have 3.5 and 3.7 simultaneously in the image, then you might as well have the other versions as well. This could be the difference between a painless or painful setup for an incoming customer. Furthermore, imagine someone who is trying to upgrade python versions: being able to step through sequential versions is so much easier than making multi-version leaps.
  • If possible, use the latest patch releases. In my experience they do matter at times, and being behind is almost always worse than being up to date. Python 2.7.11 was released in 2015!

Please let me know as things get evolve and I’ll be sure to test. Thanks again. -George

10

@gwk thanks for the input! I’m working on the fixes I outlined as well as updating that blog post you mentioned, and I’ll update this thread when I have something further to share.

1 Like
11