Under Site settings > Build & deploy > Continuous Deployment > Build image selection you can choose between the Xenial and Focal build images. You can also change the node version used in a build image.
However, this is possible more to do with the node version used to build functions which is (potentially) different to that used used in the build image. This node version is controlled via the AWS_LAMBDA_JS_RUNTIME environment variable and accepts nodejs10.x, nodejs12.x, and nodejs14.x as per this documentation.
Edit:
On 13 December, node 14 became the default for functions, which may explain the issues you are seeing