Error during download (deployment zip)

Hi I’m trying to use your script, but I keep getting the following error:

const netlifyAuth = new NetlifyAPI(authToken)
                    ^

TypeError: NetlifyAPI is not a constructor

Any ideas why?

Hey there, @almostravioli :wave:

Thanks for chiming in here. Sorry you are encountering difficulties. Which site is this regarding?

Hello everyone,

I am experiencing the same issue. Error during download of recent deploy when I never had this issue downloading this same landing page several times. Zip file is not that large. I was building it on my netlify account and need to now download and upload the files into clients netlify and am at a loss :frowning: how to get the files.

Site ID is b9d0ed13-f793-4506-af09-fc56dfabf864

any help would be lovely. Thank you! - Ryan

Hey @ryan.mccarthy,

Could you try using the tool here:

https://netlify-file-browser.netlify.app/

Thank you so much @hrishikesh This link saved my life.
My cousin smashed my USB on the ground, and I lost the code for my whole Website, as well as its content.
You just saved me.
May god bless you:
https://netlify-file-browser.netlify.app/

Hi @hrishikesh

When files are too big for this tool - is there any other way to get them downloaded?

My website is very huge and this tool stops loading before it can even show the files. Are there any other ways to get these files downloaded?

You can use the Netlify API to create a CLI based tool. I have been working on it, but did not dedicate too much time to it so far. At the moment, I have this code:

import axios, {type AxiosError} from 'axios'
import {createWriteStream, existsSync, mkdirSync} from 'node:fs'
import {cwd} from 'node:process'
import {dirname, join} from 'node:path'
interface NFile {
  deploy_id : string
  id : string
  mime_type : string
  path : string
  sha : string
  site_id : string
  size : string
}
const axiosAbortController = new AbortController()
const axiosNetlify = axios.create({
  baseURL: 'https://api.netlify.com/api/v1',
  headers: {
    authorization: `Bearer token`
  },
  signal: axiosAbortController.signal
})
const deployId = 'id'
const fileList : Array<NFile> = []
function fetchFile(path : NFile['path']) {
  return axiosNetlify<ArrayBuffer>({
    headers: {
      'content-type': 'application/vnd.bitballoon.v1.raw'
    },
    responseType: 'arraybuffer',
    url: `https://api.netlify.com/api/v1/deploys/${deployId}/files/${path.slice(1)}`
  }).then(fileRes => {
    return fileRes.data
  }, (fileErr : AxiosError) => {

  })
}
function fetchFileList(page : number = 1) : Promise<void> {
  return axiosNetlify<Array<NFile>>({
    params: {
      page
    },
    url: `/deploys/${deployId}/files`
  }).then(fileListRes => {
    if (fileListRes.data.length === 100) {
      return fetchFileList(page + 1)
    } else {
      return
    }
  }, (fileListErr : AxiosError) => {

  })
}
fetchFileList().then(() => {
  const pathOnDisk = join(cwd(), './download/', fileList[0]!.path)
  fetchFile(fileList[0]!.path).then(data2 => {
    if (!existsSync(dirname(pathOnDisk))) {
      mkdirSync(dirname(pathOnDisk), {
        recursive: true
      })
    }
    createWriteStream(pathOnDisk).write(Buffer.from(data2))
  })
})

You can extend this code to make a downloader for yourself. If not, depending on how long I get to work on this, this tool would be released accordingly.

I’m assuming, CLI should be able to manage larger deploys as there would be no browser limitations + it would use less resources as there’s no GUI to render. So in theory the CLI should be able to manage deploys of any size, but can’t say until I complete this and try a deploy that’s fairly large.

1 Like

UPDATE: Here’s a tweaked version of that, which manages to download everything successfully:

import axios from 'axios'
import {createWriteStream, existsSync, mkdirSync} from 'node:fs'
import {cwd} from 'node:process'
import {dirname, join} from 'node:path'
interface NFile {
  deploy_id : string
  id : string
  mime_type : string
  path : string
  sha : string
  site_id : string
  size : number
}
const axiosAbortController = new AbortController()
const axiosNetlify = axios.create({
  baseURL: 'https://api.netlify.com/api/v1',
  headers: {
    authorization: `Bearer token`
  },
  signal: axiosAbortController.signal
})
const deployId = 'id'
let fileList : Array<NFile> = []
async function fetchFile(path : NFile['path']) : Promise<ArrayBuffer> {
  try {
    const fileRes = await axiosNetlify<ArrayBuffer>({
      headers: {
        'content-type': 'application/vnd.bitballoon.v1.raw'
      },
      responseType: 'arraybuffer',
      url: `https://api.netlify.com/api/v1/deploys/${deployId}/files/${path.slice(1)}`
    })
    return fileRes.data
  } catch {
    throw new Error(`failed to fetch ${path}`)
  }
}
async function fetchFileList(page : number = 1) {
  try {
    const fileListRes = await axiosNetlify<Array<NFile>>({
      params: {
        page
      },
      url: `/deploys/${deployId}/files`
    })
    fileList = fileList.concat(fileListRes.data)
    if (fileListRes.data.length === 100) {
      await fetchFileList(page + 1)
    }
  } catch {
    throw new Error('failed to fetch file list')
  }
}
await fetchFileList()
for (const file of fileList) {
  const fileBuffer = await fetchFile(file.path)
  const pathOnDisk = join(cwd(), './download/', file.path)
  const pathOnDiskDir = dirname(pathOnDisk)
  if (!existsSync(pathOnDiskDir)) {
    mkdirSync(pathOnDiskDir, {
      recursive: true
    })
  }
  createWriteStream(pathOnDisk).write(Buffer.from(fileBuffer))
}

It’s far from complete as it doesn’t show any useful info on if it’s working or not, but it will at least (probably) get you unblocked right now.

Thanks for that quick reply, do you only have this script in typescript? If so, how should the config file look like? at the moment I am facing this challenge:

You can remove the TS parts:

import axios from 'axios'
import {createWriteStream, existsSync, mkdirSync} from 'node:fs'
import {cwd} from 'node:process'
import {dirname, join} from 'node:path'
const axiosAbortController = new AbortController()
const axiosNetlify = axios.create({
  baseURL: 'https://api.netlify.com/api/v1',
  headers: {
    authorization: `Bearer token`
  },
  signal: axiosAbortController.signal
})
const deployId = 'id'
let fileList = []
async function fetchFile(path) {
  try {
    const fileRes = await axiosNetlify({
      headers: {
        'content-type': 'application/vnd.bitballoon.v1.raw'
      },
      responseType: 'arraybuffer',
      url: `https://api.netlify.com/api/v1/deploys/${deployId}/files/${path.slice(1)}`
    })
    return fileRes.data
  } catch {
    throw new Error(`failed to fetch ${path}`)
  }
}
async function fetchFileList(page = 1) {
  try {
    const fileListRes = await axiosNetlify({
      params: {
        page
      },
      url: `/deploys/${deployId}/files`
    })
    fileList = fileList.concat(fileListRes.data)
    if (fileListRes.data.length === 100) {
      await fetchFileList(page + 1)
    }
  } catch {
    throw new Error('failed to fetch file list')
  }
}
await fetchFileList()
for (const file of fileList) {
  const fileBuffer = await fetchFile(file.path)
  const pathOnDisk = join(cwd(), './download/', file.path)
  const pathOnDiskDir = dirname(pathOnDisk)
  if (!existsSync(pathOnDiskDir)) {
    mkdirSync(pathOnDiskDir, {
      recursive: true
    })
  }
  createWriteStream(pathOnDisk).write(Buffer.from(fileBuffer))
}

Note that, since it downloads one file at a time, it’s going to take really long to download the entire deploy. My deploy of about 1200 files took about or more than an hour. You can customise the download logic in the last lines of the script, but then you might need to be cautious about rate limits on the API.

Ok, Thanks. without TS it seems to work except that now my only blocker is the deployID. Initially I thought it’s the site ID but that throws the (‘failed to fetch file list’) error. My site was renamed, so I do not have that default url to see the deplyID. It was a manual upload and at this point I am not sure where else can I get the deployID?

The deploy ID is shown in the URL: https://app.netlify.com/sites/<site-name>/deploys/<deploy-id> whenever you go to a deploy page.

Thanks, I got the ID but I am still not getting the files. That (‘failed to fetch file list’) error still shows up and I am sure that I am using the right deploy ID and authorization value. Access token is the authorization value, right? I wonder what could I be doing wrong… how did you see the number of files in your deploy? my deploy logs have no messages

I have provided the base-point code above. You can always tailor it to suit your needs. For example, try logging the errors in your console to get a better idea on what the error is instead of the generic message that I’m currently showing.

I simply logged console.log(fileList.length) after await fetchFileList(). But you can also check that in the Netlify API documentation response (file_count) property.

The access token generated from here: Get started with the Netlify API | Netlify Docs should be used as:

authorization: `Bearer <access_token>`

Replace the value after Bearer

1 Like

Thanks, it was how I replaced the token. It’s now working

It’s downloading but these files are not mine. How should the package.json look for javascript? Mine is shown below:
{
“name”: “cesium-recover”,
“version”: “1.0.0”,
“description”: “”,
“main”: “index.js”,
“scripts”: {
“test”: “echo "Error: no test specified" && exit 1”
},
“type”: “module”,
“author”: “”,
“license”: “ISC”,
“dependencies”: {
“axios”: “^1.5.0”,
“netlify”: “^13.1.10”,
“nodejs-file-downloader”: “^4.12.1”
}
}

If you’re using my script, you just need axios as a dependency. So:

{
  "dependencies": {
    "axios": "1.5.0"
  },
  "type": "module"
}

should do it. If you’re writing a script yourself, feel free to do it your way.

1 Like

Thank you, I got all the files.

Will this download the pages generated during build using ssg?

This will only download all the files that exist in your publish folder. The deploy logs indicate which folder is being published (like publishing files from dist). If your file exists in that folder, it will be available for download.