Angular JAMstack - Scully not prerendering API data

Support
, ,
#1

Hi, I am trying to build JAMstack site with Angular and Scully. I followed all the possible @tzmanics tutorials (awesome articles) but I can not manage to make everything work.

I have made simple login view which calls to Netlify Functions to obtain a title - works without issues (HttpClient GET /api/title). Whenever I am running scully it pauses on /login route for about 25 seconds, which I believe is a timeout caused by unavailable API. In result my rendered index.html for login route has no prerendered title. I have no clue how to run it so it fetches all the data. :sob: Same thing happens on my builds with the command:

"jam-prod": "ng build -c production && npm run scully -- --nw --scanRoutes",

My desired goal is to see live prerendered site and use ntl dev locally to preview prerendered JAMstack app.

Thanks for your time!

My netlify.toml configuration:

[build]
    base = "./"
    publish = "./dist/static"
    functions = "./functions"
    command = "npm run jam"

[context.production]
    command = "npm run jam-prod"
    [context.production.environment]
        FAUNADB_SERVER_SECRET = "fn...pM"

[context.branch-deploy]
    command = "npm run jam-dev"
    [context.branch-deploy.environment]
        FAUNADB_SERVER_SECRET = "ef...Md"

[[headers]]
    for = "/*"
    [headers.values]
        Access-Control-Allow-Credentials = "true"
        Access-Control-Allow-Headers = "Access-Control-Allow-Headers, Origin,Accept, X-Requested-With, Content-Type, Access-Control-Request-Method, Access-Control-Request-Headers"
        Access-Control-Allow-Methods = "GET,HEAD,OPTIONS,POST,PUT"
        Access-Control-Allow-Origin = "*"

[[redirects]]
    from = "/api/*"
    to = "http://localhost:34567/:splat"
    status = 200

Tools:

Angular CLI: 10.0.0
Netlify CLI: 2.43.0
Scully: 0.0.98
#3

Almost there! I made ntl dev work with the scully prerendering - everything works like a charm. Still trying to figure out how to make it work with my deployment to Netlify.

I had to make a proxy config for Scully which redirects /api calls to functions endpoint.

{
    "/api": {
        "target": "http://localhost:34567",
        "secure": false,
        "pathRewrite": {
            "^/api": ""
        }
    }
}

Then netlify dev required two things:

  1. Target port for [dev] set to static scully server: targetPort = 1668
  2. Updated run command ng build && scully && scully serve.

Also, the interesting thing was that when ntl dev got simple ng serve on port 4200 it worked without issues, all the redirects did their job. However, making scully to serve static app on port 4200 and plugging it into ntl dev results in a timeout :man_shrugging: Even though I configured target port to be a static scully app it still required proxy config to properly handle http requests.

Now I am wondering how can (or should I) run functions during my builds so scully can prerender all the content. I am thinking about controlling the build chain in which functions deploy first and then during prerendering scully uses those deployed functions as a data source. Trying to come up with a clean solution.

#4

Hiya @AverageNetlifyBoi and congrats on making it that far - you have definitely gone far beyond the knowledge of anyone on our Support team on both setting up netlify dev so carefully and on using Scully. I’m a little low on specific advice other than I can advise that functions should work in a pretty straightforward way: https://docs.netlify.com/cli/get-started/#deploy-directories shows how to configure, so I am guessing you saw that and it didn’t work in some way.

I doubt that you’ll want to increase any timeouts - either they should run in under 10s locally or they definitely won’t run in under 10s (our default function runtime limit per invocation) on AWS. How are you trying to run the functions now and what is happening when you do?

You’ll also have challenges if you want to “use functions” during build on Netlify. We deploy functions after the build completes, so if you access them on the same site during build - you’ll access the deploy that is live while you build the next deploy if you call the production paths. We don’t expose functions that you are building, during build, for you to use - they are a post-build-only artifact, so keep that in mind as you work with Netlify dev - you probably don’t want to have two vastly different configs between the two.

To debug the timeouts, you could try running with a $DEBUG=* environment variable - this leaves a more verbose log that might give you some additional clues. If you end up posting it here for advice, or posting it in an issue you file on the CLI, please make SURE you elide the API token, which will be in the log, before sharing!

1 Like
#5

Hiyyyaaa @fool The timeouts were not on your side, they were caused by my wrong Scully configuration, I misunderstood some parts. Functions are running perfectly.

As you mentioned, functions deploy after the build completes. This is my only concern left before making this whole workflow up and running. I think this is not possible with current Netlify tools :sob:

Since pre-rendering is a hot topic right now I would love to ask for the feature to have control over the deployment chain, so that during my application build I can reference freshly deployed functions.
Here is my desired chain:

  1. Deploy
  2. Functions deploy first
  3. During application build, I pre-render all the content fetching data from above functions
  4. My app goes live and there is no need to fetch data during runtime

I came up with this idea so there is no need to fetch data as new user visits my app. I would love to extended it even further and trigger deploys whenever some collections update in my database and fetch that data during the build, not runtime. Since there is one codebase for application and functions, it should be considered that functions might change during development so we can not reference the ones that are already built.

I think we can close this thread and I’m gonna create a new feature request :smile:

1 Like
#6

thanks for following up!

#7

Upgrade Angular 10.0.3 resolve this issue

#8

@fool Would it be possible within Netlify to trigger a task which prerenders only specific route ( npx scully --baseFilter /someRoute) and replaces ./some-route/index.html in my live dist without triggering a full build?

My dream configuration would be to trigger such task from Netlify functions whenever records change in my specific DB collection. This way I could be loading data to all users with a simple task, fetching data once, instead of each time user visits a page - huge optimization.

#9

If you can render things to static files, or functions, then your plan will work. We don’t “run” anything (except your functions) at browse time - so if you can find a workflow to “do whatever you intend” during build - so the static files and functions accomplish your goals, then you can do it here :slight_smile:

Files from your prior build(s) are NOT AVAILABLE in the build environment - so any files you want to carry forward from an older build and reuse without change need to be either:

  • in git
  • in our build cache (which only caches dependencies by default, but can be used to cache anything you want!)
  • fetched live from your site (you have full outgoing internet access). This seems least efficient, but is within the realm of possibility particularly for specific single files that are harder to generate - e.g. results of a long API call that only change once a week that you can “reuse” for builds during that week. For a site-wide effect like I think you are describing, it would probably not result in much of a speedup to download much of your site like a crawler, during build.

You can manipulate and debug your usage of our build cache using plugins like these:

https://app.netlify.com/teams/YOUR_TEAM_SLUG/plugins?search=cache

(note you need to replace YOUR_TEAM_SLUG with your team’s slug - or start from your site listing page, click plugins, and search for cache)

(or, you can roll your own cache handler based on your examination of the source code for our build environment: https://github.com/netlify/build-image ).

Note that no matter how you approach cache manipulation during build, please make sure your build works WITHOUT cache - as it is usually but not 100% available, and you will need to be able to (re)generate it from scratch on any build request.

Not sure this will help, but like I said, we aren’t scully experts, so if that doesn’t make sense, could you try explaining your goal in a bit different way, and we’ll get our best scully minds to take a look?

1 Like
[Static web generators] - How to handle file-name changes on a new build with atomic deploys
#10

Thanks @fool for the details! Looking at build-image repo I guess this would be possible but would require pretty big effort. I’m thinking about firing functions which execute node scripts, which modify contents of my dist directory, would that be possible? If so, how?

#11

Not possible. Deploys are atomic and read-only after they happen.

You can only change files in your dist directory (if I am reading that correctly as “the files on your website” not “the files in your functions”) by a new deploy.

#12

Alright, thanks a lot! I will find some time to dig into caching thing and hopefully figure this out :smile:

1 Like