Mechanism to skip edge data fetching on client navigation

[davidbarratt]

Clone of vercel/next.js#11897

Effectively there is no way to do both getEdgeProps() for data fetching when the page is rendered, but not do this step from the client if the user is changing routes. Currently what happens is the user makes an XHR request for the props (which are returned in JSON)

This is a blocking issue for @chickaree to adopt @flareact because it's actually takes longer to render the props on the edge than on the client (Cloudflare Workers are missing DOMParser, so the parsing has to be forwarded to a Node.js server).

As I mentioned in vercel/next.js#11897 (comment), I think this could be resolved by #92.

[traffisco]

Why use getEdgeProps if you want to render on the client?

[davidbarratt]

Why use getEdgeProps if you want to render on the client?

Because I want to have SSR when the URL is directly accessed.

[traffisco]

Why use getEdgeProps if you want to render on the client?

Because I want to have SSR when the URL is directly accessed.

So basically, you want the first load to be served from cache then, skip getEdgeProps on navigation and run it on the client?
I'm also not a fan of how the current navigation system works. I prefer pages hard cached so I'm performing hard navigation to cache each page individually. Then pages are served from cf cache and render time is not an issue.

[davidbarratt]

So basically, you want the first load to be served from cache then, skip getEdgeProps on navigation and run it on the client?
I'm also not a fan of how the current navigation system works. I prefer pages hard cached so I'm performing hard navigation to cache each page individually. Then pages are served from cf cache and render time is not an issue.

Right. so basically going directly here:
https://chickar.ee/davidwbarratt.com
would run getEdgeProps()
but if a user started at:
https://chickar.ee/ and then went to https://chickar.ee/www.nytimes.com
it would not (it would just do the data fetching on the client).

The work-around I have right now is to take a static HTML page and use HTMLRewritter() to rewrite the static page on the worker. :/

[traffisco]

So basically, you want the first load to be served from cache then, skip getEdgeProps on navigation and run it on the client?
I'm also not a fan of how the current navigation system works. I prefer pages hard cached so I'm performing hard navigation to cache each page individually. Then pages are served from cf cache and render time is not an issue.

Right. so basically going directly here:
https://chickar.ee/davidwbarratt.com
would run getEdgeProps()
but if a user started at:
https://chickar.ee/ and then went to https://chickar.ee/www.nytimes.com
it would not (it would just do the data fetching on the client).

The work-around I have right now is to take a static HTML page and use HTMLRewritter() to rewrite the static page on the worker. :/

Wouldn't it be easier then to do a hard navigation then?

[davidbarratt]

Wouldn't it be easier then to do a hard navigation then?

I'm sorry, I don't understand your question.

[traffisco]

Wouldn't it be easier then to do a hard navigation then?

I'm sorry, I don't understand your question.

I propose to perform a hard navigation when user clicks the link. On first load, page will be rendered with getEdgeProps, upon additional navigations to that route, pages will be served directly from cf cache.

[davidbarratt]

I propose to perform a hard navigation when user clicks the link. On first load, page will be rendered with getEdgeProps, upon additional navigations to that route, pages will be served directly from cf cache.

That's what already happens, it just does it in the background with XHR.

I'm saying I don't want it to hit CF at all. :)

[davidbarratt]

errr... let me be more clear.

Right now, the behavoir is that if you have getEdgeProps() on a route and you navigate to it from a different route, @flareact will make an XHR request to Cloudflare and execute getEdgeProps() (unless it is already cached) and return the result in JSON format.

Since I'm hitting a third party (and already have their URL). I need to be able to skip Cloudflare completely and hit the 3rd party directly from the user's browser.

[davidbarratt]

Some other users cases in vercel/next.js#11897 are when you already have the props locally for the page transition and there is no need to make an XHR request to get them. There is no way to stop that request from happening in @flareact or Next.js currently. In other words, you could be making a completely useless HTTP request and blocking the navigation on that request. :(

[traffisco]

errr... let me be more clear.

Right now, the behavoir is that if you have getEdgeProps() on a route and you navigate to it from a different route, @flareact will make an XHR request to Cloudflare and execute getEdgeProps() (unless it is already cached) and return the result in JSON format.

Since I'm hitting a third party (and already have their URL). I need to be able to skip Cloudflare completely and hit the 3rd party directly from the user's browser.

I think that this only happens if you use flareact Link with prefetch on. When landing on a cached route(directly or with hard navigation) you just get rendered html from cache(with props in the the flareact object, don't know why, there's a discussion about that).
I do understand why you may want more granular control but as a workaround, you can hard navigate the user. If the route is already cached, you'll hit your 3rd party api once from getEdgeProps, if the page is cached, you just get html.

[davidbarratt]

I think that this only happens if you use flareact Link with prefetch on. When landing on a cached route(directly or with hard navigation) you just get rendered html from cache(with props in the the flareact object, don't know why, there's a discussion about that).

No.
Direct Navigation calls getEdgeProps() (or pulls old result from cache), then passes it to React to render into HTML.
Client Navigation makes an XHR request which calls getEdgeProps() (or pulls old result from cache).
If your route does not have a getEdgeProps() it can skip the XHR on client navigation.

I do understand why you may want more granular control but as a workaround, you can hard navigate the user. If the route is already cached, you'll hit your 3rd party api once from getEdgeProps, if the page is cached, you just get html.

Again, that is not a work around. If I hard navigate the user, Flareact will call getEdgeProps() (or pull old result from cache). I want the client navigation to not make a request to Cloudflare at all. Again, since hitting Cloudflare would always be a Cache MISS (I can't cache the response from the 3rd party), it is faster to make a request to the 3rd party directly from the browser.

[traffisco]

I think that this only happens if you use flareact Link with prefetch on. When landing on a cached route(directly or with hard navigation) you just get rendered html from cache(with props in the the flareact object, don't know why, there's a discussion about that).

No.
Direct Navigation calls getEdgeProps() (or pulls old result from cache), then passes it to React to render into HTML.
Client Navigation makes an XHR request which calls getEdgeProps() (or pulls old result from cache).
If your route does not have a getEdgeProps() it can skip the XHR on client navigation.

I do understand why you may want more granular control but as a workaround, you can hard navigate the user. If the route is already cached, you'll hit your 3rd party api once from getEdgeProps, if the page is cached, you just get html.

Again, that is not a work around. If I hard navigate the user, Flareact will call getEdgeProps() (or pull old result from cache). I want the client navigation to not make a request to Cloudflare at all. Again, since hitting Cloudflare would always be a Cache MISS (I can't cache the response from the 3rd party), it is faster to make a request to the 3rd party directly from the browser.

You looking network in dev? I'm looking at my deployed project, no props fetched by xhr, only html response.

I don't understand what you mean by "not make a request to cf at all". The response coming from a cf worker, you always hit cf.

If you can't cache 3rd party api, how you want to use getEdgeProps? Without caching? Why not useEffect then? Or you want that only the first response be cached?

[davidbarratt]

Take a look at this function:

flareact/src/client/page-loader.js

Lines 79 to 83 in 02cd98a

	async loadPageProps(pagePath) {
	const url = getPagePropsUrl(pagePath);
	const res = await fetch(url);
	return await res.json();
	}

which is called by:

flareact/src/router.js

Line 55 in 02cd98a

const { pageProps } = await pageLoader.loadPageProps(normalizedAsPath);

I want to prevent that from happening. The only way (looking at the code) is by not having getEdgeProps() on the component. That is what I have done. But, that prevents having a dynamic SSR, you can only have a static SSR (if someone goes directly to the URL).

[traffisco]

Another option is to hard navigate to a dynamic route url, SSR will be dynamic based on that url

[davidbarratt]

Another option is to hard navigate to a dynamic route url, SSR will be dynamic based on that url

I've said this three times, but it's really not, it does the same exact thing here:

flareact/src/worker/index.js

Line 109 in 02cd98a

const props = await getPageProps(page, query, event);

If you go to https://flareact.com/ and open up the "Network" in dev tools you can see the XHR requests to get the props (in JSON) as you navigate the documentation.

[traffisco]

Another option is to hard navigate to a dynamic route url, SSR will be dynamic based on that url

I've said this three times, but it's really not, it does the same exact thing here:

flareact/src/worker/index.js

Line 109 in 02cd98a

const props = await getPageProps(page, query, event);

If you go to https://flareact.com/ and open up the "Network" in dev tools you can see the XHR requests to get the props (in JSON) as you navigate the documentation.

You mean this?
#80 (comment)

[davidbarratt]

You mean this?
#80 (comment)

The XHR is going to happen whether it is prefetched or not. Prefetching just makes it happen before you click the Link. At some point, the props must be fetched. This task is to skip fetching them completely (optional, of course).

As I mentioned before, #92 would probably fix this anyways.

[traffisco]

You mean this?
#80 (comment)

The XHR is going to happen whether it is prefetched or not. Prefetching just makes it happen before you click the Link. At some point, the props must be fetched. This task is to skip fetching them completely (optional, of course).

As I mentioned before, #92 would probably fix this anyways.

Unless you DON'T use Link. Assuming you set the cache parameter, GetEdgeProps will run only once from the worker(you referenced the code yourself) and return cached page version on following requests.

[davidbarratt]

Unless you DON'T use Link. Assuming you set the cache parameter, GetEdgeProps will run only once from the worker(you referenced the code yourself) and return cached page version on following requests.

🤦 If you don't use Link, you'll do a hard page transition (which will reload all the JS, but that is a sidepoint), which will hit the worker, and call getEdgeProps() before passing it React to be SSR'd.

[davidbarratt]

In other words, the props are cached either way, if it's returned in JSON or in HTML, that is irrelevant. What I'm trying to do is not make a request to Cloudflare at all. I'm not sure why this is so hard to understand.

[traffisco]

Unless you DON'T use Link. Assuming you set the cache parameter, GetEdgeProps will run only once from the worker(you referenced the code yourself) and return cached page version on following requests.

🤦 If you don't use Link, you'll do a hard page transition (which will reload all the JS, but that is a sidepoint), which will hit the worker, and call getEdgeProps() before passing it React to be SSR'd.

Correct. But all the heavy lifting happens only once. Upon following hits to that route, your client served with rendered html and the usual js bundle

[traffisco]

In other words, the props are cached either way, if it's returned in JSON or in HTML, that is irrelevant. What I'm trying to do is not make a request to Cloudflare at all. I'm not sure why this is so hard to understand.

What's hard to understand(for me) is why using getEdgeProps in the first place if you don't want to make requests to cf.
There's a PR or an issue about adding a parameter to getEdgeProps if it's run from the worker or the client. Maybe you can use that, look it up.

[davidbarratt]

Correct. But all the heavy lifting happens only once. Upon following hits to that route, your client served with rendered html and the usual js bundle

To put this another way, I need to say revalidate: 0
https://flareact.com/docs/data-fetching#caching-and-revalidation

Meaning it will happen every time (which is exactly what happens with JSON format too)

What's hard to understand(for me) is why using getEdgeProps in the first place if you don't want to make requests to cf.

I want to use it when a person goes to the page directly (return HTML & SSR) but not use it when someone navigates on the client (see the updated title of this issue)

[traffisco]

I saw something about passing isWorker parameter to getEdgeProps(cant find it, sorry).
If client lands on a route, you render getEdgeProps from worker, if from client(navigation), you return from getEdgeProps and use useEffect to fetch the data.
Assuming frefetch is off.
https://flareact.com/docs/flareact-link#prefetch