Vercel is serving stale versions of static files, after rebuild

[magnusriga]

Summary

I have come across a seemingly significant issue with the Vercel cache.

Background

• The app is built with Next.js 13 and is hosted on Vercel
• The app pulls data from Contentful, via GraphQL fetches
• The app's pages are statically generated (screenshot of the build output can be seen below)

Problem

• When building / redeploying the application on Vercel, the old cache should be deleted
• When visiting the website for the first time after redeploying, it shows the right (not stale) pages
• When re-visiting the website, it suddenly starts showing the old (stale) pages

---

SCREEN SHOTS

Build output:

HTTP Request and Response, on the first page visit after redeploying:

HTTP Request and Response on subsequent visits to the site:

---

Observations and Question

As seen from the x-vercel-cache header, Vercel does indeed grab the new (non-stale) files from the static build when a client first visits the page after redeploying (definition of PRERENDER here).

On subsequent visits, however, it shows stale / old data. That data was served from the Vercel cache (HIT in the x-vercel-cache header). For some bizarre reason, it does not look like the Vercel cache was updated with the latest files from the static build, during the first visit.

Any idea what is going on here, and how to fix it?

Additional information

No response

Example

No response

[amyegan]

Thanks for bringing this to our attention, Magnus. Can you share an example deployment link that we can use to get a better look at what is going wrong?

[magnusriga]

@amyegan , thank you for helping me look into this.

I created a minimal reproducible example (MRE) with the below details. The problem is still present in the MRE.

• Live URL: https://my-app-neon-ten.vercel.app/
• Source: https://github.com/magnusriga/my-app
• Framework: Next.js
• Host: Vercel
• Contentful as data CDN, which deploys to Vercel via a web hook
• I created a new Contentful space, with a single entry, for the MRE
• Data is fetched from Contentful via GraphQL, with fetch()

On the very first build, the data on the live website is correct. I.e. it represents the latest published data on Contentful.

The Error and Steps to Reproduce It

1. Make a small change to the code locally, with a subsequent git commit + git push, just to make Vercel rebuild the site (not sure if this step is needed)
2. Do one change on Contentful and hit publish, so the site rebuilds
3. Head over to the live site and move around between the front page and the test-entry page, both by reloading the browser on the front page and the test-entry page, and by moving from the front page to test-entry page with the link (it is a Next.js Link)
4. Repeat step 4
5. Now things should start behaving oddly
6. First, when reloading the browser on the test-entry page, I get data from two Contentful publishes ago (i.e. not even the latest)
7. Second, when moving back to the front page with the browser back button, reloading the browser there, then going to the test-entry page with the page link, I get data from the previous Contnetful publish
8. In summary, two ways of navigating to a page shows the page with different data, and none of these two are the latest data from Contentful

---

SCREENSHOTS

Latest Contentful data. Notice that the Description field is "foo bar", and that the entry is published. I verified that it did redeploy upon publishing, so the webhook works.

When navigating to the test-entry page with the page link (Next.js link), the html element holding the Description from Contentful equals "some dataafoobar". That was the Description value two publishes ago.

When reloading the browser on the test-entry page, the html element holding the Description from Contentful equals "some daa". That was the previous Description value, before "foo bar".

As seen, none of these two show the most recently published Description value, which is foo bar.

---

What I Tried

• Wrapping the fetch() in React's cache(), as specified here
• Adding a revalidate: 10 to the fetch().
• Adding npx rimraf .next as part of npm build, so we are 100% certain the old next folder is deleted
• Double-checked that the correct data is pulled from Contentful build-time, when re-deploying the site after publishing a change in Contentful. A console.log of the pulled data in Vercel's build log verifies that the updated Contentful data was indeed pulled upon build. In other words, the issue is not on Contentful's side.

None of the above fixed the problem. No matter what I tried, Vercel serves the correct version on the first visit after a redeploy, when the x-vercel-cache header is PRERENDER. After that, it serves an old version from its cache (x-vercel-cache header is HIT).

As a side note, one or more of the attempts led to x-vercel-cache header being set to STALE and then in the subsequent visit being HIT. Unfortunately, the data the Vercel cache shows is still old.

---

Edit:

• I added the following API route that fetches the data from Contentful on-demand: https://my-app-neon-ten.vercel.app/api/fetch
• If we do not delete the .next/cache folder on re-build (which Next does not do automatically), or include a timeout like revalidate: 10 in the fetch(), then the new Build cache in Vercel will contain stale fetch data.

---

Further tests:
I have made a few findings, these are as follows:

• By deleting the .next/cache on build, we are certain that the fetches running build-time do not get old data from .next/cache
• Deleting that cache folder build-time is a better approach than adding revalidate: X to the fetch, because in the latter alternative the first fetch in the app will always get stale data from the cache
• By deleting that folder on build, everything works as expected with next build + next start locally
• The issue still persists on the live Vercel deployment, however
• Therefore, it seems the problem is with the Vercel Edge cache
• On first hit by the browser, whether via pre-loading the test-entry page from the front page (as a link to it is visible in the viewport), or by re-loading the browser directly on the test-entry page, the correct data is retrieved from the Build cache
• Now, when we visit the test-entry page again, we see a HIT in the x-vercel-cache, but old stale data is served
• As such, it seems the Vercel Edge cache is not actually updated after the first visit (when we got PRERENDER in the x-vercel-cache header)

[amyegan]

Thanks for sending all of this info. I see you're using the App Router, which is currently still in beta, so the problem might be related to that. 🤔 I'm sharing this with the engineering team and I'll update here when I know more 🙇

[magnusriga]

Any input here @amyegan? I am in a bit of a time squeeze and have to move over to Netlify if I cannot get this to work on Vercel. I am pretty sure it is a bug, as the same test on Netlify seems to invalidate and update the cache after redeployment. I would deeply appreciate your input at your earliest convenience. Thanks a million!

[amyegan]

This seems to be related to the App Router and Next.js Cache, which are still in beta at the moment. The engineering team is working on a solution, but they shared a workaround that should get you unstuck for the moment. You can set up a custom incremental cache handler to skip the fetch cache by adding a line to your next.config.js and including a cache handler: https://github.com/vercel/next.js/blob/afd7a50a7780a56c2ebe1e3dea77d79054bf72f1/test/e2e/app-dir/app-static/next.config.js#L4

[vxna]

@magnusriga Unfortunately I doubt that my case is a Next.js issue, just tested it again with the 13.3.0 release and it's all the same 😢

We are not alone though, I am almost sure this issue is pretty much the same bug: https://github.com/orgs/vercel/discussions/2037

[amyegan]

@vxna I saw it and shared with the engineering team, and then completely missed letting you know that I did. Sorry about that!

There were some cache-related changes that went out in the most recent release that should fix some, but probably not quite all, of the problems we've been discussing here.

Everyone, please keep letting me know what you're seeing, both good and bad. And I'll keep putting new info in front of the engineers to help solve any lingering cache problems 🙇

[danielrhodes]

@vxna I saw it and shared with the engineering team, and then completely missed letting you know that I did. Sorry about that!

There were some cache-related changes that went out in the most recent release that should fix some, but probably not quite all, of the problems we've been discussing here.

Everyone, please keep letting me know what you're seeing, both good and bad. And I'll keep putting new info in front of the engineers to help solve any lingering cache problems 🙇

I tried upgrading and got 500 errors, which sound like they are related. If there's a way to get rid of this cache that would probably solve the issue.

[lindesvard]

Stumbled upon this issue as well.

First visit has x-vercel-cache: PRERENDER with up-to-date info from our headless CMS. Second time we get x-vercel-cache: HIT but stale info.

@amyegan "hack" with custom incremental cache handler solved it.

Using next@13.3.1 and appDir

[zartinn]

Same issue here: vercel/next.js#48985
I can also confirm that in another (big) project that uses 13.4.0 the issue is still there.

Happens whenever you are using a deploy hook without buildCache=false or you make a new commit.

My workaround for now is to delete the cache before merging something to an important branch:
https://vercel.com/YOUR_ACCOUNT/YOUR_PROJECT/settings/data-cache

[nguyenphu123]

I am having this issue on 13.4.12.
My workaround is putting a request param at endpoint api like this
export async function GET(request) { console.log(request.url) try { await connectDB(); const banner = await Banner.find({}); return NextResponse.json({ banner }); } catch (error) { if (error instanceof mongoose.Error.ValidationError) { let errorList = []; for (let e in error.errors) { errorList.push(error.errors[e].message); } console.log(errorList); return NextResponse.json({ msg: errorList }); } else { return NextResponse.json({ msg: ["Unable to send message."] }); } } }
and its work quite good but I don't understand why.

[jastaffen]

This is also happening in Nuxtjs, so I'm assuming this is a vercel issue and not framework specific.

[kang-theo]

This is still a problem. I am using next 14.1.3. I agree with jastaffen that this should be a vercel issue.
The build cache should have been updated according to the building log:

Deployment completed
Uploading build cache [120.06 MB]...
Build cache uploaded: 3.409s

But it still servers the stale pages.

[amyegan]

Thanks for reporting this @kang-theo. The latest version of Next.js includes a fix for a fetch data cache issue. If you upgrade to Next v14.2.3 do you still see stale pages?

[HongDucAnhNguyen]

Hi there, currently using v14.2.3 and I seem to be encountering this problem as well

[amyegan]

I haven't been able to replicate the issue myself. Do you have a minimal reproducible example that I can use to investigate?

[columk1]

This is also happening with favicons in 14.1.4. If I delete my icon file, the favicon is gone after deployment. But, if I create a new file with the same name and deploy again, the old file that was deleted will re-appear. There doesn't seem to be a way to remove these old static files from the build.

[columk1]

I used multiple devices and different browsers to test and still saw the deleted favicon files after each deployment.

There are a lot of posts about this favicon issue on stack overflow and Github and most of them have the advice to change the name of favicon.ico to icon.ico to replace the Vercel favicon. This worked for me, but it created another un-replaceable favicon. Just like favicon.ico, if I deleted or replaced icon.ico and then redeployed, the old icon was still there. Even on a new device that hadn't visited the site before.

I figured it might be related to this issue, since deleted files are being included in the build. It would be nice to be able to remove these files manually in Vercel.

[amyegan]

I've not quite been able to replicate this with my test app. I tried with a fresh create-next-app example, and new icon typically shows up for me immediately after a new deployment. But I might use different deployment steps or have something different in my project settings from what you're using.

Can you share what steps you follow to deploy? It would also be good to know the Node.js version and build settings you use for the project. That will help me investigate further.

[columk1]

Sure, I replicated it here:
https://github.com/columk1/test-next-favicon

I used create-next-app but changed the next version to 14.1.4 which is the version I was using when I had the issue. I'm deploying with default settings on Vercel. Node Version 20.x

When favicon.ico is deleted the next deployment correctly has no favicon, but when a new favicon.ico is added, the Vercel favicon re-appears. I have seen this documented in many places on Stack Overflow, Reddit, Github etc. The recommended solution is usually to replace favicon.ico with icon.ico, or to use metadata to point to a different file.

Originally, I found changing the name of the new favicon.ico to icon.ico did replace the Vercel favicon, but it suffered from the same issue when replaced itself. I even tried replacing my own working icon.ico with a new favicon.ico, which caused the Vercel logo to re-appear, despite having been deleted many deployments earlier.

[amyegan]

Thanks for sharing that example. I was able to get the favicon to reliably update by deploying without the build cache. You can can bypass the build cache by redeploying or by using vercel --force --prod from your local repo. Please give that a try and let me know whether it works for you.

vercel.com/docs/deployments/troubleshoot-a-build#managing-build-cache

[columk1]

Thank you, that's the solution.

[Tiarhai]

Hope this helps some people:

I am using Nuxt.js on Vercel (not important to the final cause), and I was having this exact same issue:

• Site was loading correctly on new window, refresh, and I would see stale content.
• Site would load correctly on a different page, I could click a client-side link to load the usually broken page and it would load fine. Once on that page, refresh, and I would see stale content.

The issue for me was that at some point in the previous week, I had created a Local Override in Chrome DevTools. I finally noticed this was distinguishable by the small purple circle on the bottom right of the document icon in the Network tab of Chrome DevTools:

Another way to spot this behaviour via the Chrome DevTools Network log, on initial page load, my document was correctly compressed to 90kB using brotli (a default, via Vercel). The problematic document on refresh was showing as 680kB - aka not being compressed, and not being served via Vercel.

This Local Override had stored the old HTML file and content as a file on my computer and was serving it to my browser under the circumstances mentioned above.

Finding and deleting this Local Override solved my issue.

FYI - Local Overrides can be created for everything in the Network log: HTML Documents, fetch/API responses, CSS files, etc. so worth checking for that purple circle!

[jastaffen]

Hi @Tiarhai, this is good to know, so thanks for sharing, but my concern is more with clients and users not seeing relevant fresh data and instead seeing stale data after a site rebuilds triggered via vercel deploy hooks.

[lyfmantle]

exactly the same issue, I can not update the favicon.ico

[shnartho]

same issue here, I removed build then add but seems like vercel keep cache in a way that one image can be uploaded only 1 time, if you remove or update nothing will work.

[shnartho]

Well, you can go all deployments -> click on 3 dots of current deployment -> Redeploy -> uncheck build cache -> build. It solved the issue for me.

[idiglove]

It's happening to me now. I've rebuilt without build cache

[shnartho]

I recommend try to check if github is keeping the cache, remove the image from github, remove from local, push and build then add the image in local and github then build. Alternatively, you can rename the image then push in github.

[vavilondev]

I also have the same issue
vercel/next.js#65797