Reducing API Calls to Boost Performance and Stay Within Rate Limits
In this post, we discussed a website that was making an excessive millions of API calls per day, completely negating all the benefits of having a cutting-edge Next.js website powered by Contentstack as the headless CMS. Our team at XCentium was able to demonstrate our knowledge and expertise around the issue and implement a new strategy centered around CDN caching and automation to bring them down to thousands of API calls per day. Showing how working with an expert can prevent costly pitfalls and unlock the full value of a headless CMS architecture.
Key Takeaways
Identifying Insufficiencies is Crucial: A lack of understanding and unchecked features like Link prefetching and SSR configurations can inadvertently drive up API requests and costs.
Caching Leveraging CDN Cache and Automation: Serving pages from the CDN cache and revalidating through Contentstack’s automations sharply reduced the API usage and increased performance.
Expert Implementation Pays Off: Implementing the right technical solutions and best practices is essential to success with a headless CMS and it is not all about technology choice.
The Motivation
In today’s digital landscape, transitioning from a traditional CMS to a headless CMS promises big gains in efficiency, scalability, and flexibility. This move enables organizations to deliver content seamlessly across multiple platforms, boost site performance, and future-proof their digital strategy. However, it’s essential to approach implementation with the right expertise and planning to avoid common pitfalls that can negate these benefits. I want to discuss a situation where a site with Next.js was making roughly millions of API calls per day, and how we were able to get that down to just thousands of calls per day!
You can easily see when our optimizations took effect.
This was a critical issue for the website because they were making way too many redundant and unnecessary API calls, which were compromising the performance and scalability of the site as driving up the costs.
Understanding the problem
Next.js Link Pre-Fetching: Normally, Next.js Links optimize the user experience, making navigation near instantaneous and smooth. It does this by prefetching the pages for each link in the background, which in this case, was leading to all those excessive and redundant API calls.
SSR and SSG Confusion: Prefetching pages really only optimizes Static Site Generation (SSG) pages. Because when using SSR, the API calls must be made every time a page is fetched. With SSG, the API only needs to be called once per page on build.
ISR Incompatibility: Incremental Static Generation sounds like it would be the perfect solution here, but there are some issues with on-demand revalidation (topic for another day)
What the problem looked like in practice is, anywhere on the page, there was a link that was visible. Next.js would go and fetch that page in the background, and in doing so, it would also make any API calls found on that page as well. Leading to not just the api calls needed for the page you are on being made, but any page linked on that page as well. So, in our case, a homepage that should only be making 6 API calls is making more than 150 API calls, which was a major concern.
Our solution was to refine our SSR approach to leverage our CDN and deliver pages through the CDN cache. This means now every time a page is called, instead of hitting the API directly, if it's already cached then we will just serve that page directly from the CDN. Now this solves the redundant API calls we were making but also introduces a new issue. If new content is published in Contentstack we need to know, to invalidate the cache for those specific pages and serve the new content and not old content.
For this issue, we created an automation recipe in Contentstack that triggered on publish of any content entry and will revalidate the cache in the CDN. You can think of it as similar to using ISR and having a webhook revalidate the cache on publish.
Conclusion
The transition to a headless architecture with Next.js and Contentstack offers immense potential, but as this case demonstrates, the "how" is just as important as the "what". By identifying the specific conflict between Next.js prefetching and SSR, and implementing a robust strategy centered on CDN caching and automated revalidation, we turned a critical performance bottleneck into a scalable success. Reducing API calls from millions to thousands isn't just a technical victory; it proves that expert implementation is key to unlocking the true efficiency and ROI of your digital strategy.
Frequently asked questions
How did optimizing CDN caching reduce API calls on the website?
By leveraging CDN caching and only revalidating caches when new content was published, the site ensured pages were served from cache, drastically reducing the number of direct API calls made for each page load.
What caused the excessive API calls in the original Next.js setup?
Unchecked features like Next.js Link prefetching and confusion between SSR and SSG resulted in many redundant API requests, as every visible link triggered background fetches causing unnecessary API usage.
Why is expert implementation important for headless CMS architectures?
Expert implementation helps avoid common technical pitfalls, ensures optimal performance, and allows organizations to fully realize the efficiency, flexibility, and cost benefits of a headless CMS.