Extended content sync Contentful to Algolia with Next.js 14

Table of Contents

Algolia — Build Plan (Free)
Contentful — Free Plan
Next.js 14 — Route
Contentful account registration
Additional information regarding GraphQL and Content Types
Endpoint creation in Next.js
Webhook creation in Contentful
Cloudapp-dev and before you leave

Let’s assume you plan to integrate an On-Site-Search on your website/blog and you don’t know what could be a viable solution for you in terms of easy implementation and at no cost.

Have you already heard about the headless CMS Contentful and the AI-Serach Algolia?

As already explained in my previous posts (Step-by-Step Guide for the whole project -> Next js Content Management, Quo Vadis ? ) there is a very generous free plan for Contentful and luckily for Algolia as well.

Algolia — Build Plan (Free)

Up to 10k search requests/month
1m records included
AI features to test

Contentful — Free Plan

5 users
2 locales
1M API calls/month
Community-based support
Structured content
Developer tools
1 Intro Space

Next.js 14 — Route

Built-in API feature from Next.js

Let’s start with registering a new Contentful account (if not already done). Here are two posts with a step-by-step guide:

Contentful account registration

Nextjs 13.5.6 — Complete Example — Typescript / App Router / Contentful and a lot more

Additional information regarding GraphQL and Content Types

Next.js 14 — Complete Example — Typescript / Tailwindcss / Contentful — Part 2

Now we have to complete the registration of an Algolia account

Where you can do a social login or via your work email. After you are logged in you have to create an application (Algolia creates a default application after the first login). First click on (1), so that you open the settings page and then on (2) to open the application overview page.

Now you can see the application overview page and there is one application without any index (1).

Let’s click on (2) to open the search overview page and then we click on the CTA “Create Index”. We will call it “example_dev”

Now let’s go back to the settings overview page (left bottom symbol -> as shown in the first Algolia screenshot), but now we click on “API Keys” in the left section “Team and Access”. We land on the API Keys overview page, where we need three values (“Application ID”, “Search-Only API Key” and ”Admin API Key”). The Search-Only API Key will be used in the next post, during the frontend implementation of Algolia. Now we need the Admin API Key because we have to write/push data to the previously created index “example_dev”.

We have both accounts (Contentful and Algolia) ready and have started creating a new Next.js API route for the sync process. If you want you can directly clone this repository

Here is the GitHub link -> GitHub - cloudapp-dev/nextjs14-SEO

which already includes all 6 previous steps (overview here) + the additional SEO topic regarding creating a complete Next.js 14 project from scratch.

We add the following env-variables to our .env.local file

NEXT_PUBLIC_ALGOLIA_APP_ID=xxxx -> Algolia ApplicationID 
NEXT_PUBLIC_ALGOLIA_API_KEY=xxxx -> Algolia Search-Only Key used later
ALGOLIA_MASTER_KEY=xxxx -> Algolia Admin Key (Write rights)
NEXT_PUBLIC_ALGOLIA_INDEX_NAME=example_dev
API_KEY=xxxxx -> Random Key, which we use for securing the endpoint

You can use this command for the creation of a new API key

node -e "console.log(crypto.randomBytes(32).toString('hex'))"

Endpoint creation in Next.js

For the new API endpoint, we create a new folder under /src/app/api called “algoliasync” with this route.ts file in it.

import { NextRequest, NextResponse } from "next/server";
import { documentToPlainTextString } from "@contentful/rich-text-plain-text-renderer";
import { client } from "@/lib/client";
import { fallbackLng, locales } from "@/app/i18n/settings";

const apikey = process.env.API_KEY;

export async function POST(request: NextRequest) {
  const document = await request.json();
  let json_de: string = "";
  let json_en: string = "";

  if (request.headers.get("x-api-key") !== apikey) {
    return new NextResponse(
      JSON.stringify({ status: "fail", message: "You are not authorized" }),
      { status: 401 }
    );
  }

  const locale = fallbackLng as string;

  const blogPagedata = await client.pageBlogPost({
    locale,
    preview: false,
    slug: document.slug,
  });

  const blogPost = blogPagedata.pageBlogPostCollection?.items[0];
  json_en = documentToPlainTextString(blogPost.content?.json);
  const tags = blogPost.contentfulMetadata?.tags;
  const tagsnew: any = [];

  tags.map((tag) => {
    tagsnew.push(tag.id);
  }); 

  const blogPagedata_de = await client.pageBlogPost({
    locale: "de-DE",
    preview: false,
    slug: document.slug,
  });

  const blogPost_de = blogPagedata_de.pageBlogPostCollection?.items[0];
  json_de = documentToPlainTextString(blogPost_de.content?.json);

  const AlgoliaUrl =
    "https://" +
    process.env.NEXT_PUBLIC_ALGOLIA_APP_ID +
    ".algolia.net/1/indexes/" +
    process.env.NEXT_PUBLIC_ALGOLIA_INDEX_NAME +
    "/" +
    document.entityId;

  const date = blogPost.publishedDate;
  const formatDate = new Date(date).toISOString().substring(0, 10);
  const timeStamp = new Date(date).getTime() / 1000;

  const jsonData: any = {
    entityId: document.entityId,
    height: blogPost?.featuredImage?.height.toString(),
    width: blogPost?.featuredImage?.width.toString(),
    image: blogPost?.featuredImage?.url,
    intName: document.intName,
    lang: {
      "de-DE": {
        content: json_de,
        shortDescription: document.shortDescription_de,
        title: document.title_de,
      },
      "en-US": {
        content: json_en,
        shortDescription: document.shortDescription_en,
        title: document.title_en,
      },
    },
    pubdate: formatDate,
    pubdatetimestamp: timeStamp,
    slug: document.slug,
    spaceId: document.spaceId,
    tags: tagsnew,
  };

  const res = await fetch(AlgoliaUrl, {
    method: "PUT",
    body: JSON.stringify(jsonData),
    headers: {
      "Content-Type": "application/json; charset=utf-8",
      "X-Algolia-API-Key": process.env.ALGOLIA_MASTER_KEY,
      "X-Algolia-Application-Id": process.env.NEXT_PUBLIC_ALGOLIA_APP_ID,
    },
  });

  if (!res.ok) {
    const text = await res.text();

    throw new Error(`
      Failed to fetch data
      Status: ${res.status}
      Response: ${text}
    `);
  }

  const result: any = await res.json();
  // console.log("result", result);

  return NextResponse.json(
    { transformed: true, date: Date.now() },
    { status: 200 }
  );
}

At the top, we import the needed packages

import { NextRequest, NextResponse } from "next/server"; -> standard Nextjs Api package
import { documentToPlainTextString } from "@contentful/rich-text-plain-text-renderer"; -> needed to extract the text part out of the JSON (Contentful Rich Text field)
import { client } from "@/lib/client"; -> client for Contentful graphQL operations
import { fallbackLng, locales } from "@/app/i18n/settings"; -> Language settings

The second step is the security check with the API Key

const apikey = process.env.API_KEY; -> getting API Key 

// Return 401 in case we provide the wrong "x-api-key" in the header

if (request.headers.get("x-api-key") !== apikey) {
    return new NextResponse(
      JSON.stringify({ status: "fail", message: "You are not authorized" }),
      { status: 401 }
    );
  }

Webhook creation in Contentful

The last step is setting up the new webhook within Contentful so that we push changes/deletions to the newly created index “example_dev”.

Here now are all the details for the webhook creation regarding the creation and updates of our data in Algolia

and here are the second webhooks for removing unpublished blog posts in Contentful from the Algolia index.

Algolia-Delete-unpublished-entries-Webhooks

In this tutorial, I used the project, that I created in the previous steps as a foundation and then the Contentful—Algolia sync was built on top.

Cloudapp-dev and before you leave

Thank you for reading until the end. Before you go:

Please consider clapping and following the writer! 👏 on our Medium Account

Extended content sync from Contentful to Algolia with Next.js 14