Next.js 14 -Data model extension and Contentful data sync with API route

Table of Contents

Used Stack
New API Route for Contentful Data fetch
New Lightweight GraphQL Query the Slug Attribute
Passing Slug to AlgoliaSync API Route
Defining JSON Data Body for Algolia
Using new Attributes in the Frontend
Complete code of the Visualization Component
Frontend with Highlight and Snippet Component
Cloudapp-dev, and before you leave us

We will walk through the process of a complete data re-sync between Contentful and Algolia with two Custom API Routes in Nextjs 14, and we will extend the Algolia Data Model as well. The story covers the full code and the configuration in the Algolia Backend.

Here is the GitHub repo with the entire code. Below, you will find the link to the example page.

Example page hosted on Vercel -> https://nextjs14-advanced-algoliasearch.vercel.app/

Used Stack

I will start with my default stack:

Next.js 14 as the web framework
TailwindCss for Styling
Contentful CMS (Free Plan)
Algolia as Search-Engine
Vercel for hosting

New API Route for Contentful Data fetch

As a first step, I created a new API route that gets all blog posts from the Contentful GraphQL Interface. For ease of use, I get all posts without filtering (where clause in the graphQL-Query). If someone needs a real-world example, please comment, and I will provide it.

I used 40 as a limit, but you can use what you need.

import { NextRequest, NextResponse } from "next/server";
import { client } from "@/lib/client";
import { fallbackLng, locales } from "@/app/i18n/settings";
import { PageBlogPostOrder } from "@/lib/__generated/sdk";

const apikey = process.env.API_KEY;

export async function POST(request: NextRequest) {
  if (request.headers.get("x-api-key") !== apikey) {
    return new NextResponse(
      JSON.stringify({ status: "fail", message: "You are not authorized" }),
      { status: 401 }
    );
  }

//fallbackLng is in our case en-US
  const locale = fallbackLng as string;

  // Getting BlogPosts
  const blogPostsData = await client.pageBlogPostCollectionSmall({
    limit: 40,
    locale: locale,
    skip: 0,
    order: PageBlogPostOrder.PublishedDateDesc,
    preview: false,
  });
  const posts = blogPostsData.pageBlogPostCollection?.items;
//getting the total count 
  const postCount = blogPostsData.pageBlogPostCollection?.total;

  if (!posts) {
    return new NextResponse(
      JSON.stringify({ status: "fail", message: "Blog posts not found" }),
      { status: 404 }
    );
  }

  const UpdateUrl = process.env.NEXT_PUBLIC_BASE_URL + "/api/algoliasync";

  const headers = {
    "Content-Type": "application/json; charset=utf-8",
    "x-api-key": process.env.API_KEY || "",
  };

  posts.map(async (post) => {
  // new const with slug attribute
    const jsonData = {
      slug: post?.slug,
    };

    // passing slug attribute to AlgoliaSync Route

    const res = await fetch(UpdateUrl, {
      method: "POST",
      body: JSON.stringify(jsonData),
      headers: headers,
    });

    if (!res.ok) {
      const text = await res.text(); // get the response body for more information

      throw new Error(`
        Failed to fetch data
        Status: ${res.status}
        Response: ${text}
      `);
    }

    const result: any = await res.json();
  });

  return NextResponse.json(
    { posts: posts, postCount: postCount },
    { status: 200 }
  );
}

New Lightweight GraphQL Query the Slug Attribute

To reduce the load on the Contentful API, I created a new GraphQL-Query (pageBlogPostCollectionSmall.graphql), which only queries the “internalName” and the “slug” attribute, so I can greatly reduce the overhead.

fragment PageBlogPostFieldsSmall on PageBlogPost {
  internalName
  slug
}

query pageBlogPostCollectionSmall(
  $locale: String
  $preview: Boolean
  $limit: Int
  $skip: Int
  $order: [PageBlogPostOrder]
  $where: PageBlogPostFilter
) {
  pageBlogPostCollection(
    limit: $limit
    skip: $skip
    locale: $locale
    preview: $preview
    order: $order
    where: $where
  ) {
    total
    items {
      ...PageBlogPostFieldsSmall
    }
  }
}

Passing Slug to AlgoliaSync API Route

After returning the data from Contentful, we use the “Slug” attribute from the response to pass it to the second API Route “/api/algoliasync”. This API uses the slug attribute in the POST Body to get all the details for every post, which we then use to make a PUT request to the Algolia Service to create or update the corresponding record/item.

import { NextRequest, NextResponse } from "next/server";
import { documentToPlainTextString } from "@contentful/rich-text-plain-text-renderer";
import { client } from "@/lib/client";
import { fallbackLng, locales } from "@/app/i18n/settings";
import { title } from "process";

const apikey = process.env.API_KEY;

export async function POST(request: NextRequest) {
  const document = await request.json();
  let json_de: string = "";
  let json_en: string = "";

  if (request.headers.get("x-api-key") !== apikey) {
    return new NextResponse(
      JSON.stringify({ status: "fail", message: "You are not authorized" }),
      { status: 401 }
    );
  }

  const locale = fallbackLng as string;

  const blogPagedata = await client.pageBlogPost({
    locale,
    preview: false,
    slug: document.slug,
  });

  const blogPost = blogPagedata.pageBlogPostCollection?.items[0];

  if (!blogPost) {
    return new NextResponse(
      JSON.stringify({ status: "fail", message: "Blog post not found" }),
      { status: 404 }
    );
  }

  json_en = documentToPlainTextString(blogPost.content?.json);
  const tags = blogPost.contentfulMetadata?.tags;
  const tagsnew: any = [];

  tags.map((tag) => {
    // console.log("tag", tag.id, tag.name);
    if (tag) tagsnew.push(tag.id);
  });

  const blogPagedata_de = await client.pageBlogPost({
    locale: "de-DE",
    preview: false,
    slug: document.slug,
  });

  const blogPost_de = blogPagedata_de.pageBlogPostCollection?.items[0];

  if (!blogPost_de) {
    return new NextResponse(
      JSON.stringify({
        status: "fail",
        message: "Blog post for language de-DE not found",
      }),
      { status: 404 }
    );
  }
  json_de = documentToPlainTextString(blogPost_de.content?.json);

  const AlgoliaUrl =
    "https://" +
    process.env.NEXT_PUBLIC_ALGOLIA_APP_ID +
    ".algolia.net/1/indexes/" +
    process.env.NEXT_PUBLIC_ALGOLIA_INDEX_NAME +
    "/" +
    blogPost.sys.id;

  const date = blogPost.publishedDate;
  const formatDate = new Date(date).toISOString().substring(0, 10);
  const timeStamp = new Date(date).getTime() / 1000;

  const jsonData: any = {
    entityId: blogPost.sys.id,
    height: blogPost?.featuredImage?.height?.toString() || "0",
    width: blogPost?.featuredImage?.width?.toString() || "0",
    image: blogPost?.featuredImage?.url,
    intName: blogPost.internalName,
    lang_de: json_de,
    lang_en: json_en,
    short_de: blogPost_de.shortDescription,
    short_en: blogPost.shortDescription,
    title_de: blogPost_de.title,
    title_en: blogPost.title,
    pubdate: formatDate,
    pubdatetimestamp: timeStamp,
    slug: document.slug,
    spaceId: blogPost.sys.spaceId,
    tags: tagsnew,
  };

  const headers = {
    "Content-Type": "application/json; charset=utf-8",
    "X-Algolia-API-Key": process.env.ALGOLIA_MASTER_KEY || "",
    "X-Algolia-Application-Id": process.env.NEXT_PUBLIC_ALGOLIA_APP_ID || "",
  };

  const res = await fetch(AlgoliaUrl, {
    method: "PUT",
    body: JSON.stringify(jsonData),
    headers: headers,
  });

  if (!res.ok) {
    const text = await res.text(); // get the response body for more information

    throw new Error(`
      Failed to fetch data
      Status: ${res.status}
      Response: ${text}
    `);
  }

  const result: any = await res.json();
  // console.log("result", result);

  return NextResponse.json(
    { transformed: true, date: Date.now(), slug: document.slug },
    { status: 200 }
  );
}

Defining JSON Data Body for Algolia

We are completely flexible regarding the data body that we create for Algolia. Algolia is a schemaless Index, so we can add and remove attributes.

const jsonData: any = {
    entityId: blogPost.sys.id,
    height: blogPost?.featuredImage?.height?.toString() || "0",
    width: blogPost?.featuredImage?.width?.toString() || "0",
    image: blogPost?.featuredImage?.url,
    intName: blogPost.internalName,
    lang_de: json_de,
    lang_en: json_en,
    short_de: blogPost_de.shortDescription,
    short_en: blogPost.shortDescription,
    title_de: blogPost_de.title,
    title_en: blogPost.title,
    pubdate: formatDate,
    pubdatetimestamp: timeStamp,
    slug: document.slug,
    spaceId: blogPost.sys.spaceId,
    tags: tagsnew,
  };

Using new Attributes in the Frontend

We adapt our type CardProps and add the six new attributes,

    lang_de: string;
    lang_en: string;
    short_de: string;
    short_en: string;
    title_de: string;
    title_en: string;

then I check the language with fallbackLng (“en-US”), so that I can show the right locale. On the top, I use the widget “Highlight” for the title attribute, and below that, I use the short attribute for the “Snippet” widget.

          {locale === fallbackLng ? (
            <Link href={`/${locale}/${result.slug}`}>
              <p className="mb-2 h3 line-clamp-2 text-gray-800 dark:text-[#AEC1CC] md:mb-3">
                <Highlight attribute="title_en" hit={result} />
              </p>
            </Link>
          ) : (
            <Link href={`/${locale}/${result.slug}`}>
              <p className="mb-2 h3 line-clamp-2 text-gray-800 dark:text-[#AEC1CC] md:mb-3">
                <Highlight attribute="title_de" hit={result} />
              </p>
            </Link>
          )}

          {locale === fallbackLng ? (
            <p className="mt-2 text-base line-clamp-2">
              <Snippet attribute="short_en" hit={result} />
            </p>
          ) : (
            <p className="mt-2 text-base line-clamp-2">
              <Snippet attribute="short_de" hit={result} />
            </p>
          )}

Complete code of the Visualization Component

"use client";

import Image from "next/image";
import Link from "next/link";
import { FormatDate } from "@/components/contentful/format-date/FormatDate";
import { useParams } from "next/navigation";
import type { LocaleTypes } from "@/app/i18n/settings";
import { twMerge } from "tailwind-merge";
import { ArticleLabel } from "@/components/contentful/ArticleLabel";
import { Hit as AlgoliaHit } from "instantsearch.js";
import { Highlight, Snippet } from "react-instantsearch";
import { fallbackLng } from "@/app/i18n/settings";

type CardProps = {
  result: AlgoliaHit<{
    intName: string;
    image: string;
    pubdate: Date;
    slug: string;
    width: number;
    height: number;
    tags: string[];
    lang_de: string;
    lang_en: string;
    short_de: string;
    short_en: string;
    title_de: string;
    title_en: string;
  }>;
};

export default function CardAlgolia({ result }: CardProps) {
  const locale = useParams()?.locale as LocaleTypes;

  const className = "md:grid-cols-2 lg:grid-cols-3";
  const classNameImage = "object-cover aspect-[16/10] w-full";

  const blurURL = new URL(result.image);
  blurURL.searchParams.set("w", "10");

  return (
    // {/* group - wird benötigt damit man unten im Classname darauf verweisen kann mit group-hover:.... */}
    <div className="flex flex-col">
      <div
        className={twMerge(
          "flex flex-1 flex-col overflow-hidden dark:shadow-white shadow-lg dark:shadow-sm-light",
          className
        )}
      >
        <Link href={`/${locale}/${result.slug}`}>
          <Image
            src={result.image}
            width={result.width || 722}
            height={result.height || 590}
            sizes="(max-width: 1200px) 100vw, 50vw"
            placeholder="blur"
            blurDataURL={blurURL.toString()}
            alt={
              (locale == fallbackLng ? result.title_en : result.title_de) || ""
            }
            className={twMerge(classNameImage, "transition-all")}
          ></Image>
        </Link>
        <div className="flex flex-col flex-1 px-4 py-3 dark:bg-gray-800 md:px-5 md:py-4 lg:px-7 lg:py-5">
          {locale === fallbackLng ? (
            <Link href={`/${locale}/${result.slug}`}>
              <p className="mb-2 h3 line-clamp-2 text-gray-800 dark:text-[#AEC1CC] md:mb-3">
                <Highlight attribute="title_en" hit={result} />
              </p>
            </Link>
          ) : (
            <Link href={`/${locale}/${result.slug}`}>
              <p className="mb-2 h3 line-clamp-2 text-gray-800 dark:text-[#AEC1CC] md:mb-3">
                <Highlight attribute="title_de" hit={result} />
              </p>
            </Link>
          )}

          {locale === fallbackLng ? (
            <p className="mt-2 text-base line-clamp-2">
              <Snippet attribute="short_en" hit={result} />
            </p>
          ) : (
            <p className="mt-2 text-base line-clamp-2">
              <Snippet attribute="short_de" hit={result} />
            </p>
          )}

          <div className="flex flex-wrap max-w-2xl gap-2 mr-auto">
            {result.tags.map((tag: string, index) => (
              <Link href={`/${locale}/search/${tag}`} key={index}>
                <ArticleLabel className="flex items-center ml-1">
                  {tag}
                </ArticleLabel>
              </Link>
            ))}
            <div className={twMerge("ml-auto pl-2 text-xs text-gray-600")}>
              <FormatDate date={result.pubdate} />
            </div>
          </div>
        </div>
      </div>
    </div>
  );
}