GitHub - ariesclark/next-sitemap-x: Sitemap generator for next.js. Generate sitemap(s) and robots.txt for all static/pre-rendered/dynamic/server-side pages.

Getting started

Installation

pnpm add next-sitemap-x

Create config file

next-sitemap-x requires a basic config file (next-sitemap-x.config.js) under your project root

✅ next-sitemap-x will load environment variables from .env files by default.

/** @type {import('next-sitemap-x').IConfig} */
module.exports = {
  siteUrl: process.env.SITE_URL || 'https://example.com',
  generateRobotsTxt: true, // (optional)
  // ...other options
}

Building sitemaps

Add next-sitemap-x as your postbuild script

{
  "build": "next build",
  "postbuild": "next-sitemap-x"
}

Custom config file

You can also use a custom config file instead of next-sitemap-x.config.js. Just pass --config <your-config-file>.js to build command (Example: custom-config-file)

{
  "build": "next build",
  "postbuild": "next-sitemap-x --config awesome.config.js"
}

Building sitemaps with pnpm

When using pnpm you need to create a .npmrc file in the root of your project if you want to use a postbuild step:

//.npmrc
enable-pre-post-scripts=true

Index sitemaps (Optional)

📣 From next-sitemap-x v2.x onwards, sitemap.xml will be Index Sitemap. It will contain urls of all other generated sitemap endpoints.

Index sitemap generation can be turned off by setting generateIndexSitemap: false in next-sitemap-x config file. (This is useful for small/hobby sites which does not require an index sitemap) (Example: no-index-sitemaps)

Splitting large sitemap into multiple files

Define the sitemapSize property in next-sitemap-x.config.js to split large sitemap into multiple files.

/** @type {import('next-sitemap-x').IConfig} */
module.exports = {
  siteUrl: 'https://example.com',
  generateRobotsTxt: true,
  sitemapSize: 7000,
}

Above is the minimal configuration to split a large sitemap. When the number of URLs in a sitemap is more than 7000, next-sitemap-x will create sitemap (e.g. sitemap-0.xml, sitemap-1.xml) and index (e.g. sitemap.xml) files.

Configuration Options

property	description	type
siteUrl	Base url of your website	string
output (optional)	Next.js output modes. Check documentation.	`standalone`, `export`
changefreq (optional)	Change frequency. Default `daily`	string
priority (optional)	Priority. Default `0.7`	number
sitemapBaseFileName (optional)	The name of the generated sitemap file before the file extension. Default `"sitemap"`	string
alternateRefs (optional)	Denote multi-language support by unique URL. Default `[]`	AlternateRef[]
sitemapSize(optional)	Split large sitemap into multiple files by specifying sitemap size. Default `5000`	number
autoLastmod (optional)	Add `<lastmod/>` property. Default `true`	true
exclude (optional)	Array of relative paths (wildcard pattern supported) to exclude from listing on `sitemap.xml` or `sitemap-.xml`. e.g.: `['/page-0', '/page-', '/private/*']`. Apart from this option `next-sitemap-x` also offers a custom `transform` option which could be used to exclude urls that match specific patterns	string[]
sourceDir (optional)	next.js build directory. Default `.next`	string
outDir (optional)	All the generated files will be exported to this directory. Default `public`	string
transform (optional)	A transformation function, which runs for each `relative-path` in the sitemap. Returning `null` value from the transformation function will result in the exclusion of that specific `path` from the generated sitemap list.	async function
additionalPaths (optional)	Async function that returns a list of additional paths to be added to the generated sitemap list.	async function
generateIndexSitemap	Generate index sitemaps. Default `true`	boolean
generateRobotsTxt (optional)	Generate a `robots.txt` file and list the generated sitemaps. Default `false`	boolean
robotsTxtOptions.transformRobotsTxt (optional)	Custom robots.txt transformer function. (Example: custom-robots-txt-transformer) Default: `async(config, robotsTxt)=> robotsTxt`	async function
robotsTxtOptions.policies (optional)	Policies for generating `robots.txt`. Default: `[{ userAgent: '*', allow: '/' }]`	IRobotPolicy[]
robotsTxtOptions.additionalSitemaps (optional)	Options to add additional sitemaps to `robots.txt` host entry	string[]
robotsTxtOptions.includeNonIndexSitemaps (optional)	From v2.4x onwards, generated `robots.txt` will only contain url of `index sitemap` and custom provided endpoints from `robotsTxtOptions.additionalSitemaps`. This is to prevent duplicate url submission (once through index-sitemap -> sitemap-url and once through robots.txt -> HOST) Set this option `true` to add all generated sitemap endpoints to `robots.txt` Default `false` (Recommended)	boolean

Custom transformation function

Custom transformation provides an extension method to add, remove or exclude path or properties from a url-set. Transform function runs for each relative path in the sitemap. And use the key: value object to add properties in the XML.

Returning null value from the transformation function will result in the exclusion of that specific relative-path from the generated sitemap list.

/** @type {import('next-sitemap-x').IConfig} */
module.exports = {
  transform: async (config, path) => {
    // custom function to ignore the path
    if (customIgnoreFunction(path)) {
      return null
    }

    // only create changefreq along with path
    // returning partial properties will result in generation of XML field with only returned values.
    if (customLimitedField(path)) {
      // This returns `path` & `changefreq`. Hence it will result in the generation of XML field with `path` and  `changefreq` properties only.
      return {
        loc: path, // => this will be exported as http(s)://<config.siteUrl>/<path>
        changefreq: 'weekly',
      }
    }

    // Use default transformation for all other cases
    return {
      loc: path, // => this will be exported as http(s)://<config.siteUrl>/<path>
      changefreq: config.changefreq,
      priority: config.priority,
      lastmod: config.autoLastmod ? new Date().toISOString() : undefined,
      alternateRefs: config.alternateRefs ?? [],
    }
  },
}

Additional paths function

additionalPaths this function can be useful if you have a large list of pages, but you don't want to render them all and use fallback: true. Result of executing this function will be added to the general list of paths and processed with sitemapSize. You are free to add dynamic paths, but unlike additionalSitemap, you do not need to split the list of paths into different files in case there are a lot of paths for one file.

If your function returns a path that already exists, then it will simply be updated, duplication will not happen.

/** @type {import('next-sitemap-x').IConfig} */
module.exports = {
  additionalPaths: async (config) => {
    const result = []

    // required value only
    result.push({ loc: '/additional-page-1' })

    // all possible values
    result.push({
      loc: '/additional-page-2',
      changefreq: 'yearly',
      priority: 0.7,
      lastmod: new Date().toISOString(),

      // acts only on '/additional-page-2'
      alternateRefs: [
        {
          href: 'https://es.example.com',
          hreflang: 'es',
        },
        {
          href: 'https://fr.example.com',
          hreflang: 'fr',
        },
      ],
    })

    // using transformation from the current configuration
    result.push(await config.transform(config, '/additional-page-3'))

    return result
  },
}

Google News, image and video sitemap

Url set can contain additional sitemaps defined by google. These are Google News sitemap, image sitemap or video sitemap. You can add the values for these sitemaps by updating entry in transform function or adding it with additionalPaths. You have to return a sitemap entry in both cases, so it's the best place for updating the output. This example will add an image and news tag to each entry but IRL you would of course use it with some condition or within additionalPaths result.

/** @type {import('next-sitemap-x').IConfig} */
const config = {
  transform: async (config, path) => {
    return {
      loc: path, // => this will be exported as http(s)://<config.siteUrl>/<path>
      changefreq: config.changefreq,
      priority: config.priority,
      lastmod: config.autoLastmod ? new Date().toISOString() : undefined,
      images: [{ loc: 'https://example.com/image.jpg' }],
      news: {
        title: 'Article 1',
        publicationName: 'Google Scholar',
        publicationLanguage: 'en',
        date: new Date(),
      },
    }
  },
}

export default config

Full configuration example

Here's an example next-sitemap-x.config.js configuration with all options

/** @type {import('next-sitemap-x').IConfig} */

module.exports = {
  siteUrl: 'https://example.com',
  changefreq: 'daily',
  priority: 0.7,
  sitemapSize: 5000,
  generateRobotsTxt: true,
  exclude: ['/protected-page', '/awesome/secret-page'],
  alternateRefs: [
    {
      href: 'https://es.example.com',
      hreflang: 'es',
    },
    {
      href: 'https://fr.example.com',
      hreflang: 'fr',
    },
  ],
  // Default transformation function
  transform: async (config, path) => {
    return {
      loc: path, // => this will be exported as http(s)://<config.siteUrl>/<path>
      changefreq: config.changefreq,
      priority: config.priority,
      lastmod: config.autoLastmod ? new Date().toISOString() : undefined,
      alternateRefs: config.alternateRefs ?? [],
    }
  },
  additionalPaths: async (config) => [
    await config.transform(config, '/additional-page'),
  ],
  robotsTxtOptions: {
    policies: [
      {
        userAgent: '*',
        allow: '/',
      },
      {
        userAgent: 'test-bot',
        allow: ['/path', '/path-2'],
      },
      {
        userAgent: 'black-listed-bot',
        disallow: ['/sub-path-1', '/path-2'],
      },
    ],
    additionalSitemaps: [
      'https://example.com/my-custom-sitemap-1.xml',
      'https://example.com/my-custom-sitemap-2.xml',
      'https://example.com/my-custom-sitemap-3.xml',
    ],
  },
}

Above configuration will generate sitemaps based on your project and a robots.txt like this.

# *
User-agent: *
Allow: /

# test-bot
User-agent: test-bot
Allow: /path
Allow: /path-2

# black-listed-bot
User-agent: black-listed-bot
Disallow: /sub-path-1
Disallow: /path-2

# Host
Host: https://example.com

# Sitemaps
Sitemap: https://example.com/sitemap.xml # Index sitemap
Sitemap: https://example.com/my-custom-sitemap-1.xml
Sitemap: https://example.com/my-custom-sitemap-2.xml
Sitemap: https://example.com/my-custom-sitemap-3.xml

Generating dynamic/server-side sitemaps

next-sitemap-x now provides two APIs to generate server side sitemaps. This will help to dynamically generate index-sitemap(s) and sitemap(s) by sourcing data from CMS or custom source.

getServerSideSitemapIndex: Generates index sitemaps based on urls provided and returns application/xml response. Supports next13+ route.{ts,js} file.
- To continue using inside pages directory, import getServerSideSitemapIndexLegacy instead.
getServerSideSitemap: Generates sitemap based on field entires and returns application/xml response. Supports next13+ route.{ts,js} file.
- To continue using inside pages directory, import getServerSideSitemapLegacy instead.

Server side index-sitemaps (getServerSideSitemapIndex)

Here's a sample script to generate index-sitemap on server side.

1. Index sitemap (app directory)

Create app/server-sitemap-index.xml/route.ts file.

// app/server-sitemap-index.xml/route.ts
import { getServerSideSitemapIndex } from 'next-sitemap-x'

export async function GET(request: Request) {
  // Method to source urls from cms
  // const urls = await fetch('https//example.com/api')

  return getServerSideSitemapIndex([
    'https://example.com/path-1.xml',
    'https://example.com/path-2.xml',
  ])
}

2. Index sitemap (pages directory) (legacy)

Create pages/server-sitemap-index.xml/index.tsx file.

// pages/server-sitemap-index.xml/index.tsx
import { getServerSideSitemapIndexLegacy } from 'next-sitemap-x'
import { GetServerSideProps } from 'next'

export const getServerSideProps: GetServerSideProps = async (ctx) => {
  // Method to source urls from cms
  // const urls = await fetch('https//example.com/api')

  return getServerSideSitemapIndexLegacy(ctx, [
    'https://example.com/path-1.xml',
    'https://example.com/path-2.xml',
  ])
}

// Default export to prevent next.js errors
export default function SitemapIndex() {}

Exclude server index sitemap from robots.txt

Now, next.js is serving the dynamic index-sitemap from http://localhost:3000/server-sitemap-index.xml.

List the dynamic sitemap page in robotsTxtOptions.additionalSitemaps and exclude this path from static sitemap list.

// next-sitemap-x.config.js

/** @type {import('next-sitemap-x').IConfig} */
module.exports = {
  siteUrl: 'https://example.com',
  generateRobotsTxt: true,
  exclude: ['/server-sitemap-index.xml'], // <= exclude here
  robotsTxtOptions: {
    additionalSitemaps: [
      'https://example.com/server-sitemap-index.xml', // <==== Add here
    ],
  },
}

In this way, next-sitemap-x will manage the sitemaps for all your static pages and your dynamic index-sitemap will be listed on robots.txt.

server side sitemap (getServerSideSitemap)

Here's a sample script to generate sitemaps on server side.

1. Sitemaps (app directory)

Create app/server-sitemap.xml/route.ts file.

// app/server-sitemap.xml/route.ts
import { getServerSideSitemap } from 'next-sitemap-x'

export async function GET(request: Request) {
  // Method to source urls from cms
  // const urls = await fetch('https//example.com/api')

  return getServerSideSitemap([
    {
      loc: 'https://example.com',
      lastmod: new Date().toISOString(),
      // changefreq
      // priority
    },
    {
      loc: 'https://example.com/dynamic-path-2',
      lastmod: new Date().toISOString(),
      // changefreq
      // priority
    },
  ])
}

2. Sitemaps (pages directory) (legacy)

Create pages/server-sitemap.xml/index.tsx file.

// pages/server-sitemap.xml/index.tsx
import { getServerSideSitemapLegacy } from 'next-sitemap-x'
import { GetServerSideProps } from 'next'

export const getServerSideProps: GetServerSideProps = async (ctx) => {
  // Method to source urls from cms
  // const urls = await fetch('https//example.com/api')

  const fields = [
    {
      loc: 'https://example.com', // Absolute url
      lastmod: new Date().toISOString(),
      // changefreq
      // priority
    },
    {
      loc: 'https://example.com/dynamic-path-2', // Absolute url
      lastmod: new Date().toISOString(),
      // changefreq
      // priority
    },
  ]

  return getServerSideSitemapLegacy(ctx, fields)
}

// Default export to prevent next.js errors
export default function Sitemap() {}

Now, next.js is serving the dynamic sitemap from http://localhost:3000/server-sitemap.xml.

List the dynamic sitemap page in robotsTxtOptions.additionalSitemaps and exclude this path from static sitemap list.

// next-sitemap-x.config.js

/** @type {import('next-sitemap-x').IConfig} */
module.exports = {
  siteUrl: 'https://example.com',
  generateRobotsTxt: true,
  exclude: ['/server-sitemap.xml'], // <= exclude here
  robotsTxtOptions: {
    additionalSitemaps: [
      'https://example.com/server-sitemap.xml', // <==== Add here
    ],
  },
}

In this way, next-sitemap-x will manage the sitemaps for all your static pages and your dynamic sitemap will be listed on robots.txt.

Typescript JSDoc

Add the following line of code in your next-sitemap-x.config.js for nice typescript autocomplete! 💖

/** @type {import('next-sitemap-x').IConfig} */
module.exports = {
  // YOUR CONFIG
}

Contribution

All PRs are welcome :)

Name		Name	Last commit message	Last commit date
Latest commit History 940 Commits
.github		.github
.vscode		.vscode
assets		assets
docs		docs
examples		examples
packages/next-sitemap-x		packages/next-sitemap-x
.eslintignore		.eslintignore
.eslintrc		.eslintrc
.gitattributes		.gitattributes
.gitignore		.gitignore
.npmrc		.npmrc
.prettierignore		.prettierignore
.prettierrc		.prettierrc
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
LICENSE		LICENSE
README.md		README.md
SECURITY.md		SECURITY.md
azure-pipeline.yml		azure-pipeline.yml
changelog.md		changelog.md
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
tsconfig.json		tsconfig.json
turbo.json		turbo.json

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

Table of contents

Getting started

Installation

Create config file

Building sitemaps

Custom config file

Building sitemaps with pnpm

Index sitemaps (Optional)

Splitting large sitemap into multiple files

Configuration Options

Custom transformation function

Additional paths function

Google News, image and video sitemap

Full configuration example

Generating dynamic/server-side sitemaps

Server side index-sitemaps (getServerSideSitemapIndex)

Exclude server index sitemap from robots.txt

server side sitemap (getServerSideSitemap)

Typescript JSDoc

Contribution

About

Releases

Packages

Languages

License

ariesclark/next-sitemap-x

Folders and files

Latest commit

History

Repository files navigation

Table of contents

Getting started

Installation

Create config file

Building sitemaps

Custom config file

Building sitemaps with pnpm

Index sitemaps (Optional)

Splitting large sitemap into multiple files

Configuration Options

Custom transformation function

Additional paths function

Google News, image and video sitemap

Full configuration example

Generating dynamic/server-side sitemaps

Server side index-sitemaps (getServerSideSitemapIndex)

Exclude server index sitemap from robots.txt

server side sitemap (getServerSideSitemap)

Typescript JSDoc

Contribution

About

Resources

License

Code of conduct

Security policy

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages