Wasp•4mo ago

How to create an insanely huge sitemap from our database

So I got a suggestion how to SEO the 💩 of our CMS, by exposing hundreds of thousands of routes based on our index tags (like https://myapp.co/tag1/tag2/tag3/some_media_page) and it occurred to me that I have no idea how to do that with wasp so thought I should throw this out there and see what comes back. 🙂

3 Replies

kapa.ai•4mo ago

Hi @conspyre. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave: ---------------------- Currently, Wasp doesn't have built-in functionality specifically for generating large sitemaps from database content. However, I can suggest a few approaches based on the available information. Current SEO Support in Wasp Wasp is primarily focused on building SPAs (Single Page Applications), which can present challenges for SEO as mentioned in the Discord discussions:

"Vite sets up SPA apps, so naturally SPAs have problems with SEO because everything is javascript, not much actual HTML gets delivered to robots." Discord support thread

Possible Approaches 1. Create Dynamic Routes You can create routes with parameters in Wasp similar to React Router:

route TagRoute { path: "/tag/:tag1/:tag2/:tag3/:mediaPage", to: TagPage }
page TagPage {
  component: import { TagPage } from "@src/TagPage"
}

route TagRoute { path: "/tag/:tag1/:tag2/:tag3/:mediaPage", to: TagPage }
page TagPage {
  component: import { TagPage } from "@src/TagPage"
}

Then in your component, you can access the parameters:

export const TagPage = (props) => {
  const { tag1, tag2, tag3, mediaPage } = props.match.params;
  // Fetch content based on these tags
  return <div>...</div>
}

export const TagPage = (props) => {
  const { tag1, tag2, tag3, mediaPage } = props.match.params;
  // Fetch content based on these tags
  return <div>...</div>
}

2. Use a Wildcard Route For extremely dynamic routing, you could use a wildcard route:

route WildcardRoute { path: "*", to: WildcardPage }
page WildcardPage {
  component: import { WildcardPage } from "@src/WildcardPage"
}

route WildcardRoute { path: "*", to: WildcardPage }
page WildcardPage {
  component: import { WildcardPage } from "@src/WildcardPage"
}

As noted in a Discord thread:

"This probably needs to be the last route in the main.wasp though right?" Discord support thread

3. Generate a Sitemap File For the actual sitemap generation, you would need to: 1. Create a server-side job or action that queries your database 2. Generate an XML sitemap file with all your routes 3. Serve this file at /sitemap.xml You could implement this as a Wasp job that runs periodically to update your sitemap. SEO Considerations Keep in mind that Wasp currently has limited built-in SEO support:

"For SEO you don't even need SSR, there is a simpler mechanism (tags) that can give you that. We don't have support for either yet though, but we will be adding it in the future" Discord support thread

Some Wasp users have implemented SEO solutions in their projects. For example:

"You can check out CoverLetterGPT.xyz's main.wasp file here (it's open-source): https://github.com/vincanger/coverlettergpt/blob/migrate-v0.12/main.wasp#L6-L16" Discord support thread

Future Improvements The Wasp team is aware of SEO needs and has open issues related to this:

"Add 'SEO' section in our docs" Github Issue

If you need more specific guidance, you might want to reach out to the Wasp team directly on Discord. ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ---------------------- Sorry @conspyre, I seem to be having a bit of trouble. Don't worry though, I've pinged my support and they will try to fix me up! :technologist: Perhaps try again in 15mins?

Filip•4mo ago

Haha, thanks for reporting @conspyre 😄 Yeah... we can't generate a sitemap yet. Someone did ask about it just the other day (and someone else a couple days before that), so it's becoming an increasingly popular request. I've added this message to the relevant issue: https://github.com/wasp-lang/wasp/issues/1944#issuecomment-2890665036 We'll go through it during the next planning/prioritization meeting so I recommend you subscribe to the isse In the meantime, I think the only option is using an external tool (like a scraper or a prerenderer) to generate the sitemap. I'm a little rusty on that front and can't really say what's the most modern approach

genyus•4mo ago

If I understand you correctly (and correct me if I don't), is your use-case that you want to use a Wasp instance to generate tag-based routes for the content coming from your CMS and then build an indexable sitemap to expose them all?

Gaming

Programming

How to create an insanely huge sitemap from our database

Did you find this page helpful?