After working on this site for nearly 6 months (1 Yr for some), its finally live. We launched 20+ ccTLD’s, 6 languages and a mind boggling amount of content. We have worked with some great teams to get AllHotels up and fluid for future development. Its only the beginning. Big thanks to Evolving Web ( a Drupal Development group) and EpicOne (Web Analytics Consultants). Not to mention the many coworkers that have wrangled this beast of a site into production.

[button link="http://www.allhotels.com" color="orange"]Visit AllHotels.com[/button]

1. You can run a crawler tool on your site to pull millions of URL’s which will take loads of time.
2. Create a script that will generate the sitemaps for you provided you have logical page names and parameters.

A guide to create your sitemap:

1. Define all the page types and URL parameters.
2. Read all about sitemaps.org protocols.
3. Use multiple sitemaps and group them into relevant categories such as media type, country or product type.
4. Make sure the URL’s are non sequential. Google is not a fan of ABC and 123 order so mix it up.
5. Find a way to prioritize your URL’s. What pages are most important?
6. Use sitemaps index files.
7. Stay within 50K max urls in a sitemap or sitemap index file.
8. Compress the files with gZip.
9. Load them to your site and submit them to the search engines.

A sitemap for millions of URL’s can be over 10MB+ in size. Compressing the files and finding a way to create dynamic Sitemaps will help you use less space. It can get tricky but remember it CAN be done.