Webmaster Forums - Website and SEO Help

Microsys Products and Webmaster Tools => A1 Sitemap Generator => Topic started by: mkv4 on July 18, 2012, 02:45:43 PM

Title: Crawl Only Selected Pages - Video Sitemap Creation
Post by: mkv4 on July 18, 2012, 02:45:43 PM
I've been trying to figure this out for a few hours now.

I'm trying to generate a video sitemap of videos that appear only on certain pages on my website so I can submit the sitemap in Webmaster Tools. I have a list of all of the URLs where the videos appear on my site, and I want A1 Sitemap Generator to crawl and analyze ONLY that list of pages (it's about 3000 URLs). Is there a way to enter that list, crawl only those pages, and then generate a video sitemap based on that analysis? THANKS!
Title: Re: Crawl Only Selected Pages - Video Sitemap Creation
Post by: Webhelpforums on July 18, 2012, 06:28:12 PM
Hi,

It should not be a problem.


1)
Switch of "easy mode"
http://www.microsystools.com/products/sitemap-generator/help/easy-sitemap-generator-mode/ (http://www.microsystools.com/products/sitemap-generator/help/easy-sitemap-generator-mode/)

2)
Follow video sitemap tutorial
http://www.microsystools.com/products/sitemap-generator/help/video-xml-sitemaps/ (http://www.microsystools.com/products/sitemap-generator/help/video-xml-sitemaps/)

3)
Then read this how to import a list of URLs and crawl those:
http://www.microsystools.com/products/sitemap-generator/help/website-sitemap-pages-list/ (http://www.microsystools.com/products/sitemap-generator/help/website-sitemap-pages-list/)
Do make sure you don't end up excluding video and image file locations

...

Another method instead of importing would be step 1+2 and then manually create exclude analysis filters
http://www.microsystools.com/products/sitemap-generator/help/website-crawler-scanner-filters/ (http://www.microsystools.com/products/sitemap-generator/help/website-crawler-scanner-filters/)
before scanning the website


Please write and/or email back if you can not get it working :)
Title: Re: Crawl Only Selected Pages - Video Sitemap Creation
Post by: mkv4 on July 19, 2012, 09:33:27 AM
Hello,

Thanks for the reply. I'm still having some trouble getting this to work. I know now (I think) that it's related to my inability to prevent crawling of pages I don't want crawled.

If, for example, I wanted the crawler to crawl only example.com/examplepage/ and not follow internal/external links and then crawl those pages, how would I tell A1 to crawl only the URLs I specify and not follow any outbound links and crawl them?

Remember that I'm only using the program to generate a video sitemap. I want A1 to crawl example.com/example/ and create a sitemap that contains the embedded video on that page only. I already have the list of all of the URLs on which the videos appear; I'm simply trying to save time by having A1 generate the video sitemap instead of having to do it manually.

Any ideas? Thanks again for your help.
Title: Re: Crawl Only Selected Pages - Video Sitemap Creation
Post by: Webhelpforums on July 19, 2012, 04:17:07 PM
After importing your URL list
see http://www.microsystools.com/products/sitemap-generator/help/website-sitemap-pages-list/ (http://www.microsystools.com/products/sitemap-generator/help/website-sitemap-pages-list/)

you can select all your page URLs and add them all to the "limit to" list in "output filters" and "analysis filters"
(by using the green button, see help, #2 screenshot)

Then use the video sitemap preset.

And hit the "start scan" button

...

If that still does not work, please drop an email to support with the relevant files attached:
http://www.microsystools.com/home/contact.php (http://www.microsystools.com/home/contact.php)