Webmaster Forums - Website and SEO Help

Microsys Products and Webmaster Tools => A1 Sitemap Generator => Topic started by: Webhelpforums on May 10, 2013, 06:43:06 AM

Title: Sitemap generator feature suggestions
Post by: Webhelpforums on May 10, 2013, 06:43:06 AM
Please post suggestions you have for A1 Sitemap Generator here :)
Title: Re: Sitemap generator feature suggestions
Post by: Burger2go on May 10, 2013, 10:09:02 AM
Please add feature to allow a searchable page tag that could be used to bypass listing specific pages when generating a sitemap using A1SG.  This would be geared more towards internal intranet sites as opposed to public facing internet websites.

Rationale:

SharePoint allows for all kinds of pages to be on a site. There are webmaster/site developer/chrome/template type pages, that one may want to filter from the actual "data" on the site.

In these cases it would be beneficial  to have a way to filter these pages from a sitemap which is exposed to the business visitors coming to the site. They are interested only in their pertinent data, and the
ability to find what they need quickly, without having to wade though a bunch of things that are not applicable to them, such as the layout template pages.

If there is a naming standard for pages in the site's URLs then that would possibly allow for some current filter that you have in place for URLs (you asked about this in a previous email). If there are several
types/naming stds in play for pages to be filtered, then using URLs will require compound logic to achieve what a simple single tag would fulfil. However, if that naming standard is not in place, the using a URL name as a filter will not work period.

Having a specific filtering tag somewhere on the page (I don't know where this would be put, but it would have to be wherever tags on pages are in SharePoint (2010 and up) which makes perfect sense.

We are mandated to use Vanilla SP – No designer) that would allow bypassing of the pages with
this tag, would seem to be the best way to accomplish this.

Once filtered, the sitemap displayed would only contain relevant pages to the business user.
The better the site is built and accessible the more likely the users are to use it, come back to it, and tell their
friends about it.

The site we are building is already a hard sell within the organization (Enterprise Architecture), so I f we can get visitors there at least once and provide them a good experience, they are more likely to come back and pass along the good word.

Cheers!
Garry
Title: Prestashop specific crawling quick settings
Post by: seofreelance on September 08, 2013, 11:03:38 AM
Hi,
Interest in Prestashop is raising as seen and consulted with many colleagues; I'm getting more and more quote requests for ecommerce SEO projects based on this platform: I would find very interesting a "pre-minded" A1SG for evaluating such sites. Is that in pipeline? If so, expected release? Thanks :)
Title: Re: Sitemap generator feature suggestions
Post by: Webhelpforums on September 08, 2013, 12:50:54 PM
Do you mean a scan preset that e.g. sets all analysis filters + output filters in a way so both website crawl and sitemaps generated are optimal and best quality?

Do note that scan presets usually just set options in

So until a preset is available, you can edit settngs yourself :)

That said, do you have some example websites that is running Prestashop?
Title: Re: Sitemap generator feature suggestions
Post by: seofreelance on September 09, 2013, 06:21:35 AM
Hi,
Thanks for the previous info.
Some Prestashop examples (not massive sites):
Title: Bulk project scan with ASG1
Post by: seofreelance on October 27, 2014, 12:01:20 PM
Would a future release have the ability to create bulk crawls?
Business case: for example to scan 9 huge competitors of a client during night time, say 1 at a time.
Thanks
Title: Re: Bulk project scan with ASG1
Post by: Webhelpforums on October 27, 2014, 12:56:06 PM
Quote from: seofreelance on October 27, 2014, 12:01:20 PM
Would a future release have the ability to create bulk crawls?
Business case: for example to scan 9 huge competitors of a client during night time, say 1 at a time.
Thanks

This is already possible through the command line inteface where you can both load predefined complete projects or a base project where you override some settings through the command line.

See this help page:
http://www.microsystools.com/products/sitemap-generator/help/automate-sitemap-build-upload/ (http://www.microsystools.com/products/sitemap-generator/help/automate-sitemap-build-upload/)
Title: Re: Sitemap generator feature suggestions
Post by: seofreelance on October 28, 2014, 04:15:28 AM
Thank you for the reference, will study it carefully.
Title: Re: Sitemap generator feature suggestions
Post by: Webhelpforums on December 03, 2014, 07:49:19 AM
I received this suggesion from Peter

QuoteOur goal is to run the tool from a scheduled task to crawl our site and generate a sitemap on a nightly basis.  We installed the Microsys A1 Sitemap Generator and have a basic proof-of-concept working.  We are launching it as a scheduled task in a batch file that contains the following call:

    C:\Program Files\Microsys\Sitemap Generator 5\Sitemap.exe -hide
    -exit -scan -build -robotstxt ":C:\path\to\config.ini"

So far, this has worked great for generating the sitemap.  However, we also need some mechanism to determine whether the sitemap generator encountered an error, and pass this back to the Windows Task Scheduler as an error.

We ran some tests on basic failure modes, but it seems like the %errorlevel% environment variable does not change when the scan / build operations fail.  Is there a setting that we can change to achieve this behavior?
Title: Pixel length included in export
Post by: seofreelance on March 16, 2015, 01:26:35 PM
Hi,
I use to export the CSVs to Excel and cross info with other aplications; a useful trick is to create a new column right next to TITLE column and use the formula "long()" so it calculates the character length of the title and saves explanations to copy people or whoever... Having pixel width next to [Title][Lenght][Pixel?] would be precious, as this cannot be achieved easily with Excel.
Thanks!
Title: Re: Sitemap generator feature suggestions
Post by: Webhelpforums on March 17, 2015, 07:26:24 AM
Hi,


A1 Website Analyzer already contains this information:

For reference it contains:

If it has been a while since you last check A1 Website Analyzer, be sure to give it a whirl again since it has continously received more data columns and reporting functions:
http://www.microsystools.com/products/website-analyzer/help/site-analysis-seo-audit/ (http://www.microsystools.com/products/website-analyzer/help/site-analysis-seo-audit/)
Title: Re: Sitemap generator feature suggestions
Post by: seofreelance on March 17, 2015, 12:39:36 PM
I have downloaded A1WA for a test, ages ago I did but I think I even lost the chance to evaluate it due to workload...
Will give it a try :) thanks
Title: Re: Sitemap generator feature suggestions
Post by: Webhelpforums on March 17, 2015, 06:22:09 PM
If the trial expired, drop me an email http://www.microsystools.com/home/contact.php (http://www.microsystools.com/home/contact.php), so I can extend it for you :)
Title: Re: Sitemap generator feature suggestions
Post by: seofreelance on March 18, 2015, 04:29:34 AM
No prob, it says "31 days". Will give a try to Keywords as well  ;)
Title: Re: Sitemap generator feature suggestions
Post by: Webhelpforums on September 24, 2015, 07:48:20 PM
Received this suggestion from Greg

QuoteIs there a way for the A1 Sitemap Generator to show any new pages that have been added to a website since its last scan? 

Title: Re: Sitemap generator feature suggestions
Post by: Webhelpforums on October 20, 2017, 07:29:21 PM
Joel suggested this:

Adobe Muse program 
http://www.adobe.com/products/muse.html
apparently creates HTML like this:

<img
class="shadow position_content temp_no_img_src" id="u235846_img"

src="images/blank-fake-image.gif?crc=xxxxxxxxx"

ata-muse-src="images/real-image.jpg?crc=xxxxxxx"
data-orig-src="images/real-image.jpg?crc=xxxxxxx"

alt="my alt example"
title="my title example"

data-image-width="426"
data-image-height="712"
data-heightwidthratio="1.6713615023474178"
/>


That makes  A1SG associate title and alt to the "src" image and not the image references in the custom "data-" attributes
Title: Re: Sitemap generator feature suggestions
Post by: Webhelpforums on October 20, 2017, 07:37:30 PM
Мыць suggested:

If you in robots.txt

"Disallow: /components/"

you normally deny a crawler access to entire folder.

But Google / GoogleBots unlike other crawlers allows extended robots.txt syntax that can override and allow a subfolder/url in a disallow folder/url e.g.



Title: Re: Sitemap generator feature suggestions
Post by: inmoon on September 17, 2019, 11:47:24 AM
Where is the download link for these programs?
Title: Re: Sitemap generator feature suggestions
Post by: Webhelpforums on November 28, 2019, 11:32:59 AM
You can get A1 Sitemap Generator here:
https://www.microsystools.com/products/sitemap-generator/ (https://www.microsystools.com/products/sitemap-generator/)
Title: Re: Sitemap generator feature suggestions
Post by: hoseinbak on June 25, 2020, 01:57:35 AM
Why doesn't the download link work? please check it
Title: Re: Sitemap generator feature suggestions
Post by: Webhelpforums on June 26, 2020, 06:42:51 AM
Quote from: hoseinbak on June 25, 2020, 01:57:35 AM
Why doesn't the download link work? please check it

Is it the "Download" button" on the main product page
https://www.microsystools.com/products/sitemap-generator/ (https://www.microsystools.com/products/sitemap-generator/)
that is not working for you?

It seems to work for me... Maybe you have something installed that blocks download links? Could you try again?

Title: Re: Sitemap generator feature suggestions
Post by: Webhelpforums on September 30, 2020, 10:52:38 AM
Been suggested to add SFTP support. For now only FTP and FTPS is supported. See also: http://www.microsystools.com/products/sitemap-generator/help/sitemap-generator-ftp-upload/ (http://www.microsystools.com/products/sitemap-generator/help/sitemap-generator-ftp-upload/)
Title: Re: Sitemap generator feature suggestions
Post by: swicon engineers on February 11, 2022, 07:07:26 AM
Would a future release have the ability to create bulk crawls?
Title: Re: Sitemap generator feature suggestions
Post by: Webhelpforums on February 11, 2022, 07:09:10 AM
You can already set these up yourself:
http://www.microsystools.com/products/sitemap-generator/help/automate-sitemap-build-upload/ (http://www.microsystools.com/products/sitemap-generator/help/automate-sitemap-build-upload/)

So yes, you can automate and execute whatever number of crawls for whatever number of websites :)

(Let me know if there is anything you need of features or questions.)
Title: Re: Sitemap generator feature suggestions
Post by: MIL on March 29, 2023, 08:00:50 AM
Hi

Thank you for the wonderful program.

But, it seems to me that it does not take into account robots.txt the rule */f/*,*, which does not allow search robots to scan more than one intersection of filters - for example, in the online store in the category of dresses there are filters - red, long, evening, but the robot can enter only single filters in the search, but it it is forbidden to scan filters red + long, red + evening, red +long +evening and so on.

Please add recognition and following this rule  in robots.txt to the program,

Thanks in advance,
MIL
Title: Re: Sitemap generator feature suggestions
Post by: MIL on March 29, 2023, 11:02:13 AM
In robots.txt There are two rules:

Disallow: */f/*/*/*
Disallow: */f/*,*

I would like the program to take them into account when reindexing the site.

Thank you in advance
Title: Re: Sitemap generator feature suggestions
Post by: Webhelpforums on April 04, 2023, 07:22:25 AM
For reference: The robots.txt help:
http://www.microsystools.com/products/sitemap-generator/help/sitemap-robots-noindex-nofollow/

If I recall correctly, A1 internally rewrites to regular expression so
*/f/*/*/* becomes something like .*?/f/.*?/.*?/.*?
I will check it out :)

Until then, you can always control the A1 crawler using

analysis filters
http://www.microsystools.com/products/sitemap-generator/help/website-crawler-scanner-filters/

output filters
http://www.microsystools.com/products/sitemap-generator/help/website-crawler-output-filters/