Robot.txt - pretty much empty after 'creation'

  • 8 Replies
  • 1233 Views
*

cwallace

  • Newbie
  • *
  • 10
  • +0/-0
    • View Profile
Robot.txt - pretty much empty after 'creation'
« on: January 06, 2019, 11:47:04 AM »
When I went to 'create robot.txt' it created a file that only has a sinlge line for sitemap entries and it refers to a 'video' format one...which I don't even have video on my site so why it would take that one and not the base index I don't know...

This is what I get:

User-agent: *

Sitemap: http://<domain name>.com/sitemap-video-OSH.xml


I have 15 or more sitemap files...I would expect to see a bigger list or as I have found with some research a main 'index' sitemap to include only...

Anyway...I would like to create a nice valid robot.txt file, but I keep getting nothing for the most part.

Thanks!!

*

cwallace

  • Newbie
  • *
  • 10
  • +0/-0
    • View Profile
Re: Robot.txt - pretty much empty after 'creation'
« Reply #1 on: January 06, 2019, 01:07:09 PM »
I also noticed several created sitemaps are HUGE in content...one has nearly 500k entries and google is kicking it back telling me to break it down to 50k per file...

However, I am not sure that google interprets those sitemaps and are even valid in the sitemaps for google...

Can you point me to where I might sort out WHAT files should be uploaded and entered into the google search concole sitemap submitter??

Does that make sense??

I do 'create all' and it creates 35 sitemap files...I upload them and then submit to google.

A lot of them have errors...should I not be using them or are these errors in my configuration or setup before sending?

Things like missing XML tag (i.e. there was no URL tag as there was no content in a specific 'video' sitemap). Should I just delete it?

This is still one the best software purchases I have ever made in regard to SEO!!

Thanks in advance...
Chris

*

Webhelpforums

  • Administrator
  • Hero Member
  • *****
  • 1408
  • +6/-0
  • Shared between Microsys, WebHelpForums and helpers
    • View Profile
    • Webmaster and Website Help Forums
Re: Robot.txt - pretty much empty after 'creation'
« Reply #2 on: January 13, 2019, 08:32:08 AM »
Hi,

Sorry for the delay (email support was still within 24 hours though, but the forum took an unintended backseat for a while)

There are many of the sitemap file types you should not submit to Google, e.g. HTML sitemaps but there are quite a few more that are not meant to be submitted to Google (as they used for entirely different purposes)

if you simply submit all generated then you will get lots of errors and warnings about XML etc. :)



Concerning sitemaps differences see this:
https://www.microsystools.com/products/sitemap-generator/help/website-sitemap-kinds-comparison/

But if you do not have video or image content you should simply generate and upload a standard XML sitemap.

I suggest you do not use "Build all", but instead select XML sitemap in the dropdown of choices and click "Build selected". (And maybe clear the output directory first.) That way you avoid uploading old/wrong files to Google Webmaster Tools.

If your website is large, A1 Sitemap Generator will default to split it across multiple files including creating a sitemap index file:
https://www.microsystools.com/products/sitemap-generator/help/xml-sitemaps-page-limit/

From your robots.txt question I suspect you may have configured output paths wrongly (but I may be wrong - I will need to see the project file) - for reference here is the online help:
https://www.microsystools.com/products/sitemap-generator/help/sitemap-robots-disallow-urls/



If I can see your project file, I will have a much better chance of helping, so please do this:

1) Make sure you are using a recent version of the software
2) Email info@microsystools.com your project file ("File | Save As... Which generates a ".ini" file you can send)
3) Report examples of errors in Google Webmaster Tools if they are still there (Note: sometimes resubmitting and waiting a day helps) - the more specific the better
« Last Edit: January 13, 2019, 09:00:16 AM by Webhelpforums »
TechSEO360 | MicrosysTools.com  | A1 Sitemap Generator, A1 Website Analyzer etc.

*

cwallace

  • Newbie
  • *
  • 10
  • +0/-0
    • View Profile
Re: Robot.txt - pretty much empty after 'creation'
« Reply #3 on: May 13, 2019, 01:18:55 PM »
Hello. I have been absent a while, but have sent an e-mail in regard to this post. I am sure something might trigger activity again. :)

I am obviously in no rush. Still using the software as I have...just made some changes and have done some testing and benchmarking stuff.

Thanks!!
Chris

*

Webhelpforums

  • Administrator
  • Hero Member
  • *****
  • 1408
  • +6/-0
  • Shared between Microsys, WebHelpForums and helpers
    • View Profile
    • Webmaster and Website Help Forums
Re: Robot.txt - pretty much empty after 'creation'
« Reply #4 on: May 20, 2019, 05:23:55 AM »
Hi Chris,

We emailed some days ago back and forth - in my last email I asked for further information - did you miss it by any chance?

TechSEO360 | MicrosysTools.com  | A1 Sitemap Generator, A1 Website Analyzer etc.

*

cwallace

  • Newbie
  • *
  • 10
  • +0/-0
    • View Profile
Re: Robot.txt - pretty much empty after 'creation'
« Reply #5 on: May 21, 2019, 02:13:19 PM »
Checking now.

Chris

*

cwallace

  • Newbie
  • *
  • 10
  • +0/-0
    • View Profile
Re: Robot.txt - pretty much empty after 'creation'
« Reply #6 on: May 21, 2019, 02:19:33 PM »
Checked my e-mail and even spam and saw nothing. I am assuming coming from the info@micr...

Chris

*

Webhelpforums

  • Administrator
  • Hero Member
  • *****
  • 1408
  • +6/-0
  • Shared between Microsys, WebHelpForums and helpers
    • View Profile
    • Webmaster and Website Help Forums
Re: Robot.txt - pretty much empty after 'creation'
« Reply #7 on: May 25, 2019, 02:39:27 PM »
See you replied to my emails, so we will continue there :)
TechSEO360 | MicrosysTools.com  | A1 Sitemap Generator, A1 Website Analyzer etc.

*

FannieHoase

  • FannieHoaseSC
  • Newbie
  • *
  • 1
  • +0/-0
    • View Profile
    • Are there any normal guys here?
XEvil 4.0: revolution in automatic CAPTCHA solution
« Reply #8 on: August 23, 2019, 07:56:08 PM »
XEvil 4.0 + RCModule: NEW best program for breaking any Captcha
 

 
1. Powerful Neuronets: XEvil OCR can solve more than 12000 of different types of Captcha's,
including such of popular, like Google CAPTCHA's ReCaptcha v2, v3 (in XEvil 4.0 Ultra only!), Captcha.Com, SolveMedia, Bing-Captcha,
Facebook-captcha and Ucoz-captcha, DLE-captcha, VBulletin-Captcha, and a lot of other types!
 
2. Very High speed and precision: recognition speed 0.01 .. 0.02 second per image (but only 1 second in DEMO version!),
XEvil can bypass and solve wide types of captcha with high precision,
without depending of difficulty, distortion, noises, fonts, colors.
 
Exception is Google ReCaptcha-2: about 20-30 seconds per solution,
on ReCaptcha-2 need to click a lot of images with a special delay.
 
3. Very simple UI: just 3 main buttons to start recognition,
so it easy to use with a wide spectre programs of SEO, SMM, Analytics,
Mass Auto-Registering/Posting/Sending/Bruteforcing/CryptoCurrency Mining programs.
 
Interested? ;)
Just do Search in google "XEvil for Free" .
Free DEMO version is available!
 
Thank you for your attention! :)
 
Website with Free DEMO version:
http://XEvil.net/
(Free DEMO version very restricted in functionality!)
 
Website with full version with XEvil 4.0 Ultra:
http://xrumersale.site/
 
 
 
____
XEvil can break ANY Captcha!... , Google ReCaptcha-2 was broken!, Google ReCaptcha-2 was broken!
 
XRumer 19.0 + XEvil 4.0: массовое размещение на 8 миллионах веб-сайтов! 36e1915

 




See Our Webmaster Tools for Windows and Mac

A1 Sitemap Generator
      
A1 Website Analyzer
      
A1 Keyword Research
      
A1 Website Download
      
A1 Website Search Engine
      
A1 Website Scraper