Tagged: 

Viewing 5 posts - 1 through 5 (of 5 total)
  • Author
    Posts
  • #435135

    Hey,
    After reading a lot of posts on this support page, I haven’t founded an answer about how to optimize a robot.txt file with Enfold. Generally, such a robot.txt file is structured like this (based on different articles founded on the web) :
    User-agent: *
    Disallow: /*?
    Disallow: /wp-login.php
    Disallow: */trackback
    Disallow: /*/feed
    Disallow: /*/comments
    Disallow: /cgi-bin
    Disallow: /*.php$
    Disallow: /*.inc$
    Disallow: /*.gz
    Disallow: /*.cgi
    Allow: /*css?*
    Allow: /*js?*

    I think it could be nice for Enfold. But naturally, not sure. Especially, it seems that it forbidden to recognize the website as responsive.
    Perhaps do you have some good advices on this.
    Thanks for help and have a nice day.

    #435143

    Sometime, we can find such a file : What do you think about it to optimize Enfold ?
    User-agent: *
    # On empêche l’indexation des dossiers sensibles
    Disallow: /cgi-bin
    Disallow: /wp-login.php
    Disallow: /wp-admin
    Disallow: /wp-includes
    Disallow: /wp-content/plugins
    Disallow: /wp-content/cache
    Disallow: /wp-content/themes
    Disallow: /category/*/*
    Disallow: */trackback
    Disallow: */feed
    Disallow: */comments
    Disallow: /*?
    # On empêche l’indexation des fichiers sensibles
    User-agent: Googlebot
    Disallow: /*.php$
    Disallow: /*.js$
    Disallow: /*.inc$
    Disallow: /*.css$
    Disallow: /*.gz$
    Disallow: /*.swf$
    Disallow: /*.wmv$
    Disallow: /*.cgi$
    Disallow: /*.xhtml$
    # Autoriser Google Image
    User-agent: Googlebot-Image
    Disallow:
    Allow: /*
    # Autoriser Google AdSense
    User-agent: Mediapartners-Google*
    Disallow:
    Allow: /*
    # On indique au spider le lien vers notre sitemap
    Sitemap: http://www.website.com/sitemap.xml

    #436169

    Hi!

    What are you trying to accomplish with your robots.txt file? If your wanting to optimize your site then the best way would be to install a caching plugin. A robots.txt is for blocking access to certain areas.

    Best regards,
    Elliott

    • This reply was modified 8 years, 11 months ago by Elliott.
    #436264

    Hi,

    Thanks for your help Elliott. I’ve already a caching plugin. Its just to know witch files to disallow without causing problems. For example, if I use a slider, or a java script, I dont know if I can disallow /.js. I have read a comment on Enfold’s support that explain that it could cause some problems with responsive design too. Perhaps I’ve badly understand.

    Thanks again. Best regards.
    Ogau

    #436890

    Hi!

    Your supposed to use it for preventing the search engine crawlers to index certain areas of your site. Like if you did not want certain pages to display in the search engines or preventing the archives from loading.

    I don’t think it’s going to matter performance wise if you disable the javascript. It might actually hurt your SEO though if the crawler needs the JS to parse your page. I’m not too sure about it. I would try asking on an SEO forum or the google webmaster forums.

    Cheers!
    Elliott

    • This reply was modified 8 years, 11 months ago by Elliott.
Viewing 5 posts - 1 through 5 (of 5 total)
  • You must be logged in to reply to this topic.