Create robots.txt Online for Free | SEO Tool

https://

#robots.txt generated by howblogstuffworks.blogspot.com
User-agent: Mediapartners-Google
Disallow: 
User-agent: Googlebot
Disallow: 
User-agent: Googlebot-Image
Disallow: 
User-agent: Googlebot-Mobile
Disallow: 
User-agent: Googlebot-News
Disallow: 
User-agent: Googlebot-Video
Disallow:
User-agent: Adsbot-Google
Disallow: 
User-agent: bingbot
Disallow: 
User-agent: Twitterbot
Disallow: 
User-agent: Applebot
Disallow: 
User-agent: Bingbot
Disallow: 
User-agent: EchoboxBot
Disallow: 
User-agent: Facebot
Disallow: 
User-agent: SiteAuditBot
Disallow: 
User-agent: Screaming Frog SEO Spider
Disallow: 
User-agent: grapeshot
Disallow: 
User-agent: ias_crawler
Disallow: 
User-agent: Publication-Access-for-Facebook
Disallow: 
User-agent: proximic
Disallow: 
User-agent: facebookexternalhit
Disallow: 
User-agent: Flipboard
Disallow: 
User-agent: FlipboardProxy
Disallow: 
User-agent: weborama-fetcher
Disallow: 
User-agent: Taboolabot
Disallow: 
User-agent: upday
Disallow: 
User-agent: MSNBot
Disallow: 
User-agent: Slurp
Disallow: 
User-agent: Teoma
Disallow: 
User-agent: Gigabot
Disallow: 
User-agent: Robozilla
Disallow: 
User-agent: Nutch
Disallow: 
User-agent: ia_archiver
Disallow: 
User-agent: baiduspider
Disallow: 
User-agent: naverbot
Disallow: 
User-agent: yeti
Disallow: 
User-agent: yahoo-mmcrawler
Disallow: 
User-agent: psbot
Disallow: 
User-agent: yahoo-blogs/v3.9
Disallow: 
User-agent: Yahoo Pipes 1.0
Disallow: 
User-agent: 008
Disallow: 
User-agent: voltron
Disallow: 
User-agent: Bytespider
Disallow: 
User-agent: *
Disallow: /search
Allow: /

Copied

Views: