آموزش ساخت ربات هاست

          قیاسی 
            بازدید : 1898
          1392/08/08
           نظرات (0)
        

یکی از مهمترین یک آدرس اینترنتی داشتن فایل ربات توی هاست هست

ربات سئو وب

حال مسئله این هست چطور فایل ربات بسازیم

برای ساخت یک نوت پد توی سیستم باز کرده با فرمت txt و به نام robots

سپس می تونید از سایتهای سازنده ربات اگه علم نوشتن آن را ندارید استفاده کنید

مانند

http://www.easysoft.ir/services/robotstxt

آموزش ساخت را در پایین ادرس توضیح داده شده است

جهت استاندارد بودن فایل ربات بعد از آپلود از لینک زیر چک کنید

http://www.frobee.com/robots-txt-check

حال رباتهای معروف و دسترسی ها رو میگم و در آینده رباتهای معروف به اسپمر میگم چه طور دسترسی آنها را ببندیم

Robot /Crawler	Robot Purpose
Googlebot	Indexing
Inktomi Slurp	Indexing, Statistics
MSNBot	MSN Search Crawler
AskJeeves	Indexing, Maintenance
Scooter (Alta Vista)	Indexing
ABCdatos BotLink	Maintenance
Acme.Spider	Indexing Maintenance Statistics
Ahoy! The Homepage Finder	Maintenance
Alkaline	Indexing
ananzi	Indexing
Anthill	Indexing
Arachnophilia
Arale	Maintenance
Araneo	Indexing, Statistics
AraybOt	Indexing Maintenance
ArchitextSpider	Indexing, Statistics
Aretha
ARIADNE	Statistics, Development of Focused Crawling Strategies
arks	Indexing
ASpider (Associative Spider)	Indexing
ATN Worldwide	Indexing
Atomz.com Search Robot	Indexing
AURESYS	Indexing,Statistics
BackRub	Indexing, Statistics
BBot	Indexing, Maintenance
Big Brother	Maintenance
Bjaaland	Indexing
BlackWidow	Indexing, Statistics
Bloodhound	Web Site Download
Borg-Bot	Indexing Statistics
BoxSeaBot	Indexing
bright.net caching robot	caching
BSpider	Indexing
CACTVS Chemistry Spider	Indexing
Calif	Indexing
Cassandra	Indexing
Checkbot	Maintenance
ChristCrawler	Indexing
churl	Maintenance
cIeNcIaFiCcIoN.nEt	Indexing
CMC/0.01	Maintenance
Collective
Combine System	Indexing
ComputingSite Robi/1.0	Indexing, Maintenance
ConfuzzledBot	Indexing of luxembourgish sites
CoolBot	Indexing
Cusco	Indexing
CyberSpyder Link Test	link validation, some html validation
CydralSpider	gather Web content for image search engine service
Desert Realm Spider	Indexing
DeWeb(c) Katalog/Index	Indexing, Mirroring, Statistics
Die Blinde Kuh	Indexing
DienstSpider	Indexing
Digger	Indexing
Digimarc MarcSpider	Maintenance
Digital Integrity Robot	WWW Indexing
Direct Hit Grabber	Indexing and Statistics
DNAbot	Indexing
DownLoad Express	graphic download
DragonBot	Indexing
DWCP (Dridus' Web Cataloging Project)	Indexing, Statistics
EbiNess	Statistics
e-collector	email collector
EIT Link Verifier Robot	EIT-Link-Verifier-Robot/0.2
ELFINBOT	Indexing for the Lets Find It Now search Engine
Emacs-w3 Search Engine	Indexing
esculapio	link validation
Esther	Indexing
Evliya Celebi	Indexing turkish content
FastCrawler	Indexing
Felix IDE	Indexing, Statistics
FetchRover	Maintenance, Statistics
fido	Indexing
Fish search	Indexing
Fluid Dynamics Search Engine robot	Indexing
Fouineur	Indexing, Statistics
Freecrawl	Indexing
FunnelWeb	Indexing, statisitics
gammaSpider, FocusedCrawler	Indexing, Maintenance
gazz	Statistics
GCreep	Indexing
GetBot	Maintenance
GetterroboPlus Puu	Indexing
GetURL	Maintenance, Mirroring
Golem	Maintenance
Grapnel/0.01 Experiment	Indexing
Griffon	Indexing
Gromit	Indexing
Gulper Bot	Indexing
Hämähäkki	Indexing
HamBot	Indexing
Harvest	Indexing
havIndex	Indexing
HI (HTML Index) Search	Indexing
HKU WWW Octopus	Indexing
Hometown Spider Pro	Indexing
ht://Dig	Indexing
html_analyzer	maintainance
HTMLgobble	mirror
Hyper-Decontextualizer	Indexing
I, Robot	Indexing
iajaBot	Indexing
IBM_Planetwide	Indexing, Maintenance, Mirroring
image.kapsi.net	Indexing
Imagelock	Maintenance
IncyWincy
Informant	Indexing
InfoSeek Robot	Indexing
Infoseek Sidewinder	Indexing
InfoSpiders	search
Ingrid	Indexing
Inspector Web	Maintenance: link validation, html validation, image size validation, etc
IntelliAgent	Indexing
Internet Cruiser Robot	Indexing
Internet Shinchakubin	Locate New Links and Changed Pages
Iron33	Indexing, Statistics
Israeli-search	Indexing
JavaBee	Stealing Java Code
JBot Java Web Robot	Java web crawler to download web sites
JCrawler	Indexing
JoBo Java Web Robot	downloading, Mirroring, Indexing
Jobot	Indexing
JoeBot
JumpStation	Indexing
Katipo	Maintenance
KDD-Explorer	Indexing
Kilroy	Indexing,Statistics
KIT-Fireball	Indexing
KO_Yappo_Robot	Indexing
LabelGrabber	Grabs PICS labels from web pages, submits them to a label bueau
larbin	General
legs	Indexing
Link Validator	Maintenance
LinkScan	Link checker, SiteMapper, and HTML Validator
LinkWalker	Maintenance, Statistics
Lockon	Indexing
logo.gif Crawler	Indexing
Lycos	Indexing
Mac WWWWorm	Indexing
Magpie	Indexing, Statistics
marvin/infoseek	Indexing
Mattie	Procurement Spider
MediaFox	Indexing and Maintenance
MerzScope	WebMapping
MindCrawler	Indexing
mnoGoSearch	Indexing
moget	Indexing, Statistics
MOMspider	Maintenance, Statistics
Monster	Maintenance, Mirroring
Motor	Indexing
Muncher	Indexing
Muninn	Indexing
Muscat Ferret	Indexing
Mwd.Search	Indexing
NDSpider	Indexing
NEC-MeshExplorer	Indexing
Nederland.zoek	Indexing
NetCarta WebMap Engine	Indexing, Maintenance, Mirroring, Statistics
NetMechanic	Link and HTML validation
NetScoop	Indexing
newscan-online	Indexing
NHSE Web Forager	Indexing
Nomad	Indexing
Northern Light Gulliver	Indexing
nzexplorer	Indexing, Statistics
ObjectsSearch	Indexing
Occam	Indexing
OntoSpider	Statistics
Openfind Data Gatherer	Indexing
Orb Search	Indexing
Pack Rat	Maintenance and Mirroring
PageBoy	Indexing
ParaSite	Indexing
Patric	Statistics
Pegasus	Indexing
PerlCrawler	Indexing
Phantom	Indexing
PhpDig	Indexing
PiltdownMan	Statistics
Pimptrain.com's robot	Indexing
Pioneer	Indexing, Statistics
Poppi	Indexing
Popular Iconoclast	Statistics
Portal Juice Spider	Indexing
PortalB Spider	Indexing
psbot	Indexing
Raven Search	Indexing
RBSE Spider	Indexing, Statistics
Resume Robot	Indexing
RixBot	Indexing
Road Runner: The ImageScape Robot	Indexing
RoadHouse Crawling System	Indexing
Robbie the Robot	Indexing
RoboCrawl Spider	Indexing
RoboFox	Site Download
Robot Francoroute	Indexing, Mirroring, Statistics
Robozilla	Maintenance
Roverbot	Indexing
RuLeS	Indexing
SafetyNet Robot	Indexing
Search.Aus-AU.COM	Indexing
SearchProcess	Statistics
Senrigan	Indexing
SG-Scout	Indexing
ShagSeeker	Indexing
Shai'Hulud	Mirroring
Sift	Indexing
Simmany Robot	Indexing, Maintenance, Statistics
Site Searcher	Indexing
Site Valet	Maintenance
SiteTech-Rover	Indexing
Skymob.com	Indexing
SLCrawler	Sitemap Builder
Sleek	Indexing
Smart Spider	Indexing
Snooper	Indexing
Solbot	Indexing
Speedy Spider	Indexing
spider_monkey	Indexing
SpiderBot	Indexing, Mirroring
Spiderline Crawler	Indexing
SpiderMan	Indexing
SpiderView(tm)	Maintenance
Spry Wizard Robot	Indexing
Suke	Indexing
Suntek search engine	Indexing
Sven	Indexing
Sygol	Indexing
TACH Black Widow	Maintenance: link validation
Tarantula	Indexing
Tarspider	Mirroring
Tcl W3 Robot	Maintenance, Statistics
TechBOT	Statistics, Maintenance
Templeton	Mirroring, Mapping, Automating Web Applications
The Jubii Indexing Robot	Indexing, maintainance
The NorthStar Robot	Indexing
The NWI Robot	Indexing, Statistics
The Peregrinator	Indexing
The Python Robot
The TkWWW Robot	Indexing
The Web Moose	Statistics, Maintenance
TITAN	Indexing
TitIn	Indexing, Statistics
TLSpider	Indexing
UCSD Crawl	Indexing, Statistics
UdmSearch	Indexing, validation
UptimeBot	Indexing, Statistics
URL Check	Maintenance
URL Spider Pro	Indexing
Valkyrie	Indexing
Verticrawl	Indexing, Seartching, Classyfing
Victoria	Maintenance
vision-search	Indexing
void-bot	Indexing, Maintenance
Voyager	Indexing, Maintenance
VWbot	Indexing
w@pSpider by wap4.com	Indexing, Maintenance (dedicated to wap/wml pages)
W3M2	Indexing, Maintenance, Statistics
w3mir	Mirroring
Walhello appie	Indexing
WallPaper / Crawlpaper	Indexing
Web Core / Roots	Indexing, Maintenance
Web Wombat	Indexing, Statistics.
WebBandit Web Spider	Resource Gathering / Server Benchmarking
WebCatcher	Indexing
WebCopy	Mirroring
webfetcher	Mirroring
Webfoot Robot
Webinator	Information Retrieval
weblayers	maintainance
WebLinker	Maintenance
Weblog Monitor	Statistics
WebMirror	Mirroring
WebQuest	Indexing
WebReaper	Indexing/ Offline Browsing
webs	Statistics
Websnarf
WebSpider	Maintenance, Link Diagnostics
WebStolperer	Indexing
WebVac	Mirroring
webwalk	Indexing, Maintenance, Mirroring, Statistics
WebWalker	Maintenance
WebWatch	maintainance, Statistics
Wget	Mirroring, Maintenance
whatUseek Winona	Site-level Search and Meta-Search Engines.
WhoWhere Robot	Indexing
Wild Ferret Web Hopper	Indexing, Maintenance Statistics
World Wide Web Wanderer	Statistics
World Wide Web Worm	Indexing
WWWC	Maintenance
XGET	Mirroring
XYLEME Robot	Indexing

آموزش موتور جستجو

موتور گوگل

موتور بینگ

موتور یاهو

آموزش سئو بهینه سازی

سئو بهینه سازی قالب وبلاگ

سئو بهینه سازی مدیریت محتوا

سئو بهینه سازی سایت

سفارش سئو بهینه سازی

خدمات سئو بهینه سازی سیستم وبلاگ نویسی

خدمات سئو بهینه سازی سیستم وردپرس و جوملا

خدمات سئو بهینه سازی سیستم دات نت

آموزش اینترنت و متفرقه

آموزش وب مستر سایت

آموزش تگ های سئو

آموزش ساخت ربات هاست