Are you over 18 and want to see adult content?
More Annotations
A complete backup of fundamentosdeadministracion20014.blogspot.com
Are you over 18 and want to see adult content?
A complete backup of availablecar.com
Are you over 18 and want to see adult content?
A complete backup of gogofreegames.com
Are you over 18 and want to see adult content?
A complete backup of sekkusu.tumblr.com
Are you over 18 and want to see adult content?
A complete backup of ronswebdesign.com
Are you over 18 and want to see adult content?
Favourite Annotations
A complete backup of wendytraylor.com
Are you over 18 and want to see adult content?
A complete backup of ivandeliosanctus.com.br
Are you over 18 and want to see adult content?
A complete backup of theferventshaker.wordpress.com
Are you over 18 and want to see adult content?
A complete backup of stemsanjuangroup8.wordpress.com
Are you over 18 and want to see adult content?
Text
ARCHIVETEAM
Archive Team is a loose collective of rogue archivists, programmers, writers and loudmouths dedicated to saving our digital heritage. Since 2009 this variant force of nature has caught wind of shutdowns, shutoffs, mergers, and plain old deletions - and done our best to save the history before it's lost forever. IMGSRC.RU - ARCHIVETEAM Imgsrc.ru. Imgsrc.ru is a simple photo sharing website which is especially popular in East Europe and Germany (Alexa rank ~1000) and has around a million registered users with 50 millions claimed uploads. Registered users can upload photos and organise them in simple albums, which can also be GOOGLE+ - ARCHIVETEAMSEE MORE ON WIKI.ARCHIVETEAM.ORG KONGREGATE - ARCHIVETEAM IRC channel. #archiveteam (on EFnet) Project lead. Unknown. Kongregate is a web game hosting and game forum website. With over 5 million forum posts and 1 million games, it remains to this day one of the most popular online game websites in the world. ANGELFIRE - ARCHIVETEAMSEE MORE ON WIKI.ARCHIVETEAM.ORG REGRETSY - ARCHIVETEAM Regretsy was a website dedicated to highlighting bizarre, weird or offbeat items offered for sale on the handmade and vintage marketplace Etsy. During its run from September 2009 to January 2013, the site posted hundreds of items with snarky commentary, which eventually ledto a
ANYHUB - ARCHIVETEAM AnyHub was a fast, free and simple file host that anyone can use. Signup not required, and upload files of up to 10 GiB at a time. Files uploaded will generally beMIXER - ARCHIVETEAM
Microsoft-owned video game streaming service. Mixer was a Microsoft-owned service for video game streaming and had direct streaming capabilities from Xbox. It originated from the acquisition of Beam. Notably, much of the video data on the site was temporary in nature. For example, past streams (VoDs) are kept for 14 days forregular and pro
TINDECK - ARCHIVETEAM Tindeck was a free audio hosting service for Creative Commons-licensed music.. Shutdown notice. Sometime between 2018-06-25 and 2018-07-02, Tindeck announced with a banner on the website that they would shut down on 2018-08-01: . In light of the European Union's insistence on destroying the free and open internet, we regretfully inform you that Tindeck will permanently cease operations as of WAPER.RU - ARCHIVETEAM Waper.ru. waper.ru is a russian website for mobile users with a lot of user-generated content. It features communities, forums, photo and video, blogs, anecdotes and uploads. The WAP-version is available under z.waper.ru. It has been operating since May 2007, the users are still fairly active, yet the administrator stated in 2015 that hedoesn
ARCHIVETEAM
Archive Team is a loose collective of rogue archivists, programmers, writers and loudmouths dedicated to saving our digital heritage. Since 2009 this variant force of nature has caught wind of shutdowns, shutoffs, mergers, and plain old deletions - and done our best to save the history before it's lost forever. IMGSRC.RU - ARCHIVETEAM Imgsrc.ru. Imgsrc.ru is a simple photo sharing website which is especially popular in East Europe and Germany (Alexa rank ~1000) and has around a million registered users with 50 millions claimed uploads. Registered users can upload photos and organise them in simple albums, which can also be GOOGLE+ - ARCHIVETEAMSEE MORE ON WIKI.ARCHIVETEAM.ORG KONGREGATE - ARCHIVETEAM IRC channel. #archiveteam (on EFnet) Project lead. Unknown. Kongregate is a web game hosting and game forum website. With over 5 million forum posts and 1 million games, it remains to this day one of the most popular online game websites in the world. ANGELFIRE - ARCHIVETEAMSEE MORE ON WIKI.ARCHIVETEAM.ORG REGRETSY - ARCHIVETEAM Regretsy was a website dedicated to highlighting bizarre, weird or offbeat items offered for sale on the handmade and vintage marketplace Etsy. During its run from September 2009 to January 2013, the site posted hundreds of items with snarky commentary, which eventually ledto a
ANYHUB - ARCHIVETEAM AnyHub was a fast, free and simple file host that anyone can use. Signup not required, and upload files of up to 10 GiB at a time. Files uploaded will generally beMIXER - ARCHIVETEAM
Microsoft-owned video game streaming service. Mixer was a Microsoft-owned service for video game streaming and had direct streaming capabilities from Xbox. It originated from the acquisition of Beam. Notably, much of the video data on the site was temporary in nature. For example, past streams (VoDs) are kept for 14 days forregular and pro
TINDECK - ARCHIVETEAM Tindeck was a free audio hosting service for Creative Commons-licensed music.. Shutdown notice. Sometime between 2018-06-25 and 2018-07-02, Tindeck announced with a banner on the website that they would shut down on 2018-08-01: . In light of the European Union's insistence on destroying the free and open internet, we regretfully inform you that Tindeck will permanently cease operations as of WAPER.RU - ARCHIVETEAM Waper.ru. waper.ru is a russian website for mobile users with a lot of user-generated content. It features communities, forums, photo and video, blogs, anecdotes and uploads. The WAP-version is available under z.waper.ru. It has been operating since May 2007, the users are still fairly active, yet the administrator stated in 2015 that hedoesn
GOOGLE+ - ARCHIVETEAM Archiving data from Google+ is over. Hundreds of volunteers took part in ArchiveTeam's biggest action to date that resulted in over 1.56 PB (petabytes) – that's 1,560,000,000,000,000 bytes – of WARCs in less than four weeks. This, as usual, is hosted by the InternetArchive.
ANGELFIRE - ARCHIVETEAM Angelfire is a web hosting service since 1996, containing big chunks of early WWW history (which people love to mock at).. It is not expected that the Angelfire archive can ever be truly complete, as Angelfire, like other free hosts such as Homestead, has or had a policy of deleting "inactive" accounts.ENJIN - ARCHIVETEAM
Enjin is a community hosting platform, targeted in particular at gamers. It hosts communities with forums, blogs, game integration, wiki, ticket system, and numerous other 'modules'. Site status. Starting in early 2021, there have been various signs of instability, and the platform is deteriorating quickly: some modules started breaking, the mobile apps were pulled from app stores without EDITTHIS - ARCHIVETEAM EditThis is a wikifarm.According to our estimates, there are over 1,300 wikis.There is a 2014 backup for most of them.. This farm is quite hard to archive, because of old software (MediaWiki 1.15) with several weirdnesses, both at application and webserver level (like directory structure, URL rewrites, l10n in MediaWiki namespace); EVERYPLAY - ARCHIVETEAM Everyplay was a video hosting service used in many games such as Geometry Dash as a replay system. It shut down October 1st, 2018, due to "inactivity", though it was most likely due to the constantly-expressed toxicity of the platform. It had 38 millionvideos.
EMULARITY - ARCHIVETEAM The Emularity is both the name of a specific set of scripts meant to load emulators, and the entire ongoing project to shove Javascript-ported emulators into browsers. Now years in the making, the project is mature and in use by millions of people worldwide. This page is meant to be a one-stop stop for keeping track of what needs tobe worked
FORMSPRING - ARCHIVETEAM Formspring, later known as spring.me, was an anonymous question-and-answer site.In 2013, it was the subject of Archive Team'srecovery operation.
YAHOO! - ARCHIVETEAM One of the ArchiveBot commands, !yahoo, has been named after Yahoo!. This command makes the bot archive the page in a more aggressive manner to speed up the archival process. The name was insipred by Yahoo! being either lenient about or incapable of restricting such aggressive retrieval.PHOTOBUCKET
This makes it possible to use wget to grab all the files yourself, or grab-site to archive with WARC. Usage: Obtain all the urls of a Photobucket Album, even subalbums (using the -r parameter), and put them in links-.txt. This url file can be given to wget or grab-site todownload.
REDDIT - ARCHIVETEAM Reddit is a content aggregator and social bookmarking service similar to the likes of Digg. Users can submit links, text posts, images and videos, vote and comment on submissions in communities called "subreddits". It received considerable attention from its twelve-hour SOPA blackout early in January 2012.ARCHIVETEAM
Archive Team is a loose collective of rogue archivists, programmers, writers and loudmouths dedicated to saving our digital heritage. Since 2009 this variant force of nature has caught wind of shutdowns, shutoffs, mergers, and plain old deletions - and done our best to save the history before it's lost forever. IMGSRC.RU - ARCHIVETEAM Imgsrc.ru. Imgsrc.ru is a simple photo sharing website which is especially popular in East Europe and Germany (Alexa rank ~1000) and has around a million registered users with 50 millions claimed uploads. Registered users can upload photos and organise them in simple albums, which can also be KONGREGATE - ARCHIVETEAM IRC channel. #archiveteam (on EFnet) Project lead. Unknown. Kongregate is a web game hosting and game forum website. With over 5 million forum posts and 1 million games, it remains to this day one of the most popular online game websites in the world. ANYHUB - ARCHIVETEAM AnyHub was a fast, free and simple file host that anyone can use. Signup not required, and upload files of up to 10 GiB at a time. Files uploaded will generally beBBC MIXITAL
MIRAHEZE - ARCHIVETEAM Miraheze makes their own backups of their services regularly to an offsite server (provided by Backupsy). In April 2019, Miraheze launched and deployed the DataDump extension on all its wikis, allowing wiki operators to generate and download dumps through the "Special:DataDump" page. By default, only admins possess the "view-dump", "delete-dump SOUP.IO - ARCHIVETEAM Soup.io was a social networking and microblogging site from Vienna that shut down on 20 July 2020.. Shutdown notice. The following notice was posted on the official Soup on 2020-07-10: . The sadest news in the soup history :(Dear soup.io fans and users, TINDECK - ARCHIVETEAM Tindeck was a free audio hosting service for Creative Commons-licensed music.. Shutdown notice. Sometime between 2018-06-25 and 2018-07-02, Tindeck announced with a banner on the website that they would shut down on 2018-08-01: . In light of the European Union's insistence on destroying the free and open internet, we regretfully inform you that Tindeck will permanently cease operations as ofCNET FORUMS
CNET Forums were part of the popular CNET online magazine. According to its own numbers, there were over 530k forum topics. Shutdown notice. Thank you for being a valued part of the CNET community.YAHOO! BRIEFCASE
Yahoo Briefcase was a service that allowed people using Yahoo to store up to 25 megabytes of data, for "free". As the 25mb (not gigabytes, and not 250mb, but twenty-five megabytes) indicates, this service stored a very large amount of data, therefore requiring Yahoo to reduce the amount of data allowed stored by account.Why Yahoo suddenly decided to stop supporting this service is unknown, butARCHIVETEAM
Archive Team is a loose collective of rogue archivists, programmers, writers and loudmouths dedicated to saving our digital heritage. Since 2009 this variant force of nature has caught wind of shutdowns, shutoffs, mergers, and plain old deletions - and done our best to save the history before it's lost forever. IMGSRC.RU - ARCHIVETEAM Imgsrc.ru. Imgsrc.ru is a simple photo sharing website which is especially popular in East Europe and Germany (Alexa rank ~1000) and has around a million registered users with 50 millions claimed uploads. Registered users can upload photos and organise them in simple albums, which can also be KONGREGATE - ARCHIVETEAM IRC channel. #archiveteam (on EFnet) Project lead. Unknown. Kongregate is a web game hosting and game forum website. With over 5 million forum posts and 1 million games, it remains to this day one of the most popular online game websites in the world. ANYHUB - ARCHIVETEAM AnyHub was a fast, free and simple file host that anyone can use. Signup not required, and upload files of up to 10 GiB at a time. Files uploaded will generally beBBC MIXITAL
MIRAHEZE - ARCHIVETEAM Miraheze makes their own backups of their services regularly to an offsite server (provided by Backupsy). In April 2019, Miraheze launched and deployed the DataDump extension on all its wikis, allowing wiki operators to generate and download dumps through the "Special:DataDump" page. By default, only admins possess the "view-dump", "delete-dump SOUP.IO - ARCHIVETEAM Soup.io was a social networking and microblogging site from Vienna that shut down on 20 July 2020.. Shutdown notice. The following notice was posted on the official Soup on 2020-07-10: . The sadest news in the soup history :(Dear soup.io fans and users, TINDECK - ARCHIVETEAM Tindeck was a free audio hosting service for Creative Commons-licensed music.. Shutdown notice. Sometime between 2018-06-25 and 2018-07-02, Tindeck announced with a banner on the website that they would shut down on 2018-08-01: . In light of the European Union's insistence on destroying the free and open internet, we regretfully inform you that Tindeck will permanently cease operations as ofCNET FORUMS
CNET Forums were part of the popular CNET online magazine. According to its own numbers, there were over 530k forum topics. Shutdown notice. Thank you for being a valued part of the CNET community.YAHOO! BRIEFCASE
Yahoo Briefcase was a service that allowed people using Yahoo to store up to 25 megabytes of data, for "free". As the 25mb (not gigabytes, and not 250mb, but twenty-five megabytes) indicates, this service stored a very large amount of data, therefore requiring Yahoo to reduce the amount of data allowed stored by account.Why Yahoo suddenly decided to stop supporting this service is unknown, butENJIN - ARCHIVETEAM
Enjin is a community hosting platform, targeted in particular at gamers. It hosts communities with forums, blogs, game integration, wiki, ticket system, and numerous other 'modules'. Site status. Starting in early 2021, there have been various signs of instability, and the platform is deteriorating quickly: some modules started breaking, the mobile apps were pulled from app stores withoutBETAARCHIVE
FTP Server. The core of the whole site. To access it, you need to be in the FTP Access Group on BetaArchive's forums (guidelines are here).Some of what's on the FTP can be found elsewhere, but there's a lot of unique content on the FTP that could get lost if BetaArchive ever permanently went down. MEDIUM - ARCHIVETEAM Medium.com is a blogging website for people who'd like a Wordpress.com with more whitespace, less features, more barriers to access, more surveillance and more gambling their future.. As a VC-backed startup with no profitability in sight, it's likely to disappear suddenly and therefore it merits a permanent Deathwatch.. TimelineXFIRE - ARCHIVETEAM
Xfire was a proprietary freeware instant messaging service for gamers that also served as a game server browser with various other features.. social.xfire.com was a community site for Xfire users, allowing them to upload screenshots (photos and videos) and to make contacts.. On June 10, 2015 Xfire announced that they were shutting down the Xfire client and purge all user content on socialFURAFFINITY
FurAffinity is an art gallery website, and the largest web community of the furry fandom.It was recently acquired by IMVU (see the recent journals of the site's mascot account), and while the site and community have endured multiple prior divisive events, the prior owner's continued control of the site's maintenance is now uncertain due to the acquisition, and the site's future is similarlyMIXER - ARCHIVETEAM
Microsoft-owned video game streaming service. Mixer was a Microsoft-owned service for video game streaming and had direct streaming capabilities from Xbox. It originated from the acquisition of Beam. Notably, much of the video data on the site was temporary in nature. For example, past streams (VoDs) are kept for 14 days forregular and pro
LIBRARY GENESIS
IRC channel. #archiveteam (on EFnet) Project lead. Unknown. Library Genesis is a Russian project to create a free online library of ebooks. They currently have about 4 million of them. The project is in blatant violation of copyright, and their domain name is blocked insome countries.
XANGA - ARCHIVETEAM
Xanga is a blogging community which is going to be relaunched as a paid service. The users will be able to download archives of their current blogs until July the 31st, after which the site will either shut down due to fundraising failure, or get relaunched as a WordPress-powered version.DRUNKENDONKEY
DrunkenDonkey. DrunkenDonkey or DD was/is an Italian eMule community. The forum is used to share tips and ed2k links. It went through various shutdowns and migrations under various names: DDUniverse, DDUnlimited, perhaps also DigitalReload. The forum uses phpBB and has at times preserved the user database across migrations. ZIPPCAST - ARCHIVETEAM zippcast. IRC channel. #archiveteam (on EFnet) Project lead. Unknown. ZippCast was a video sharing service, started in 2009. On May 31, 2016 Louis Gualtieri (owner) decided to shut the site down on June 5, 2016, due to financial and personal reasons. The service was interrupted onJune 10, 2016.
ARCHIVEBOT
From Archiveteam
Jump to navigation Jump to search Imagine Motoko Kusanagi as anarchivist.
ARCHIVEBOT is an IRC bot designed to automate the archival of smaller websites (e.g. up to a few hundred thousand URLs). You give it a URL to start at, and it grabs all content under that URL, records it in a WARC file, and then uploads that WARC to ArchiveTeam servers for eventual injection into the Internet Archive 's Wayback Machine (or other archive sites).CONTENTS
* 1 Details
* 2 Components
* 3 Source Code
* 4 People
* 5 Volunteer to run a Pipeline* 5.1 Caveats
* 6 Installation
* 7 Disclaimers
* 8 Bad behavior
* 9 Trivia
* 10 Usage, Dashboards, and Completed Job Viewer * 11 Usage Caveats a.k.a Things Not To Do With ArchiveBot a.k.a HowTo Lose Your Voice
* 12 Suggested things to focus on archiving with ArchiveBot* 13 Related Links
* 14 More
* 15 Notes
DETAILS
To use ArchiveBot, drop by the IRC channel #ARCHIVEBOT on hackint(webchat
). To
interact with ArchiveBot, you issue COMMANDSby typing
them into the channel. Note that you will need channel operator (@) or voice (+) permissions in order to issue archiving jobs; please ask for assistance or leave a message describing the website you want toarchive.
The ARCHIVEBOT DASHBOARD publicly shows the sites being currently downloaded. The pipelinemonitor station
shows the status of deployed instances of crawlers. The viewerassists in browsing
and searching archives.COMPONENTS
IRC interface
The bot listens for commands in the IRC channel and then reports back status on the IRC channel. You can ask it to archive a whole website or single webpage, check whether the URL has been saved, change the delay time between requests, or add some ignore rules to avoid crawling certain web cruft. This IRC interface is collaborative, meaning anyone with permission can adjust the parameter of jobs. Note that the bot isn't a chat bot so it will ignore you if it doesn't understand a command.Dashboard
The ARCHIVEBOT DASHBOARD is a web-based front-end displaying the URLs being downloaded by the various web crawls. Each URL line in the dashboard is categorized by its HTTP code into successes, warnings, and errors. It will be highlighted in yellow or red. The dashboard also provides RSS feeds and a list of pending jobs.
Backend
The backend contains the database of all jobs and several maintenance tasks such as trimming logs and posting Tweets on Twitter. The backend is the centralized portion of ArchiveBot.Crawler
The crawler will download and spider the website into WARC files. The crawler is the distributed portion of ArchiveBot. Volunteers run pipeline nodes connected to the backend. The backend will tell the nodes/pipelines what jobs to run. Once the crawl job has finished, the pipeline reports back to the backend and uploads the WARC files to the staging server. This process is handled by a supervisor script calleda pipeline.
Staging server
The staging server, known as Fortress of Solitude (FOS) , is the place where all the WARC files are temporarily uploaded. Once the current batch has been approved, the files will be uploaded to the Internet Archive for consumption by the Wayback Machine.SOURCE CODE
ArchiveBot's source code can be found at https://github.com/ArchiveTeam/ArchiveBot. Contributions welcomed ! Any issues or feature requests may be filed at theissue tracker .
PEOPLE
The main server that controls the IRC bot, pipeline manager backend, and web dashboard is operated by yipdw , although a few other ArchiveTeam members were given SSH access in late 2017. The staging server Fortress of Solitude (FOS) , where the data sits for final checks before being moved over to the Internet Archive serves, is operated by SketchCow . The pipelines are operated by various volunteers around the world. Each pipeline typically runs two or three web crawl jobs at any given time. VOLUNTEER TO RUN A PIPELINE As of November 2017, ArchiveBot has again started accepting applications from volunteers who want to set up new pipelines. You'll need to have a machine with: * lots of disk space (40 GB minimum / 200 GB recommended / 500 GBatypical)
* 512 MB RAM (2 GB recommended, 2 GB swap recommended) * 10 Mb/s upload/download speeds (100 Mb/s recommended) * long-term availability (2 months minimum) * always-on unrestricted internet access (absolutely no firewall/proxies/censorship/ISP-injected-ads/DNS-redirection/free-cafe-wifi) Suggestion: the $40/month Digital Ocean droplets (4 GB memory/2 CPU/60 GB hard drive) running Ubuntu work pretty well. Note that we currently only accept pipelines from people who have been active on ArchiveTeam for a while. If you have a suitable server available and would like to volunteer, please review the Pipeline Install instructions. Then contact ArchiveTeam members Asparagirl, astrid
, JAA
, yipdw ,
or other ArchiveTeam members hanging out in #archivebot, and we can hook you up, adding your machine to the list of approved pipelines, so that it will start processing incoming ArchiveBot jobs.CAVEATS
As of August 2018, there are a few things you need to be aware of when operating an ArchiveBot pipeline: * NEVER, EVER PRESS ^C ON THE PIPELINE. Use touch STOP in the ArchiveBot/pipeline directory instead to stop the pipeline. * Please give access to the pipeline for maintenance work when you're away (e.g. holidays, busy IRL) to someone who's around frequently. This is to avoid situations where jobs or pipelines are stuck for weeks or months without anyone being able to intervene. * Jobs that crash with an error need to be killed manually usingkill -9.
* The log files of jobs that are aborted or crash are not uploaded to the Internet Archive. Please keep the temporary tmp-wpull-*.log.gz files in the pipeline directory, rename them so the filename follows the same format as the JSON file (with extension .log.gz instead of .json), and upload them to FOS manually. * You can find the job ID for these files in the second line. * Finding the correct filename can be a bit tricky. You can use the viewer or the archivebot-archives repository. Keep in mind that the timestamp in the filename should approximately match the one at the beginning of the log file, though there is usually a difference between the two of at least a few seconds (the log file timestamps being later than the filenametimestamp).
* Be careful with the filename if there were multiple jobs for the same URL (i.e. the same job ID). * Here is a public gist on GitHub explaining step by step how to find the proper log file for your crashed or killed job, how to properly rename it, and how to rsync it up to FOS: * Contact User:JustAnotherArchivist if you need help with this. * Due to a bug somewhere deep in the network stack, connections get stuck from time to time. This causes jobs to slow down or haltentirely.
* As a workaround, you can use the kill-wpull-connections script; it requires pgrep, lsof, and gdb. Depending on the machine configuration (specifically, the value of kernel.yama.ptrace_scope in /proc/sys/kernel/yama/ptrace_scope), it may also require root/sudoprivileges.
* In very rare cases, you may need to use killcx to close the connections. * tcp_closer works even when the two methods above fail. It uses the SOCK_DESTROY kernel operation provided by Linux >= 4.5. * Also due to a bug suspected to be in the network stack, wpull processes sometimes use a lot of RAM (and CPU). If a process uses more than 300 MB continuously, that's likely the case. kill-wpull-connections seems to "fix" this issue, though it takes a while (minutes, rarely even an hour or more) from running the script until the usage actually drops down. * If wpull paused due to high RAM usage try creating a swap file and forcing RAM pages to swap. wpull only checks RAM usage. dd if=/dev/zero of=swapfile bs=1024 count=1024000mkswap swapfile
swapon swapfile
perl -e '$tmp = "a" x 999999999'swapoff swapfile
rm swapfile
* Make sure that you don't have any search or domain line in /etc/resolv.conf. We've grabbed a number of copies of the websites of OVH and Online.net as a result of such lines and broken http://www/ links... (Cf this issue on GitHub)
INSTALLATION
Installing the ArchiveBot can be difficult. The Pipeline Install instructions are online, but are tricky. But there is a Travis.yml automated install script for Travis-cl that is designed to test the ArchiveBot. Since it's good enough for testing... it's good enough for installation, right? There must be a way to convert it into aninstaller script.
DISCLAIMERS
* Everything is provided on a best-effort basis; nothing is guaranteed to work. (We're volunteers, not a support team.) * We can decide to stop a job or ban a user if a job is deemed unnecessary. (We don't want to run up operator bandwidth bills and waste Internet Archive donations on costs.) * We're not Internet Archive. (We do what we want.) * We're not the Wayback Machine. Specifically, we are not ia_archiver or archive.org_bot. (We don't run crawlers on behalf ofother crawlers.)
Occasionally, we had to ban blocks of IP addresses from the channel. If you think a ban does not apply to you but cannot join the #archivebot channel, please join the main #archiveteam channelinstead.
BAD BEHAVIOR
If you are a website operator and you notice ArchiveBot misbehaving, please contact us on #archivebot or #archiveteam on hackint (see topof page for links).
ArchiveBot understands robots.txt (please read the article) but does not match any directives. It uses it for discovering more links such as sitemaps however. Also, please remember that WE ARE NOT THE INTERNET ARCHIVE.
TRIVIA
* One of the ArchiveBot commands, !yahoo, has been named after Yahoo! . This command makes the bot archive the page in a more aggressive manner to speed up the archival process. USAGE, DASHBOARDS, AND COMPLETED JOB VIEWERFUNCTION
URL
ArchiveBot documentation, usage guide, manual http://archivebot.rtfd.io/ ArchiveBot traditional dashboard - shows currently active jobs http://dashboard.at.ninjawedding.org or http://archivebot.com/ ArchiveBot newer-style dashboard - shows currently active jobs http://dashboard.at.ninjawedding.org/3 or http://archivebot.com/3 ArchiveBot dashboard - shows pending jobs http://dashboard.at.ninjawedding.org/pending or http://archivebot.com/pending ArchiveBot dashboard - shows ignores for a specific job http://dashboard.at.ninjawedding.org/ignores/YOUR VOICE
Do not do these things (unless otherwise approved by #archivebotchannel ops):
* attempt to archive file storage mirrors, whether on HTTP or FTP (original file storage sites might be okay...but ask first. Mirroring every Linux ISO mirror site is not a good use of archive resources.) * attempt to archive very large sites (this may include forums) * attempt to archive anything a channel op has told you not toarchive
* continue to archive stuff after a channel op has asked you a question and has not cleared you to continue SUGGESTED THINGS TO FOCUS ON ARCHIVING WITH ARCHIVEBOT * Company acquisitions and mergers - companies that are being, or have been, acquired or merged * ISP Hosting - user homepages on ISPs that have not previously been saved * Web presence/content/social media of people that have newly become notable/famous/infamous/featured in the news media. Renewed focus applies if they make important or outrageous public statements that may later be retracted, and also when they pass away as their online records may change/vanish afterwards.RELATED LINKS
* http://archivebot.com is a DNS alias for the ArchiveBot dashboard * Chromebot is an IRC bot parallel to ArchiveBot that uses Google Chrome and thus is able to archive JavaScript-heavy and pages with endless scrolling. It is available in the #archivebot channel. * https://twitter.com/ArchiveBot - dormant current Twitter feed of ArchiveBot activity. Tweets may lag live dashboard. Dormant since2018-04-12.
* https://twitter.com/ATArchiveBot - former Twitter feed of ArchiveBot activity. Last used 2014-07-28 as @ATArchiveBot was replaced by @ArchiveBot. * https://archive.org/details/archivebot - ArchiveTeam ArchiveBot collection at the Internet ArchiveMORE
Like ArchiveBot? Check out our homepage andother projects !
NOTES
COLLAPSEV  · T
·
E
ARCHIVEBOT
CORPORATIONS
Facebook (people
) · GitHub
(people
) · IBM
(people
) · Microsoft
(people
) · Reddit
· Telegram
· Yahoo
· Oldest companies
GLAM
National Archives (Other ) · National Film Archives(Other
) · National Galleries(Other
) · National Libraries(Other
) · National Museums(Other
)
* _Museums:_ Africa  · Italy·
Poland - _Related:_ Damaged or destroyed cultural institutions·
Knowledge preservation initiativesGOVERNMENTS
Algeria  · Antarctica· Brazil
· Cape Verde
· Greenland
· Iran
· Malta
· Micronesia
· North Korea
· Oman
· Philippines
· Sahrawi Arab
Democratic Republic
·
Spain  · Syria
· Sudan
· Yemen
HISTORY AND CULTURE
Languages  · Memoria HistóricaPEOPLE
Archivists  · Cancer patients· Travelers
· People with physicaldisabilities
POLITICS
Elections  · Venezuela politics· Yellow Vests
- _Related:_ Alternative media(political left
)
SPORTS
World championships in 2019(2019 FIFA Women's
World Cup )
TOPICS
Artificial Intelligence· Astronomy
· Banned stuff
· Futurology
· Micronations
· Rare stuff
WIKIS
WikiLeaks  · Wikis· Wikidata lists
OTHER
Datasheets  · Educationalinstitutions  ·
Internet campaigns
Tutorial: ArchiveBot/Bot documentation& ArchiveBot/Test
EXPANDV  · T
·
E
ARCHIVE
TEAM
CURRENT EVENTS
Alive... OR ARE THEY  · Deathwatch· Projects
ARCHIVING PROJECTS
APKMirror  · Archive.is· BetaArchive  ·
Government Backup (#datarefuge · ftp-gov ) · Gmane· Internet Archive
· It Died  ·
Megalodon.jp  · OldApps.com· OldVersion.com
· OSBetaArchive
· TEXTFILES.COM
· The Dead, the Dying & The Damned· The Mail Archive
· UK Web Archive
· WebCite  ·
Vaporwave.me
BLOGGING
Blog.pl  · Blogger  ·Blogster  ·
Blogter.hu  · Freeblog.hu· Fuelmyblog
· Jux
· LiveJournal  · My Opera · Nolblog.hu  · Open Diary  · ownlog.com· Posterous  ·
Powerblogs  · Proust  · Roon  · Splinder  · Tumblr· Vox
· Weblog.nl
· Windows Live Spaces· Wordpress.com
· Xanga  · Yahoo!
Blog  · Zapd
CLOUD HOSTING /FILE SHARING aDrive  · AnyHub  · Box· Dropbox
· Docstoc
· Fast.io  · Google Drive· Fileplanet
· LayerVault  ·
MediaCrush  · MediaFire · Mega  · MegaUpload· MobileMe  ·
OneDrive  · Pomf.se · RapidShare  · Ubuntu One  · Yahoo! BriefcaseCORPORATIONS
Apple  · IBM
· Loblaw
· Lycos Europe
· Microsoft  ·
Yahoo!
EVENTS
Arab Spring  ·
Great Ape-Snake War
·
Spanish Revolution
FONT REPOS
DaFont  · Google WebFonts  ·
GNU FreeFont  ·
Fontspace
FORUMS/MESSAGE BOARDS 4chan  · Captain Luffy Forums · College Confidential· DSLReports
· ESPN Forums  ·
Facepunch Forums  · forums.starwars.com· HeavenGames
· JamiiForums  ·
Invisionfree  · NeoGAF · Textream  · The Classic Horror Film Board  · Yahoo! Messages  · Yahoo! Neighbors· Yuku.com  ·
Zetaboards
GAMING
Atomicgamer  · Bazaar.tf· City of Heroes
· Club Nintendo
· Clutch  ·
Counter-Strike: Global Offensive·
CS:GO Lounge  · Desura· Dota 2
· Dota 2 Lounge
· Emulation Zone
· ESEA
· GameBanana
· GameMaker
Sandbox  · GameTrailers · Halo  · HLTV.org· HQ Trivia  ·
Infinite Crisis  · joinDOTA· League of
Legends  · Liquipedia· Minecraft.net
· Player.me
· Playfire
· Raptr
· SingStar
· Steam  · SteamDB · SteamGridDB  · TeamFortress 2
· TF2
Outpost  · Warhammer· Xfire
IMAGE HOSTING
500px  · AOL Pictures· Blipfoto  ·
Blingee  · Canv.as  · Camera+  · Cameroid· DailyBooth
· Degree Confluence Project· DeviantART
· Demotivalo.net
· Flickr  ·
Fotoalbum.hu  · Fotolog.com· Fotopedia  ·
Frontback  · Geograph Britain and Ireland· Giphy
· GTF Képhost  ·
ImageShack  · Imgh.us · Imgur  · Inkblazers· Instagram  ·
Kepfeltoltes.hu  · Kephost.com· Kephost.hu  ·
Kepkezelo.com  · Keptarad.hu· Madden GIFERATOR
· MLKSHK  ·
Microsoft Clip Art  · Microsoft Photosynth  · Nokia Memories· noob.hu  ·
Odysee  · Panoramio  · Photobucket  · Picasa· Picplz  · Pixiv
· Portalgraphics.net· PSharing  ·
Ptch  · puu.sh  · Rawporter· Relay.im  ·
ScreenshotsDatabase.com  · Sketch  · Smack Jeeves· Snapjoy  ·
Streetfiles  · Tabblo · Tinypic  · Trovebox · TwitPic  · Wallbase· Wallhaven  ·
Webshots  · Wikimedia CommonsKNOWLEDGE/WIKIS
arXiv  · Citizendium· Clipboard.com
· Deletionpedia
· EditThis  ·
Encyclopedia Dramatica  · Etherpad  · Everything2· infoAnarchy
· GeoNames
· GNUPedia  · Google Books (Google Books Ngram ) · Horror Movie Database· Insurgency Wiki
· Knol  · Lost Media Wiki  · Neoseeker.com· Notepad.cc  ·
Nupedia  · OpenCourseWare· OpenStreetMap
· Orain  · Pastebin · Patch.com  · Project Gutenberg  · Puella Magi· Referata  ·
Resedagboken  · SongMeanings · ShoutWiki  · The Internet Movie Database  · TropicalWikis  · Uncyclopedia· Urban Dictionary
· Urban
Exploration Resource  · Webmonkey  · Wikia  · Wikidot  · WikiHow  · Wikkii  · WikiLeaks  · Wikipedia (Simple English Wikipedia) · Wikispaces
· Wikispot  · Wik.is· Wiki-Site  ·
WikiTravel  · Word Count Journal MAGAZINES/BLOGS/NEWS Cyberpunkreview.com  · Game Developer Magazine  · Gigaom· Hardware Canucks
· Helium  · JPG
Magazine  · Make Magazine· The Escapist
· Polygamia.pl
· San Fransisco Bay Guardian· Scoop
· Regretsy  · Yahoo!Voices
MICROBLOGGING
Heello  · Identi.ca  ·Jaiku  · Mommo.hu
· Plurk  · Sina Weibo· Tencent Weibo
· Twitter  ·
TwitLonger
MUSIC/AUDIO
8tracks  · AOL Music  ·Audimated.com
· Cinch
· digCCmixter  ·
Dogmazic.net  · Earbits · exfm  · Free Music Archive· Gogoyoko
· Indaba Music
· Instacast
· Instaudio  ·
Jamendo  · Last.fm  · Music Unlimited  · MOG· PureVolume
· Reverbnation
·
ShareTheMusic
· SoundCloud
· Soundpedia
· Spotify
· This Is My Jam
· TuneWiki  ·
Twaud.io  · WinAmpPEOPLE
Aaron Swartz  · Michael S. Hart· Steve Jobs
· Mark Pilgrim
· Dennis Ritchie
· Len Sassaman Project PROTOCOLS/INFRASTRUCTURE FTP  · Gopher  · IRC · Usenet  · World Wide WebBitTorrent DHT
Q&A
Askville  ·
Answerbag  ·
Answers.com  · Ask.com· Askalo
· Baidu Knows
· Blurtit
· ChaCha
· Experts Exchange
·
Formspring  · GirlsAskGuys· JustAnswer
· MetaFilter
· Quora
· Retrospring  ·
StackExchange
· The
AnswerBank  ·
The Internet Oracle
· Uclue
· WikiAnswers
· Yahoo!
Answers
RECIPES/FOOD
Allrecipes  ·
Epicurious  ·
Food.com  · Foodily· Food Network
· Punchfork
· ZipList
SOCIAL BOOKMARKING
Addinto  · Backflip· Balatarin
· BibSonomy
· Bkmrx
· Blinklist
· BlogMarks
· BookmarkSync
· CiteULike
· Connotea
· Delicious
· Designer News
· Digg
· Diigo
· Dir.eccion.es
· Evernote
· Excite Bookmark
· Faves
· Favilous
· folkd
· Freelish
· Getboo
· GiveALink.org
· Gnolia
· Google Bookmarks
· Hacker
News  · HeyStaks
· IndianPad
· Kippt
· Knowledge Plaza
· Licorize
· Linkwad
· Menéame
· Microsoft
Developer Network  · myVIP· Mister Wong
· My Web
· Mylink Vault
· Newsvine
· Oneview
· Pearltrees
· Pinboard
· Scloog
· Scuttle
· Simpy
· SiteBar
· Slashdot
· Squidoo
· StumbleUpon
· Twine
· Voat
· Vizited
· Yummymarks
· Xmarks
· Yahoo! Buzz
· Zootool
· Zotero
SOCIAL NETWORKS
Bebo  · BlackPlanet·
Classmates.com
· Cyworld
· Dogster  · Dopplr· douban
· Ello
· Facebook  · Flixster· FriendFeed
· Friendster  ·
Friends Reunited  · Gaia Online· Google+
· Habbo
· hi5
· Hyves
· iWiW  · LinkedIn · Miiverse  · mixi· MyHeritage
· MyLife
· Myspace
· myVIP  · Netlog
· Odnoklassniki
· Orkut
· Plaxo
· Qzone
· Renren
· Skyrock
· Sonico.com
· Storylane
· Tagged
· tvtag
· Upcoming  · Viadeo· Vine
· Vkontakte
· WeeWorld
· Yahoo! Upcoming
· more sites...
SHOPPING/RETAIL
Alibaba  ·
AliExpress  ·
Amazon  · Apple Store· Barnes &
Noble  ·
DirectCanada  · eBay· Kmart
· NCIX
· Printfection  ·
RadioShack  · Sears  · Sears Canada  · Target· The Book
Depository  · ThinkGeek· Toys "R" Us
· Walmart
SOFTWARE/CODE HOSTING Android Development  · Alioth· Assembla
· BerliOS
· Betavine
· Bitbucket
· BountySource
· Codecademy
· CodePlex  ·
Freepository  ·
Free Software Foundation·
GNU Savannah  ·
GitHost  · GitHub  · GitHub Downloads  · Gitorious · Gna!  · Google Code · ibiblio  · java.net· JavaForge
· KnowledgeForge
· Launchpad
· LuaForge
· Maemo
· mozdev
· OSOR.eu
· OW2 Consortium
· Openmoko
· OpenSolaris
· Ourproject.org
· Ovi Store
· Project Kenai
· RubyForge
· SEUL.org
· SourceForge
· Stypi  · TestFlight· tigris.org
· Transifex
· TuxFamily
· Yahoo!
Downloads
TELEVISION/RADIO
ABC  · Austin CityLimits  ·
BBC  · CBC
· CBS
· Computer Chronicles· CTV
· Fox
· G4
· Global TV
· Jeopardy!
· NBC
· NHK
· PBS
· Penn & Teller:
Bullshit!
·
The Howard Stern Show· TV
News Archive (Understanding 9/11)
TORRENTING/PIRACY
ExtraTorrent  · EZTV· isoHunt
· KickassTorrents
· The
Pirate Bay  · Torrentz· Library Genesis
VIDEO HOSTING
Academic Earth  · Bambuser· Blip.tv  · Epic
· Freshlive  · Google Video  · Justin.tv · Mixer  · Niconico· Nokia Trailers
· Oddshot.tv  ·
Periscope  · Plays.tv · Qwiki  · Skillfeed · Stickam  · TED Talks· Ticker.tv  ·
Twitch.tv  · Ustream  · Videoplayer.hu  · Viddler· Viddy  · Vidme
· Vimeo  · Vine
· Vstreamers  · Yahoo! Video  · YouTube  · Famous Internet videos (Me at thezoo )
WEB HOSTING
Angelfire  · Brace.io· BT Internet  ·
CableAmerica Personal Web Space · Claranet NetherlandsPersonal Web Pages
· Comcast
Personal Web Pages  · Extra.hu  · FortuneCity· Free ProHosting
· GeoCities
(patch ) · Google Business Sitebuilder  · Google Sites· Internet Centrum
· MBinternet
· MSN TV  · Nifty
· Nwnyet  · Parodius Networking  · Prodigy.net· Saunalahti Iso G
· Swipnet  ·
Telenor  · Tripod  · University of Michigan personal webpages· Verizon
Mysite  · Verizon Personal Web Space· Webs  ·
Webzdarma  · Virgin MediaWEB APPLICATIONS
Mailman  · MediaWiki  · phpBB  · Simple Machines Forum· vBulletin
INFORMATION
A Million Ways to Die on the Web· Backup Tips
· Cheap storage
· Collecting items randomly · Data compression algorithmsand tools  · Dev
· Discovery Data  · DOS Floppies  · Fortress of Solitude· Keywords
· Naughty List  ·
Nightmare Projects  · Rescuing floppy disks  · Rescuing optical media· Site exploration
· The WARC Ecosystem · Working with ARCHIVE.ORGPROJECTS
ArchiveCorps  · Audit2014· Emularity  ·
Faceoff  · FlickrFckr· Froogle  ·
INTERNETARCHIVE.BAK (Internet Archive Census ) · IRC Quotes· JSMESS  ·
JSVLC  · Just Solve the Problem· NewsGrabber
· Project Newsletter· Valhalla  ·
Web Roasting (ISP Hosting · University Web Hosting) · Woohoo
TOOLS
ArchiveBot · ArchiveTeam Warrior (Tracker ) · Google Takeout· HTTrack
· Video downloaders
· Wget (Lua
· WARC
)
TEAMS
Bibliotheca Anonoma  · LibreTeam · URLTeam  · Yahoo Video Warroom  · WikiTeamOTHER
800notes  · AOL  · Akoha· Ancestry.com  ·
April Fools' Day  · Amplicate · AutoAdmit  · Bre.ad · Circavie  · Cobook · Co.mments  · Countdown· Discourse  ·
Distill  · Dmoz  · Easel · Eircode  · ElectronicFrontier Foundation
·
FanFiction.Net  · Feedly · Ficlets  · Forrst · FunnyExam.com  · FurAffinity  · Google Helpouts· Google Moderator
· Google Poly
· Google Reader
· ICQmail  · IFTTT · Jajah  · JuniorNet· Lulu Poetry  ·
Mobile Phone Applications  · Mochi Media  · Mozilla Firefox· MyBlogLog  ·
NBII  · Newgrounds  · Neopets  · Quantcast  · Quizilla  · Salon Table Talk· Shutdownify
· Slidecast  ·
Stack Overflow  · SOPA blackout pages · starwars.yahoo.com· TechNet  ·
Toshiba Support  · USA-Gov · Volán  · Widgetbox · Windows Technical Preview· Wunderlist
· YTMND  · ZoocasaABOUT ARCHIVE TEAM
Introduction  · Philosophy · Who We Are  · Our stance on robots.txt  · Why Back Up?· Software  ·
Formats  · Storage Media · Recommended Reading · Films and documentaries aboutarchiving  ·
Talks  · In The Media· FAQ
Retrieved from
"https://wiki.archiveteam.org/index.php?title=ArchiveBot&oldid=46353"Categories :
* ArchiveBot
* Archive Team
* Bots
NAVIGATION MENU
PERSONAL TOOLS
* Create account
* Log in
NAMESPACES
* Page
* Discussion
VARIANTS
VIEWS
* Read
* View source
* View history
MORE
SEARCH
NAVIGATION
* Main page
* Recent changes
* Random page
* Help about MediaWikiTOOLS
* What links here
* Related changes
* Special pages
* Printable version
* Permanent link
* Page information
* This page was last edited on 21 February 2021, at 17:53.* Privacy policy
* About Archiveteam
* Disclaimers
*
Details
Copyright © 2024 ArchiveBay.com. All rights reserved. Terms of Use | Privacy Policy | DMCA | 2021 | Feedback | Advertising | RSS 2.0