Web crawling is the process of a number of computers scouring websites to save copies of everything there, allowing it to be saved and retained for the future. Then, browsers can use these scanned documents as answers to users’ queries, such as showing them a relevant document to a question that is asked.
However, this important feature of the internet is bandwidth-intensive, meaning sites only give access to some crawlers on the internet, while blocking the rest. Most commonly, Google is given priority over other crawlers, giving it a monopoly over information. The Knuckleheads’ Club wants to change that. For more information on how you can end this natural monopoly over the internet, visit their site here.
Photo by Glenn Carstens-Peters on Unsplash
