Uri Klarman
ABSTRACT
Four major search engines, Google in particular, hold a unique position in enabling the use of the Internet, as they alone direct over 98% of Internet users to the content they seek, using proprietary indices. While the contribution of these companies is undeniable, their design is necessarily affected by their economic interests, which may or may not align with those of the users, raising concerns regarding their effect on the availability of information around the globe. While multiple academic and commercial projects aimed to distribute and democratize the Web search, they failed to gain much traction, mostly due to inferior results and lack of incentives for participation. In this paper, we show how complex networking-intensive tasks can be crowdsourced using Bitcoin's incentive model. We present Webcoin, a novel distributed digital-currency which utilizes networking resources rather then computational, and can only be mined through Web indexing. Webcoin provides both the incentives and the means to create Google-scale indices, freely available to competing services and the public. Webcoin's design overcomes numerous unique challenges, such as index verification, scalability, and nodes' ability to actively manipulate webpages. We deploy 200 fully-functioning Webcoin nodes and demonstrate their low bandwidth requirements.
Supplemental Material
- Antminer S9 ASIC Bitcoin Miner. http://www.bitmain.com/.Google Scholar
- ASICS for Litecoin. Here They Come. https://bitcoinmagazine.com/articles/asics-litecoin-come-1394826069/.Google Scholar
- Bitcoin Energy Consumption Index - Digiconomist. https://digiconomist.net/bitcoin-energy-consumption.Google Scholar
- Bitcoinity.org. http://www.data.bitcoinity.org.Google Scholar
- Bitcoinstats. http://www.bitcoinstats.com.Google Scholar
- Compass. www.compass-project.org/.Google Scholar
- Faroo Search Engine. http://www.faroo.com/index.en.html.Google Scholar
- GOOGLE'S PRIVACY WHIPLASH SHOWS BIG TECH'S INHERENT CONTRADICTIONS. https://www.wired.com/story/googles-privacy-whiplash-shows-big-techs-inherent-contradictions/.Google Scholar
- Majestic 12 Search Engine. https://www.majestic12.co.uk/about.php.Google Scholar
- Neighborhood Pool Watch. http://www.organofcorti.blogspot.mx/.Google Scholar
- Pyminer. http://www.github.com/jgarzik/pyminer.Google Scholar
- Scrapy. http://www.scrapy.org.Google Scholar
- Steem - An incentivized, blockchain-based, public content platform. www.steem.io/SteemWhitePaper.pdf.Google Scholar
- YaCy Search Engine. http://www.yacy.net.Google Scholar
- Yes Facebook is using your 2FA phone number to target you with ads. https://www.techcrunch.com/2018/09/27/yes-facebook-is-using-your-2fa-phone-number-to-target-you-with-ads/.Google Scholar
- Operating system market share, 2017. http://www.netmarketshare.com/operating-systemmarket-share/.Google Scholar
- Abliz, M., and Znati, T. A Guided Tour Puzzle for Denial of Service Prevention. In Computer Security Applications Conference, 2009. ACSAC'09. Annual (2009), IEEE, pp. 279--288. Google ScholarDigital Library
- Ali, M., Nelson, J. C., Shea, R., and Freedman, M. J. Blockstack: A global naming and storage system secured by blockchains. In USENIX Annual Technical Conference (2016), pp. 181--194. Google ScholarDigital Library
- Anh, V. N., and Moffat, A. Inverted Index Compression using Word-Aligned Binary Codes. Information Retrieval 8, 1 (2005), 151--166. Google ScholarDigital Library
- Arias, E., and Guinot, B. Coordinated Universal Time UTC: Historical Background and Perspectives. In Journees systemes de reference spatio-temporels (2004).Google Scholar
- Barber, S., Boyen, X., Shi, E., and Uzun, E. Bitter to Better - How to Make Bitcoin a Better Currency. In International Conference on Financial Cryptography and Data Security (2012), Springer, pp. 399--414.Google ScholarCross Ref
- Bender, M., Michel, S., Triantafillou, P., Weikum, G., and Zimmer, C. Minerva: Collaborative P2P Search. In Proceedings of the 31st international conference on Very large data bases (2005), VLDB Endowment, pp. 1263--1266. Google ScholarDigital Library
- Boldi, P., Codenotti, B., Santini, M., and Vigna, S. Ubicrawler: A Scalable Fully Distributed Web Crawler. Software: Practice and Experience 34, 8 (2004), 711--726. Google ScholarDigital Library
- Bosch, A., Bogers, T., and Kunder, M. Estimating Search Engine Index Size Variability: a 9-Year Longitudinal Study. Scientometrics 107, 2 (2016), 839--856. Google ScholarDigital Library
- Chun, B., Culler, D., Roscoe, T., Bavier, A., Peterson, L., Wawrzoniak, M., and Bowman, M. Planetlab: an Overlay Testbed for Broad-Coverage Services. ACM SIGCOMM Computer Communication Review 33, 3 (2003), 3--12. Google ScholarDigital Library
- De Kunder, M. The Size of the World Wide Web. WorldWideWebSize (2012).Google Scholar
- Dimitri, N. Bitcoin mining as a contest. Ledger 2 (2017), 31--37.Google ScholarCross Ref
- Dziembowski, S. Introduction to Cryptocurrencies. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security (2015), ACM, pp. 1700--1701. Google ScholarDigital Library
- Eyal, I., and Sirer, E. G. Majority is Not Enough: Bitcoin Mining is Vulnerable. In International Conference on Financial Cryptography and Data Security (2014), Springer, pp. 436--454.Google ScholarCross Ref
- Feily, M., Shahrestani, A., and Ramadass, S. A Survey of Botnet and Botnet Detection. In Emerging Security Information, Systems and Technologies, 2009. SECURWARE'09. Third International Conference on (2009), IEEE, pp. 268--273. Google ScholarDigital Library
- Freund, A., and Stanko, D. The wolf and the caribou: Coexistence of decentralized economies and competitive markets. Journal of Risk and Financial Management 11, 2 (2018), 26.Google ScholarCross Ref
- Gilbert, H., and Handschuh, H. Security Analysis of SHA-256 and Sisters. In International Workshop on Selected Areas in Cryptography (2003), Springer, pp. 175--193.Google Scholar
- Gormley, C., and Tong, Z. Elasticsearch: The Definitive Guide. "O'Reilly Media, Inc.", 2015. Google ScholarDigital Library
- Grainger, T., Potter, T., and Seeley, Y. Solr in Action. Manning, 2014. Google ScholarDigital Library
- Hari, A., and Lakshman, T. The Internet Blockchain: A Distributed, Tamper-Resistant Transaction Framework for the Internet. In Proceedings of the 15th ACM Workshop on Hot Topics in Networks (2016), ACM, pp. 204--210. Google ScholarDigital Library
- Isele, R., Umbrich, J., Bizer, C., and Harth, A. LDspider: An Open-Source Crawling Framework for the Web of Linked Data. In Proceedings of the 2010 International Conference on Posters & Demonstrations Track-Volume 658 (2010), CEUR-WS. org, pp. 29--32. Google ScholarDigital Library
- Jagatic, T. N., Johnson, N. A., Jakobsson, M., and Menczer, F. Social Phishing. Communications of the ACM 50, 10 (2007), 94--100. Google ScholarDigital Library
- Jung, J., Krishnamurthy, B., and Rabinovich, M. Flash Crowds and Denial of Service Attacks: Characterization and Implications for CDNs and Web Sites. In Proceedings of the 11th international conference on World Wide Web (2002), ACM, pp. 293--304. Google ScholarDigital Library
- King, S., and Nadal, S. Ppcoin: Peer-to-Peer Crypto-Currency with Proof-of-Stake. self-published paper, August 19 (2012). www.wallet.peercoin.net/assets/paper/peercoin-paper.pdf.Google Scholar
- Kong, J. S., Sarshar, N., and Roychowdhury, V. P. Experience versus talent shapes the structure of the web. Proceedings of the National Academy of Sciences 105, 37 (2008), 13724--13729.Google ScholarCross Ref
- Langschaedel, J., Armstrong, B. D., and Ehrsam, F. E. Hot wallet for holding bitcoin, Sept. 17 2015. US Patent App. 14/660,418.Google Scholar
- Laurie, B., and Clayton, R. "Proof-of-Work" Proves Not to Work; version 0.2. In Workshop on Economics and Information, Security (2004).Google Scholar
- Lee, H.-T., Leonard, D., Wang, X., and Loguinov, D. IRLbot: Scaling to 6 Billion Pages and Beyond. ACM Transactions on the Web (TWEB) 3, 3 (2009), 8. Google ScholarDigital Library
- Leela, K., and Haritsa, J. SphinX: Schema-Conscious XML Indexing, 2001. http://dsl.cds.iisc.ac.in/pub/TR/TR-2001-04.pdf.Google Scholar
- McCandless, M., Hatcher, E., and Gospodnetic, O. Lucene in Action: Covers Apache Lucene 3.0. Manning Publications Co., 2010. Google ScholarDigital Library
- Meiklejohn, S., Pomarole, M., Jordan, G., Levchenko, K., McCoy, D., Voelker, G. M., and Savage, S. A Fistful of Bitcoins: Characterizing Payments Among Men with No Names. In Proceedings of the 2013 Conference on Internet Measurement Conference (2013), ACM. Google ScholarDigital Library
- Miller, A., Juels, A., Shi, E., Parno, B., and Katz, J. Permacoin: Repurposing Bitcoin Work for Data Preservation. In 2014 IEEE Symposium on Security and Privacy (2014), IEEE, pp. 475--490. Google ScholarDigital Library
- Nakamoto, S. Bitcoin: A Peer-to-Peer Electronic Cash System, 2008. www.bitcoin.org.Google Scholar
- of Money Research Collaborative:, F., Nelms, T. C., Maurer, B., Swartz, L., and Mainwaring, S. Social payments: Innovation, trust, bitcoin, and the sharing economy. Theory, Culture & Society 35, 3 (2018), 13--33.Google Scholar
- Park, S., Pietrzak, K., Alwen, J., Fuchsbauer, G., and Gazi, P. Spacecoin: A Cryptocurrency Based on Proofs of Space. Tech. rep., IACR Cryptology ePrint Archive, 2015: 528, 2015.Google Scholar
- Parreira, J. X., Donato, D., Michel, S., and Weikum, G. Efficient and Decentralized Pagerank Approximation in a Peer-to-Peer Web Search Network. In Proceedings of the 32nd international conference on Very large data bases (2006), VLDB Endowment, pp. 415--426. Google ScholarDigital Library
- Percival, C. Stronger Key Derivation via Sequential Memory-Hard Functions. Self-published (2009), 1--16. www.bsdcan.org/2009/schedule/attachments/87_scrypt.pdf.Google Scholar
- Pujol, J., and Rodriguez, P. Porqpine: a Distributed Social Search Engine. In In Proc. of WWW'09 (Madrid, Spain, Apr. 2009).Google Scholar
- Sankaralingam, K., Sethumadhavan, S., and Browne, J. C. Distributed Pagerank for P2P Systems. In High Performance Distributed Computing, 2003. Proceedings. 12th IEEE International Symposium on (2003), IEEE, pp. 58--68. Google ScholarDigital Library
- Serapiglia, A., Serapiglia, C. P., and McIntyre, J. Crypto Currencies: Core Information Technology and Information System Fundamentals Enabling Currency without Borders. Information Systems Education Journal 13, 3 (2015), 43.Google Scholar
- Swartz, L. Blockchain dreams: Imagining techno-economic alternatives after bitcoin. Another economy is possible (2017), 82--105.Google Scholar
- Wang, J., and Guo, Y. Scrapy-Based Crawling and User-Behavior Characteristics Analysis on Taobao. In Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC), 2012 International Conference on (2012), IEEE, pp. 44--52. Google ScholarDigital Library
- Wood, G. Ethereum: A Secure Decentralised Generalised Transaction Ledger. Ethereum Project Yellow Paper (2014).Google Scholar
- Wustrow, E., and VanderSloot, B. Ddoscoin: Cryptocurrency with a malicious proof-of-work. In Proceedings of the 10th USENIX Workshop on Offensive Technologies (2016), USENIX Association. Google ScholarDigital Library
Recommendations
Nonlinear Pricing of Information Goods
This paper analyzes optimal pricing for information goods under incomplete information, when both unlimited-usage (fixed-fee) pricing and usage-based pricing are feasible and administering usage-based pricing may involve transaction costs. It is shown ...
Incentivizing exploration
EC '14: Proceedings of the fifteenth ACM conference on Economics and computationWe study a Bayesian multi-armed bandit (MAB) setting in which a principal seeks to maximize the sum of expected time-discounted rewards obtained by pulling arms, when the arms are actually pulled by selfish and myopic individuals. Since such individuals ...
Core-selecting package auctions
AbstractAuctions that select core allocations with respect to reported values generate competitive levels of sales revenues at equilibrium and limit bidder incentives to use shills. Among core-selecting auctions, the ones that minimize seller revenues ...
Comments