SCIENTIA SINICA Informationis, Volume 47 , Issue 12 : 1601-1622(2017) https://doi.org/10.1360/N112017-00117

Software development based on collective intelligence on the Internet: feasibility, state-of-the-practice, and challenges

Wei ZHANG 1,2,*, Hong MEI 1,2,3
More info
  • ReceivedMay 20, 2017
  • AcceptedJul 21, 2017
  • PublishedNov 28, 2017


Funded by




[1] Brooks F P. No silver bullet: essence and accidents of software engineering. IEEE Comput, 1987, 20: 10--19. Google Scholar

[2] Brooks F P. The Mythical Man-Month: Essays on Software Engineering. 2nd ed. Boston: Addison-Wesley Professional, 1995. Google Scholar

[3] Brooks F P. The Mythical Man-Month: Essays on Software Engineering. 1st ed. Boston: Addison-Wesley Professional, 1975. Google Scholar

[4] Malone T W. The Future of Work: How the New Order of Business Will Shape Your Organization, Your Management Style and Your Life. Brighton: Harvard Business Review Press, 2004. Google Scholar

[5] Espinas A. Des Societes Animales: etudes de Psychologie Comparee. French ed. Ann Arbor: University of Michigan Library, 1877. Google Scholar

[6] Spencer H. The Principles of Sociology. London: Williams & Norgate, 1882. Google Scholar

[7] Thomas L. Lives of a Cell: Notes of a Biology Watcher. London: Penguin Books, 1978. Google Scholar

[8] Bonabeau E, Dorigo M, Theraulaz G. Swarm Intelligence: From Natural to Artificial Systems. Oxford: Oxford University Press, 1999. Google Scholar

[9] Engels F. Dialectics of Nature. 1883. Google Scholar

[10] Camazine S, Deneubourg J L, Franks N R, et al. Self-Organization in Biological Systems. Princeton: Princeton University Press, 2003. Google Scholar

[11] Surowiecki J. The Wisdom of Crowds. Norwell: Anchor, 2005. Google Scholar

[12] Fisher L. The Perfect Swarm: the Science of Complexity in Everyday Life. New York: Basic Books, 2009. Google Scholar

[13] Herculano-Houzel S. The human brain in numbers: a linearly scaled-up primate brain. Front Hum Neurosci, 2009, 3: 31. Google Scholar

[14] Malone T W, Laubacher R, Dellarocas C. The collective intelligence genome. MIT Sloan Manage Rev, 2010, 51: 21--31. Google Scholar

[15] Lee J, Kladwang W, Lee M, et al. RNA design rules from a massive open laboratory. Proc National Acad Sci, 2014, 111: 2122--2127. Google Scholar

[16] Lakhani K, Garvin D, Lonstein E. TopCoder (A): developing software through crowdsourcing. Harvard Business School General Management Unit Case No. 610--032, 2010. http://ssrn.com/abstract=2002884. Google Scholar

[17] Herbsleb J D. Global software engineering: the future of socio-technical coordination. In: Proceedings of Future of Software Engineering, Minneapolis, 2007. 188--198. Google Scholar

[18] Kogut B, Metiu A. Open source software development and distributed innovation. Oxford Rev Econ Policy, 2001, 17: 248--264. Google Scholar

[19] Weber S. The Success of Open Source. Cambridge: Harvard University Press, 2004. Google Scholar

[20] Raymond E. The cathedral and the bazaar. Knowl Technol Policy, 1999, 12: 23--49. Google Scholar

[21] Ghosh R A, Glott R, Krieger B, et al. Free/Libre and Open Source Software: Survey and Study. Maastricht: University of Maastricht, 2002. Google Scholar

[22] Ipeirotis P G, Provost F, Wang J. Quality management on amazon mechanical turk. In: Proceedings of the ACM SIGKDD Workshop on Human Computation, Washington, 2010. 64--67. Google Scholar

[23] Ipeirotis P G. Analyzing the amazon mechanical turk marketplace. XRDS Crossroads ACM Mag Students, 2010, 17: 16--21. Google Scholar

[24] Tsai W T, Wu W, Huhns M N. Cloud-based software crowdsourcing. IEEE Internet Comput, 2014, 18: 78--83. Google Scholar

[25] Wu W, Tsai W T, Li W. An evaluation framework for software crowdsourcing. Front Comput Sci, 2013, 7: 694--709. Google Scholar

[26] L$\acute{\rm~~e}$vy P. Collective Intelligence: Mankinds Emerging World in Cyberspace. New York: Basic Books, 1999. Google Scholar

[27] Mayer-Kress G, Barczys C. The global brain as an emergent structure from the worldwide computing network, and its implications for modeling. Inf Soc, 1995, 11: 1--27. Google Scholar

[28] Heylighen F, Bollen J. The world-wide web as a super-brain: from metaphor to model. In: Proceedings of Cybernetics and Systems 96, Vienna, 1996. 917--922. Google Scholar

[29] Bernstein A, Klein M, Malone T W. Programming the global brain. Commun ACM, 2012, 55: 41--43. Google Scholar

[30] Ahn L V. Human computation. Dissertation for Ph.D. Degree. Pittsburgh: Carnegie Mellon University, 2005. Google Scholar

[31] Quinn A, Bederson B. Human computation: a survey and taxonomy of a growing field. In: Proceedings of SIGCHI Conference on Human Factors in Computing Systems, Vancouver, 2011. 1403--1412. Google Scholar

[32] Lerner J, Tirole J. Some simple economics of open source. J Ind Econ, 2002, 50: 197--234. Google Scholar

[33] Mockus A, Fielding R, Herbsleb J. Two case studies of open source software development: apache and mozilla. ACM Trans Softw Eng Meth, 2002, 11: 309--346. Google Scholar

[34] Ye Y, Kishida K. Toward an understanding of the motivation of open source software developers. In: Proceedings of the 25th International Conference on Software Engineering, Portland, 2003. 419--429. Google Scholar

[35] Zhou M, Mockus A. What make long term contributors: willingness and opportunity in OSS community. In: Proceedings of the 34th International Conference on Software Engineering, Zurich, 2012. 518--528. Google Scholar

[36] Glance D. Release criteria for the Linux kernel. First Monday, 2004, 9: 52--64. Google Scholar

[37] Scacchi W. Free and open source development practices in the game community. IEEE Softw, 2004, 21: 59--66. Google Scholar

[38] Thomas C. Improving verification, validation, and test of the Linux kernel: the Linux stabilization project. In: Proceedings of the 3rd Workshop on Open Source Software Engineering, Washington, 2003. 133--136. Google Scholar

[39] Decker B, Ras E, Rech J, et al. Wiki-based stakeholder participation in requirements engineering. IEEE Softw, 2007, 24: 28--35. Google Scholar

[40] Nowak M, Pautasso C. Team situational awareness and architectural decision making with the software architecture warehouse. In: Proceedings of the 7th European Conference on Software Architecture, Montpellier, 2013. 146--161. Google Scholar

[41] Bruch M, Bodden E, Monperrus M, et al. IDE 2.0: collective intelligence in software development. In: Proceedings of FSE/SDP Workshop on the Future of Software Engineering Research, Santa Fe, 2010. 53--58. Google Scholar

[42] Wang H M, Yin G, Xie B, et al. Research on network-based large-scale collaborative development and evolution of trustworthy software. Sci Sin Inform, 2014, 44: 1--19. Google Scholar

[43] Zhang W, Zhao H, Jiang Y, et al. Stigmergy-based construction of Internetware artifacts. IEEE Softw, 2015, 32: 58--66. Google Scholar

[44] Zhang W, Yi L, Zhao H Y, et al. Feature-oriented stigmergy-based collaborative requirements modeling: an exploratory approach for requirements elicitation and evolution based on web-enabled collective intelligence. Sci China Inf Sci, 2013, 56: 082108. Google Scholar

[45] Miller G A. The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol Rev, 1956, 63: 81--97. Google Scholar

[46] Cowan N. The magical number 4 in short-term memory: a reconsideration of mental storage capacity. Behav Brain Sci, 2001, 24: 87--114. Google Scholar

[47] Sinha V, Chandra S, Sengupta B. A research agenda for distributed software development. In: Proceedings of the 28th International Conference on Software Engineering, Shanghai, 2006. 731--740. Google Scholar

[48] France R, Rumpe B. Model-driven development of complex software: a research roadmap. In: Proceedings of Future of Software Engineering, Minneapolis, 2007. 37--54. Google Scholar

[49] Mellor S J, Clark A N, Futagami T. Model-driven development. IEEE Softw, 2003, 20: 14--18. Google Scholar

[50] Theraulaz G, Bonbeau E. A brief history of stigmergy. Artif Life, 1999, 5: 97--116. Google Scholar

[51] Hippel E V, Krogh G V. Open source software and the “private-collective innovation model: issues for organization science. Organ Sci, 2003, 14: 209--113. Google Scholar

[52] Howe J. The rise of crowdsourcing. Wired Mag, 2006, 14: 1--4. Google Scholar

[53] Howe J. Crowdsourcing: Why the Power of the Crowd is Driving the Future of Business. London: Crown Business, 2008. Google Scholar

[54] Doan A, Ramakrishnan R, Halevy A Y. Crowdsourcing systems on the world-wide web. Commun ACM, 2011, 54: 86--96. Google Scholar

[55] Grier D A. When Computers Were Human. Princeton: Princeton University Press, 2005. Google Scholar

[56] Dawson R, Bynghall S. Getting Results From Crowds: the Definitive Guide to Using Crowdsourcing to Grow your Business. San Francisco: Advanced Human Technologies, 2011. Google Scholar

[57] Feng Y, Chen Z, Jones J, et al. Test report prioritization to assist crowdsourced testing. In: Proceedings of the 10th Joint Meeting on Foundations of Software Engineering, Bergamo, 2015. 225--236. Google Scholar

[58] Nielsen M. Reinventing Discovery: the New Era of Networked Science. Princeton: Princeton University Press, 2011. Google Scholar

[59] Heylighen F. Collective intelligence and its implementation on the web: algorithms to develop a collective mental map. Comput Math Organ Theory, 1999, 5: 253--280. Google Scholar

  • Figure 1

    A conceptual framework for organizational mechanisms of human collectives

  • Figure 2

    The evolution of organizational mechanisms of (a) human societies, (b) businesses, (c) software engineering

  • Figure 3

    Ideal Internet collective intelligence

  • Figure 4

    A framework for software engineering based on Internet collective intelligence

  • Table 1   Organizational mechanisms of human collectives with different decentralization degrees
    Organizational mechanism Description
    Centralized hierarchy Decision-making authority originates from a few individuals, and a collective of individuals are organized as a hierarchy according to the degree of decision-making authority: the higher a level is, the fewer individuals are at this level; and the higher level individuals are at, the more important decisions they can make.
    Loose hierarchy Similar to the centralized hierarchy in that the decision-making authority originates from a few high-level individuals. The difference is that considerable decision-making authority is delegated to lower-level individuals.
    Democracy Decision-making authority originates from each individual, and decisions are made by voting, or by a few individuals who are elected by voting as delegates of other individuals in a collective.
    Market No individual is bound by a decision this individual does not agree. Each pair of individuals (as a buyer and a seller, respectively) can make their complementary decisions, subject only to their own wills and abilities, and also the rules of a market.
    Individual independence More decentralized than market in that individuals are independent of each other: no decision of an individual is bound to a trade between individuals.
  • Table 2   The maturity analysis of sofware development based on Internet collective intelligence
    Aspect Theory/Technique Platform Practice
    Maturity ★☆☆☆☆ ★★☆☆☆ ★★☆☆☆
    Embodiment (1) Exist theories about collective intelligence phenomena in nature. (2) Exist post-analysis/ explaination about collective intelligence phenomena in software development, but lack theories that are commonly recognized. (3) Lack systematic understanding of key techniques. (1) Lack systematic platforms to enable collective intelligence in software development. (2) Exists platforms based on general-purpose collaboration techniques (i.e., online forum, and email list). (3) Exist elementary platforms to support souce code management and defect management. (1) Exist many naturally-formed Internet-based software development practices. (2) Many software systems produced by these practices show their market success. (3) The success experience of is unreproducible. (4) Big difference with ideal collective intelligence based software development.