Collective Intelligence and open source

For the past 20 years, I have been for an open source author and contributor (I will use the term ‘open source’ but I could say ‘free software’). I had the occasion to think about proper collaborative environments for group collaboration around writing some piece of software. When I held a leadership position in the Tikiwiki community my main concern was to enable an efficient context for massive participation. At that time I was called a ‘reluctant leader’ for the project, while I was considering myself an ‘attentive gardener’.

Collaboration can be instinctive in some contexts. But engineering projects need some flavor of coordination to make things go smoothly. The goal being to reach the threshold of the sum. It happens when it’s more efficient to join the collective than to work in isolation. The collective action is more than the sum of the parts. It’s quite easy to achieve and it happens in every business. But in an environment without enforcement of an authority, like in open source projects, the setup has to rely on autonomy of the participants and their willingness to play their part.

In such context, I immediately got seduced by my encounter with collective intelligence. The emerging whole, the autonomy of the parts, the object-link, the need for holopticism and defined standards are many aspects that we could find in an open source development team.

Since that time, Github, in 2008, changed the way open source is done. Before, on sourceforge or such platforms, contributors needed to ‘join a group’, request access, whatever the protocol was decided by the maintainers that already had access to CSV or SVN repositories. Now on git-based projects, contributions don’t require joining any group, Pull Request are just an easier way to submit patches without the need to agree on any prerequisite. Collaboration began to become punctual, sporadic, in many cases just a one-shot contribution. Of course there are still communities that keep a strong context beyond that sporadic aspect. Core teams still exist following various models (I saw a nice video recently explaining some of those models). In most cases the link between participants was not as tight as it could have been previously.

But from the perspective of the gardener, it doesn’t change the equation. Variables are different, but the logic is the same. We still want, in a loosely linked relationship, to extract more impact than if the relationship was not there. And the recipe for designing an environment will follow the same principles.

Here are the elements that characterize an emerging collective intelligence, they apply pretty well to the context of community-oriented open source software development:

  • an emerging whole: the result of the collaboration cannot be predicted, it emerges from the autonomous actions from the participants. Steering is possible, but if the given direction doesn’t match individual interests of the participants, it will just fail. In that regard, open participation and flexibility make the emergence easier.

  • holopticism: this weird word is synonym of transparency, visibility and reachability. Each participant, being autonomous, needs to have a view on what others are doing and where the whole is going. It’s all about creating bridges instead of walls. The opposite of holopticism is panopticism, well known in hierarchical companies where everybody knows their part and cubicle protects from distractions. Such closed context creates a strong dependence to a ‘manager’ and negates any possibility of autonomy.

  • rules of the game: in a strongly oriented technical context, there are various rules that are supposed to be already known by the participants, related to the conventions of the craft. But any other non technical rules, defining a social contract, need to be clear and explicit for each participant. It’s about license of the common good, ways to interact, maybe code of conduct and any other convention that make the collaboration clear and rules enforceable by each participant.

  • switchable roles: because nobody can be hold accountable for completion of any task, several participants should be able to accomplish any task required for the accomplishment of the shared goals. For example a release process, code review, mentoring of newcomers etc. Those tasks cannot be exclusive to designated people, because if they got stuck, the whole collective gets stuck. This polymorphism of the group is necessary for its resilience.

  • linking object: in order to help autonomous participants to synchronize their efforts, a common object is required to be defined. The term ‘object’ is wide in the sense that it can be a milestone, a todo list, an objective that is likely to attract the approbation of all participants, or simply the maintenance of a codebase or an infrastructure. This object can be evolving if this evolution follows the rules of the holopticism: everybody has to be aware of the change (release done, infrastructure in crisis, milestone passed, changes in a todo list, …)

  • transactive memory system: this is the way for newcomers to acquire knowledge of past iterations of the project, of the life of the group and the linking object. This can be served by availability of logs, mentoring sessions, conversational approach of story-telling, stored minutes of group interactions, etc. But it also includes specific parts of knowledge required for activity in the group, possibly tutorials, guides and any learning material. It also includes clear description of the procedures to achieve any tasks, enabling the ‘switchable roles’ principle.

  • collaborative economy: also called a gift economy, as opposed to a contractual economy. In a collaborative economy, the gift comes first, and the reward comes from the relevance of the gift to the common goal. In a contractual economy, the reward is agreed first and the task is accomplished in the conditions agreed upon. For nurturing a rewarding collaborative economy, a culture of recognition of peers is required. Good actions need to be acknowledged and praised.

Those 7 principles are extracted and adapted from the definition of the original collective intelligence as presented in the Collective Intelligence, the invisible revolution from JF Noubel.

Now that those principles are identified, I can think of some more point that need attention.

  • architects, catalysts and gardeners: all the principles above are not occurring naturally, they need to be designed, engineered, nudged, maintained and renewed. A great part of the job is at the start of a project to prepare the ground for the incoming collaborators, like a seed that will possibly be modified under the influence of the future participants. But it has to start somewhere. And during the lifetime of the collaboration, attention has to be dedicated to keep all those elements active and vibrant. Some participants, being concerned by the core principles, have to step up, pay attention to the environment, and operate towards the balance of the whole for its harmonious development.

  • gradual involvement pathways: there should be a choice for any participant to chose their degree of participation. It means that someone that just wants to fix a typo on a text should have one or two paragraphs to read before proposing their help. Someone willing to help in translations should have not more than a page to read to understand the translation process and workflow. Someone that wants to dig deeper should be able to enter the cycle of self-learning with all available sources from the collective.

  • specialized sub-collectives: some tasks cannot be shared widely, like a root access to a server for example. But each group in charge of a specific task (system administration, finances, legal, etc.) should be composed by more than one person and follow the same rules, being closed doors, as the rest of the collective. Pragmatically this setup is only realist for projects of a certain size. Contingency often dictates that only one person is holding responsibility for some specialized task. But it should be kept in mind as a vivid truth that this is not an optimal situation. It is a vulnerability in the setup when only one person is holding a specialized role in the collective. But in any case, the default position regarding information should remain the path of sharing with others, and the secrecy is being decided as an exception for very specific reasons (security, privacy, legality, etc.).

  • arbitration mechanism: when vision fails to converge, arbitration can be required. Its procedure should be clear as part of the rules of the game, and can involve voting, call for a referee or a jury, consultation of participants whose reputation is above all questioning if there are any, mediation by someone playing the role of a catalyst, etc. It’s not required, and it would actually be harmful, to write a full book of law. A couple of paragraphs describing the arbitration convention is enough.

  • tools and platforms: various tools need to be available for synchronous and asynchronous exchange between participant, collective gathering of activity logs, centralized or replicated availability of the rules of the game and transactive memory elements. Depending the nature of the project, some more specialized tools may be required, especially for the holopticism part. Transparency is not a natural state and it requires a good toolbox for its enforcement: graphs and statistics, monitoring of activity, track records, audit logs, periodic summaries of activity, subscription-based diffusion of activity logs, reports builders, etc. There is never too much of it. Abundant information can be filtered selectively, but missing information cannot be guessed.

  • interpersonal bonding: it’s human nature. It’s much more enjoyable to get involved in a collective initiative when it creates personal links. The collaboration space should not only be geared toward efficiency, but should leave space for random and personal interactions. While it’s common wisdom to avoid speaking about religion or politics, any other subject can be shared among participants (well, in various cases I also have seen religion and politics debated without harm among mature participants). Well, like in real life. We are social beings, after all. And the social link is an extra element of satisfaction.

There are various other details but those are the main ones, at least in my experience. The reality of things is certainly far from perfect and there is a majority of open source projects that miss many of the points I list here. But for those who match this list of characteristic, I notice that they are much more successful, efficient and healthy. When the state of Collective Intelligence is reached, the shared benefit can be felt by all participants. Beyond the fact that one does participate in full autonomy and voluntarily, perceiving the added value of the collaboration is a form of accomplishment.

comments powered by Disqus