In my previous post, I described the elements necessary for fostering collective intelligence in Open Source projects. That synthetic description was very absolute, and for a proper implementation it needs some adjustment regarding both space and time.

The space variable is environmental, where the project takes in account its surrounding. Partnerships, dependencies, markets, and customers are some of the impacting external factors that need to be taken in account.

The time parameter involves considering the intermediary steps leading to the birth of a project, the seeding, the evolutions, the mutability and transmission procedures between actors, and the death of the project. Where space is all about environment, time is all about evolution.

By addressing both those factors, I think we are much closer to a blueprint that would make sense to the adoption of Collective Intelligence in Enterprise.

In Space

Space, or environment, or external impacting factors, are bringing some constraints:

Deadlines

• events taking place at given dates,
• synchronization with other projects,
• unexpected hazards like security alerts,
• partnership contractual obligation,
• legal obligations from local state,

Instinctively one could think that deadline are a time parameter, but actually their impact is more environmental than evolutionary. Setting hard time constraint has for only purpose synchronization with external factor: customers communication, market events or strong trends, predictability in the case of fixed deadlines. All those factors are geared toward collaboration.

Deadlines impact the emerging whole, in the sense that an organized group cannot rely on unpredictable emergence, and requires some enforcement mechanism to comply with deadlines. It leads to the necessity to setup a steering system, that will watch over deadlines and adjust the whole for timely emergence. This can be done in a very soft way, by nudging more than hardcore enforcement. But there are many situations where it’s obvious that arbitrary decisions have to be made for the sake of meeting deadlines.

Many Open Source projects have very small concern for deadlines, because they reduce their environmental surface area. But large Open Source projects can’t escape this. For example it’s very convenient for large projects to make a contract with user about the release cycle periodicity. Gitlab releases every month, there is a new Ubuntu release every 6 months, and yes, both those projects are lead by companies.

The ways to address this constraint requires an extra attention to the holopticism. Each participant needs to be totally aware of deadlines so they can adjust their own pace.

Certain deadlines involve specific tasks, like releases for example, which can be subject to a rotation for better resiliency, in the spirit of switchable roles. The example of how Gitlab manages releases illustrates this concept pretty well.

Often there is going to be the need to identify certain roles for deadline enforcement in the case of events. Depending the scale, it can be clever to create specialized sub-collectives. Like the “2017 rubyconf team” or the “first tuesday meetup team”. But it also can be a switchable role in case of small requirements for the task.

For an optimal and fluid operation, those roles and constraints need to be clear in the rules of the game and possibly in the arbitration mechanism too. Sometimes failing on a deadline can be very harmful for the project and the collective, and the collective then need some aggressive process to protect its survival.

It’s especially true for security alerts, in case of software development projects or projects managing infrastructures, which require a strongly steered crisis management with very tight deadlines.

Dependencies

• with vendors
• with customers
• with contractors
• with specialized third parties (press, social media, integrators)

We saw that specialized sub-collectives can be required to handle deadlines. The same goes for dependencies with other projects or vendors, or even customers. The Collectively Intelligent organization usually needs to provide some dedicated contact role which can be either a person or a group. Such a group or person will then be in charge to inform the rest of the organization about their own deadlines and other requirements.

Those sub-collectives are in charge of doing a proxy-triage work in the communication between the collective and the rest of the world. For example the Support team can handle user requests, communication team the press releases and press events, the community team would handle events, etc.

The responsibility on such teams is both internal and external. They need to make sure that the collective knows about external inputs, and the external actors know about the internal activity that may impact them.

Conformity

• legal requirements
• standards compliance
• intellectual property liability
• contractual liability with partners or vendors

There can be a wide range of conformity requirement that a project can’t ignore. It may be hard to predict them all, but a general approach could be taken about them. Such requirements can be very disruptive in the engineering of internal collective intelligence, in some occasions. It’s even worse when conformity requires some form of secrecy, which tampers the need for transparency.

For open conformity requirements, they need to be made public, at least in the circle of the project, by the way of the rules of the game. It may be necessary to appoint a specialized sub-collective for enforcing certain type of conformity, like for example a legal team, or a special quality assurance team that will make sure contractual requirements are met. Some conformity rules may involve some special skill or certifications.

But as a general rule, the impact surface of conformity compliance should be kept at its minimum if possible.

Physicality

• office conventions
• resources management
• expenses coverage
• toolset and procedures

When a physical space is involved, some chores have to be organized for its maintenance. They can be shared by some sort of common greed convention, or handled by a dedicated group. There is not much challenge about this, as it calls for practices that are very well documented.

When it comes to tools and procedures, it’s about making the space adequate for collaboration. Open space stimulates transparency, for example, and direct and quick access to others.

There is a trend currently that frowns upon open spaces in the workplace, because it’s disturbing concentration and breaks the ‘zone’ state that developers can try to maintain. But in my opinion shared physical spaces should only be used for interactive activities. If one needs concentration, they should isolate themselves, either at home or in a closed space.

The consequence of such a statement is that remote activity should be possible, depending the kind of activity of each member of the collective. Some will need more isolation than others.

Virtuality

• online sync and async tools
• knowledge database and shared memory
• remote presence tools
• structured and transparent tasks tracking

Taking in account that economy cannot be made of a virtual activity except in rare cases, a proper thinking has to be made about the tools deployed to serve such purpose. Details will obvisouly depend the craft of the trade at hand, but there are some constant.

A common space for synchroneous communication is required, for all participant to access each others in a fluid and interactive way. If this interaction can be seen by all others, it’s even better. Transparencey by default is a factor of success. Nowadays solutions like IRC, Campfire, Hipchat, Slack, Rocketchat, satisfy the requirement for such a platform.

The common sync space needs to be archived and its content needs to be searchable. This ensures that async knowledge can be guaranteed.

For the past 20 years, I have been for an open source author and contributor (I will use the term ‘open source’ but I could say ‘free software’). I had the occasion to think about proper collaborative environments for group collaboration around writing some piece of software. When I held a leadership position in the Tikiwiki community my main concern was to enable an efficient context for massive participation. At that time I was called a ‘reluctant leader’ for the project, while I was considering myself an ‘attentive gardener’.

Collaboration can be instinctive in some contexts. But engineering projects need some flavor of coordination to make things go smoothly. The goal being to reach the threshold of the sum. It happens when it’s more efficient to join the collective than to work in isolation. The collective action is more than the sum of the parts. It’s quite easy to achieve and it happens in every business. But in an environment without enforcement of an authority, like in open source projects, the setup has to rely on autonomy of the participants and their willingness to play their part.

In such context, I immediately got seduced by my encounter with collective intelligence. The emerging whole, the autonomy of the parts, the object-link, the need for holopticism and defined standards are many aspects that we could find in an open source development team.

Since that time, Github, in 2008, changed the way open source is done. Before, on sourceforge or such platforms, contributors needed to ‘join a group’, request access, whatever the protocol was decided by the maintainers that already had access to CSV or SVN repositories. Now on git-based projects, contributions don’t require joining any group, Pull Request are just an easier way to submit patches without the need to agree on any prerequisite. Collaboration began to become punctual, sporadic, in many cases just a one-shot contribution. Of course there are still communities that keep a strong context beyond that sporadic aspect. Core teams still exist following various models (I saw a nice video recently explaining some of those models). In most cases the link between participants was not as tight as it could have been previously.

But from the perspective of the gardener, it doesn’t change the equation. Variables are different, but the logic is the same. We still want, in a loosely linked relationship, to extract more impact than if the relationship was not there. And the recipe for designing an environment will follow the same principles.

Here are the elements that characterize an emerging collective intelligence, they apply pretty well to the context of community-oriented open source software development:

• an emerging whole: the result of the collaboration cannot be predicted, it emerges from the autonomous actions from the participants. Steering is possible, but if the given direction doesn’t match individual interests of the participants, it will just fail. In that regard, open participation and flexibility make the emergence easier.

• holopticism: this weird word is synonym of transparency, visibility and reachability. Each participant, being autonomous, needs to have a view on what others are doing and where the whole is going. It’s all about creating bridges instead of walls. The opposite of holopticism is panopticism, well known in hierarchical companies where everybody knows their part and cubicle protects from distractions. Such closed context creates a strong dependence to a ‘manager’ and negates any possibility of autonomy.

• rules of the game: in a strongly oriented technical context, there are various rules that are supposed to be already known by the participants, related to the conventions of the craft. But any other non technical rules, defining a social contract, need to be clear and explicit for each participant. It’s about license of the common good, ways to interact, maybe code of conduct and any other convention that make the collaboration clear and rules enforceable by each participant.

• switchable roles: because nobody can be hold accountable for completion of any task, several participants should be able to accomplish any task required for the accomplishment of the shared goals. For example a release process, code review, mentoring of newcomers etc. Those tasks cannot be exclusive to designated people, because if they got stuck, the whole collective gets stuck. This polymorphism of the group is necessary for its resilience.

• linking object: in order to help autonomous participants to synchronize their efforts, a common object is required to be defined. The term ‘object’ is wide in the sense that it can be a milestone, a todo list, an objective that is likely to attract the approbation of all participants, or simply the maintenance of a codebase or an infrastructure. This object can be evolving if this evolution follows the rules of the holopticism: everybody has to be aware of the change (release done, infrastructure in crisis, milestone passed, changes in a todo list, …)

• transactive memory system: this is the way for newcomers to acquire knowledge of past iterations of the project, of the life of the group and the linking object. This can be served by availability of logs, mentoring sessions, conversational approach of story-telling, stored minutes of group interactions, etc. But it also includes specific parts of knowledge required for activity in the group, possibly tutorials, guides and any learning material. It also includes clear description of the procedures to achieve any tasks, enabling the ‘switchable roles’ principle.

• collaborative economy: also called a gift economy, as opposed to a contractual economy. In a collaborative economy, the gift comes first, and the reward comes from the relevance of the gift to the common goal. In a contractual economy, the reward is agreed first and the task is accomplished in the conditions agreed upon. For nurturing a rewarding collaborative economy, a culture of recognition of peers is required. Good actions need to be acknowledged and praised.

Those 7 principles are extracted and adapted from the definition of the original collective intelligence as presented in the Collective Intelligence, the invisible revolution from JF Noubel.

Now that those principles are identified, I can think of some more point that need attention.

• architects, catalysts and gardeners: all the principles above are not occurring naturally, they need to be designed, engineered, nudged, maintained and renewed. A great part of the job is at the start of a project to prepare the ground for the incoming collaborators, like a seed that will possibly be modified under the influence of the future participants. But it has to start somewhere. And during the lifetime of the collaboration, attention has to be dedicated to keep all those elements active and vibrant. Some participants, being concerned by the core principles, have to step up, pay attention to the environment, and operate towards the balance of the whole for its harmonious development.

• gradual involvement pathways: there should be a choice for any participant to chose their degree of participation. It means that someone that just wants to fix a typo on a text should have one or two paragraphs to read before proposing their help. Someone willing to help in translations should have not more than a page to read to understand the translation process and workflow. Someone that wants to dig deeper should be able to enter the cycle of self-learning with all available sources from the collective.

• specialized sub-collectives: some tasks cannot be shared widely, like a root access to a server for example. But each group in charge of a specific task (system administration, finances, legal, etc.) should be composed by more than one person and follow the same rules, being closed doors, as the rest of the collective. Pragmatically this setup is only realist for projects of a certain size. Contingency often dictates that only one person is holding responsibility for some specialized task. But it should be kept in mind as a vivid truth that this is not an optimal situation. It is a vulnerability in the setup when only one person is holding a specialized role in the collective. But in any case, the default position regarding information should remain the path of sharing with others, and the secrecy is being decided as an exception for very specific reasons (security, privacy, legality, etc.).

• arbitration mechanism: when vision fails to converge, arbitration can be required. Its procedure should be clear as part of the rules of the game, and can involve voting, call for a referee or a jury, consultation of participants whose reputation is above all questioning if there are any, mediation by someone playing the role of a catalyst, etc. It’s not required, and it would actually be harmful, to write a full book of law. A couple of paragraphs describing the arbitration convention is enough.

• tools and platforms: various tools need to be available for synchronous and asynchronous exchange between participant, collective gathering of activity logs, centralized or replicated availability of the rules of the game and transactive memory elements. Depending the nature of the project, some more specialized tools may be required, especially for the holopticism part. Transparency is not a natural state and it requires a good toolbox for its enforcement: graphs and statistics, monitoring of activity, track records, audit logs, periodic summaries of activity, subscription-based diffusion of activity logs, reports builders, etc. There is never too much of it. Abundant information can be filtered selectively, but missing information cannot be guessed.

• interpersonal bonding: it’s human nature. It’s much more enjoyable to get involved in a collective initiative when it creates personal links. The collaboration space should not only be geared toward efficiency, but should leave space for random and personal interactions. While it’s common wisdom to avoid speaking about religion or politics, any other subject can be shared among participants (well, in various cases I also have seen religion and politics debated without harm among mature participants). Well, like in real life. We are social beings, after all. And the social link is an extra element of satisfaction.

There are various other details but those are the main ones, at least in my experience. The reality of things is certainly far from perfect and there is a majority of open source projects that miss many of the points I list here. But for those who match this list of characteristic, I notice that they are much more successful, efficient and healthy. When the state of Collective Intelligence is reached, the shared benefit can be felt by all participants. Beyond the fact that one does participate in full autonomy and voluntarily, perceiving the added value of the collaboration is a form of accomplishment.

Because I had some time recently I watched the whole batch of videos of the “Collective Intelligence Conference” from 2014 on youtube. It’s part of my exploration of that field, see what changed in the past years.

Well, those were pretty interesting talks but it stroke me that there was a clear ambiguity of terms there. It’s related to the english usage of the “intelligence” word. The exact same word in french only means the capacity to process information and take decisions. This is a human trait, somehow a way to distinguish us from the rest of the animal realm.

But in english it also is used by government as “data collection”, as in “business intelligence”, or “the intelligence community”. In french, we call that “renseignement”, so it’s a totally separate concept. So, from my french perspective, I didn’t anticipate the impact of this double meaning for the word “intelligence” while watching talks about “Collective Intelligence”.

So there was a lot of talks about crowdsourcing, about new ways to gather information from previously untaped sources by the way of massive internet participation. And not that much about the other intelligence which is about organizing data and structural design of human interaction.

They held conferences in 2012 but the links to videos failed, in 2014, which is the one I found videos for, in 2015 found no videos, in 2016 videos are in google drive. The 2017 edition is just happening right now and I hope there will be videos.

There are a lot of academic papers linked to those talks. Pretty good content.

But what I’m interested in seems to be a very small subset of what is called Collective Intelligence nowadays. I may need to define the term more explicitly. Let’s call it Intentional Collective Intelligence Engineering. Because as a software developer and a systems architect, I’m interested in building things, designing its cogs, anticipating its impact on usage and on norms.

While I like collecting feedback from the users of my solutions, I also know that the numerous choices you do when designing a system impact on the usages in very deep ways. Ways that the user cannot anticipate or imagine. Those are not always conscious habits that are influenced. There are always natural (or systemic) forces that drive a group to adopt specific strategies after having analyzed and matured the knowledge of its context.

And I’m pretty convinced that the Collective Intelligence I seek can be engineered. A proper environment, set of conventions, incentives and obstacles, can be deliberately designed for the purpose of emergence of that collective intelligence in a group of participants.

I have seen glimpses of it happen while participating in open source community development. I know there are bricks already existing to create various conditions necessary to the emergence. It’s not an utopia.

I plan to share some thoughts about Collective Intelligence from a software perspective. But this comes from an old story that I thought I should make public first.

The origins

When I first began to program web applications, it was mostly for my own needs as an activist (freedom of speech on internet, cryptography, and such matters). Around 1998 I had a fully working solution called ‘Collaborative writer’ or cWriter for short, written in PHP. It was like a sort of wiki geared towards small groups self-organization.

When I got to Claranet I deployed it as an internal solution, and worked on a full company intranet called ‘in’. It was the occasion to release cWriter as open source in 2000 on Sourceforge.

At that time it appeared obvious that my area of preference was about engineering collaborative work. I was putting a lot of thoughts into that. I had this clear understanding that the tool shapes the usages, especially in this new realm that was internet communication.

Later on, I dedicated my time in various projects, mostly Tikiwiki groupware in 2003. That experience was also rich in valuable experiments. I became leader in that community because being unemployed at that time, I took the occasion to just invest time in open source (being payed by labor insurance money). And I was just very prolific. I established various bases for that community that still thrives today. I had a recorded interview with the guys for the 10 years for the project in 2012.

The discovery

Somewhere in 2004 I met with JF Noubel, that had some need for help on his own Tikiwiki instance, called thetransitionner.org. His website (not available anymore nowadays) was dedicated to promotion and exploration of Collective Intelligence, following the work, among others, of Pierre Lévy.

I quickly identified this area of research as something that I was practicing and aiming at, without having formally identified it. So I got more and more interested into that field in the following years.

Jean-Francois became a friend, his approach evolved since then but I still got stuck with his original presentation describing the basics of Collective Intelligence.

Anyways it led me to some other kind of activism, revolving around some alternative currency projects (like openmoney), other projects concerning governance (like democracy 2.0, from Mikael Nordfors cited in the transcript on archive.org), and various other topics that would be too long to list here.

Ultimately it led to get involved in Angenius with Than Nghiem in a background of sustainable development between 2006 and 2008. Including going to live in a farm in Normandy. Which was amazing but I was just not made for it.

The migration

In 2008 I decided to leave France and go live in Taiwan. I wanted to take a break and learn Chinese. It was my decision for my 40th birthday. Since then I got various jobs in development and then in system administration, grinding more programming languages and technical skills. But I still kept all this time the background thinking on collective intelligence.

What I wanted to experience was just another cultural background. I wanted to see patterns a little better about open source communities and collaborative work in a different context. Well. This could be the topic for another post. I got some surprises.

What’s next?

Now, 9 years after, I feel it’s more than time I get back to my past ventures. Well, I didn’t live like an hermit, I’m involved in various local and global communities. But I feel I could share a little more about what I learned in my past experiments. Now that I reach my 50’s, maybe I just feel like I have to begin writing stuff.

For years I noticed the difficulty to extract decent information from developers and craftsmen about their work. Developers clearly lack the skill or the taste for documentation. It extends to the specifications, which also provides an occasion to notice bad performance.

To work this around, specialized project managers have to fill this role, but it can create a gap between developers ownership and the final result. Sometimes luck creates the inspiration and developers produce a decent amount of documentation. But honestly, this is pure luck, and not a rule.

I found a way to mitigate this issue, that I began to experiment some time ago at work. It’s based on a recorded interaction between a project person and a person of the craft.

Here are the rules:

• the project person organizes interviews with stakeholders from all specialties including developers, business people, operations people, sometimes users, the more diverse population possible. But it can also be done for no reason for sharing a certain kind of knowledge with an expert.
• interviews take place by written interactive communication in a chat, in a way that enables logging (irc, slack, whatever)
• it lasts 20 minutes more or less, but the format can extend to hours if the need is felt
• once the interview is finished, the interviewer cleans up the logs, removes out of topic details, fixes typoes, removes elements that are purely belonging to the chat way of communication
• the cleaned log is reviewed by the interviewee for consensual agreement on its publication
• the log can then be added to whatever project space is dedicated to the topic at hand (documentation, specification annexes, study, paper)

There are various beneficial side effects to this endeavor:

• actors have a better feeling of engagement in the project or topic at hand
• it creates a bond between actors, facilitates future exchanges
• it gives an equal chance for everyone to speak, even the shy ones, because the 1 to 1 context is much less frightening. In a meeting, there are people that never speak. Are they stupid? they are not!
• it creates content that can be shared with other actors so that they have a better chance to understand the point of views of other parties.
• it creates a useful reference for the project, a raw material which can be annexed and can be used for summaries. It also creates more content for an eventual search engine if the publication space has one.
• there is less risk to cite someone out of context because the full context is provided.
• it is much easier to organize interviews or hold them on the fly than setup meetings.
• there is no feeling of loss of time like in a meeting: the time dedicated is intense and very interactive, there is no waste.

I had the occasion to try that technique in a context where internal communication is not optimal, for knowledge sharing at first, then for exploring options on a new project and set up specifications. I have been happy to notice that this formula creates pretty good output and generates a nice feeling in all parties involved.

Certainly more study will come on this approach. It’s a lot of fun.

In the long road of my developer life I had the chance to experience a very wide variety of organizational models. The most pleasant was in the context of very large open source projects, where actions are not planned but still organized, and things fall in their place seemingly naturally. Of course there is nothing natural in that. There is a category of people, that can be called catalysts, working as gardeners and building the pathways to collaboration. But because there is no predefined hierarchy, I thought chaos had some virtues.

In other hierarchical models, there is so much waste following the rule rather than its essence. It’s like there was an abstraction layer for efficiency and the staff follows the abstraction, paying no respect to the efficiency. Because after a time the set of rules is not making sense anymore. The environment moves fast and habits are hard to change. It’s taking long time for an organization to change its own internal rules.

But I also have seen non-hierarchical model totally fail. When you try to apply an open-source kind of organization inside a company, it cannot be done half-way, but it cannot be done fully.

For example the volatility of contributors is an essential part of the open-source organization model. Things are working the way they do because people are free to leave and join at will, or stop working when they decide. This is totally different in a company, even if you can get some approximation, leaving and joining is a more complicated process, and has a different set of motivations. And let’s not talk about the freedom to stop working at will.

This single factor leads the free-formed communities to get various incentives for contributors to feel good about their interaction in the community. The ones that don’t play well along other people just end up either in a leadership position because they are geniuses, or just leave because they don’t fit in. Or they stay and kill the project because everybody else leaves. But most likely they are the reason why forks exist.

But in a non-hierarchic company, those cowboys may end up hurting the whole process of collaboration by capturing some processes, getting very good at them, and give hell to everybody else for a time, under the privilege of the Power of the Bottleneck. It’s very hard to get those people to share knowledge because their position depends on it. If nothing is done, the situation will become uneasy and awkward at best.

Certainly in that type of situation, if there is some power in place to mitigate this danger, all can be good and well. But from my experience such power is hard to come by. Especially if the non-hierarchic aspect of the organization depends on him/her/it. Maybe there is some way to have some kind of catalyst role, but where I have seen such role in a company, it was informal and not an official position.

That’s too bad because I would love to experience again some real collective intelligence in the workplace the same way I have experienced it in some open source communities. I think maybe there are some companies out there that are doing that well, but most of the time it’s not going to be structural. Most likely it will come from a specific set of people that do real good in collaboration. I still wait to see a company that includes in its genetic code, in its fundamental principle, the seeds that make it possible to be efficient and still instinctive.