TAO/Analytical framework for the evaluation of online communities

This analytical framework was developed by Markus Marquard (Ulm University, Centre for General Scientific Continuing Education) based on a discussion with Andreas Ninck (Bern University of Applied Sciences). It is inspired on the Business Model Canvas by Alexander Osterwalder and meant to serve consultants and administrators of online communities in strategy development. It emphasizes a systematic view on the internal workings of the online community and a clear definition of its boundaries.

Problem Statement

Every analysis and every consulting process should begin with the problem statement. There needs to be some kind of goal the online community is supposed to achieve. The next step is a strategy with several action steps on how to achieve this goal. This goal doesn’t need to be identical with the focal point of the online community. It may address short term interests or secondary goals as well.

The definition of the problem may be carried out in the form of a future workshop, a method with preparation, critique, fantasy and implementation phases developed by Robert Jungk. Other viable methods are interviews, focus groups or the metaplan technique.

The problem statement should establish clear goals for consultants or administrators of the online community. It should also include criteria for the measurement of the success of the implementation of the strategy.

Purpose / Identity

The analysis should then enunciate the purpose and the identity of the online community. While doing this, consider the purpose may differ depending on what part of the community you are looking at (micro vs. macro level analysis). The purpose might also shift of the course of time. In some cases, it makes sense to include future purposes of the online community as well.

The purpose forms the basis for the identity of the community. It shapes not only internal structures but establishes the boundaries of the community. Consulting processes should match the problem statement with the purpose/identity of the online community; otherwise conflicts might arise during the action steps. Purpose/identity of the online community is central during efforts to attract new members and supporters. It decides what kind of challenges or difficulty might when new members are introduced. Sometimes the strength of an online community manifests as a clear delineation of who belongs to the community and who doesn’t, but this can turn out to be a major factor in including new members.

Members / Target Audiences

Members are at the core of any online community. This includes employees as well. The analysis should establish which target audiences they belong to as well as what their interests, needs and motives are, maybe even that type of learning they prefer or what social milieu they belong to.

An aspect in this context is the concept of empowerment: Do the online community’s purpose and identity support a feeling of being able to contribute to society?

Other important aspects to consider are how networks in the online community form, what kinds of subgroups exist and what their dynamics are – for example interactions between their core and a periphery or how one can determine whether a single member belongs to the core or to the periphery. There also might be ranks or a hierarchy among members.

A different way of analyzing the online community is looking at the “life cycle” of a member, from the joining, to becoming a participant and to leaving the community. When it comes to the “drop outs”, which have left the community, including them in the analysis as well might yield interesting results.

Activities

What kinds of activities does the community pursue? What are roles, duties or fields of expertise in the community? How did roles, duties, fields of activities develop over time? Are activities focused on a single purpose or are they diversified? How did this diversity develop over time? How does the diversity of activities affect interactions within the community? If there indeed is a diverse set of activities to be found in the community, the analysis should cover if certain activities are considered more attractive be members or if certain activities are tied to ranks within the community.

Communication

What kinds of content and how does the community produce? In systems theory, the key to understanding an online community is gaining knowledge about what is being communicated, with special attention to self-referential communication within the system. The analysis may look at the forms of communication, problem during communication and distortions of meaning during communication. Another aspect worth delving into may be the modus of communication (i.e. face-to-face or computer-mediated). The size of the community, its norms and policies influence communication as well. Lastly, the technical infrastructure shapes of the community shapes communication.

Collaboration

This aspect of the analysis focuses on the way collaboration within the community is organized. Do members contribute as individuals or do they forms groups to work collaboratively on projects? When and why do groups dissolve? Are there effects of synergy and win-win-situations?

Networking

What kind of connections form between members of the community? Which factors influence these connections? Are connections based on sympathy or do they form on the basis of shared goals? Networking within the community is shaped by its structure (see: purpose / identity, members / target audiences, activities) and its size (e.g. small and welcoming vs. large and diverse). The technical infrastructure factors into this as well.

Relations with the external

What kind of relations do members of the community form with the external, i.e. the sphere beyond the boundaries of the community? How do these relations influence what’s happening in the community? Are members interested in what’s happening outside their community (extrovert) or do they tend to close themselves off from influences of the external (introvert)?

Technical infrastructure

The technical infrastructure is the foundation of the online community. Without it, communication and collaboration would not be possible. The technical infrastructure can exist in the form of mailing lists and emails. Communities may also use commercial social networking services like Facebook or Google+. Other technical infrastructures used are chat systems, online forums, virtual classrooms or web conferencing tools. It is important to consider the costs associated with the use of the technical infrastructure, its usability and its impact on communication and collaboration. For example, chats and web conferences are transient, while forums offer the option to discuss topics over the course of a longer time span. Virtual classrooms are geared towards working on group-based lectures or projects. The technical infrastructure should fit the purpose, the identity and the planned activities of the online community.

Organizational Culture

Is the organizational culture of the online community “top down” or “bottom up”? Is participation based on voluntary contributions, paid contracts or a mix of both? What formal and informal structures influence (and maybe direct) activities within the community? Are existing structures efficient? Do they allow for quick decisions and direct contact? Do members have to wait for decisions? Are there conflicts between members or groups of members on how to organize activities? Are there conflicts between volunteers and employees? Are roles, norms and policies transparent and reasonable? Is there a hierarchy of members and how does it influence the organizational culture?

Value Creation

Online communities require funds in order to continue to operate. Funding is tied to value creation. Value is created either for the members of the community (e.g. information and social support) or for stakeholders beyond the boundaries of the community (e.g. advertisers who buy banner ads displayed in the community). The created value has to be in harmony with the purpose and identity of the community (e.g. ads for lactose-free dairy products in a community supporting people with lactose intolerance). Value can be based on ideals (e.g. free knowledge) or fiscal goals (e.g. ad revenue). Administrators of online communities should think about who contributes to the creation of value and how these volunteers or employees can be fairly compensated for their contributions. Ideally, synergy effects form between contributors or administrators and members benefit from a win-win-situation.

Business Model

After the analysis of how value is created, efforts of administrators and consultants should focus on reformulating the business model of the online community. The key question here is whether the business model is sustainable on a long-term basis – or not. Online communities can be formed on a not-for-profit basis. They can also be focused on revenue, be it membership fees, donations or services subject to costs. Services (e.g. product testing by community members) can be performed by professionals, by semiprofessionals or by volunteers. Administrators should be wary of the effects of their business model on interactions within the community, e.g. some communities might reject certain business partnerships (e.g. ads for dairy products in a vegan community).

Strategy and Action Steps

In the process of consulting an online community, a strategy and concrete action steps should be the result of the analysis. Unquestionably, the analysis will have uncovered ideas for improvement. These ideas should be further developed and formed to action steps. These can be sorted by applying them to the various categories of this analytical framework. If there turn out to be more action steps than the community can handle at a given time, focus on the most important ones and leave the rest for a later time. Another possibility to reduce the amount of action steps is to focus on one category of the analytical framework. For example, only do action steps in the category “communication”. However you choose to proceed, remember the action steps shouldn’t stretch the resources of the community to its limits of what the community can handle and respect its purpose and its identity. Action steps also need to be realistic; no use in coming up with detailed plans that cannot possibly achieved. They should be specific, measurable, attainable, relevant and time-bound (SMART). It’s useful to set specific milestones and evaluate if they have been met. The progress of the action steps defined within the overarching strategy should be continually discussed with all people involved. If it becomes clear the current strategy is not working as intended, it should be reevaluated and modified according to the feedback.

Consulting concept

Consulting within TAO follows the analytical framework outlined above. The consulting process itself can be divided into several phases for several sub-goals or milestones. The consulting process should account for

problems that community administrators have to address;
conflicts between goals of administrators and members;
barriers to participation of stakeholders in community development.

The more thorough the evaluation of the platform and the more precise the problem statement are, the better consultants can work with administrators of an online community. Since online communities or complex entities, several evaluation phases during the consulting process should check for unintended consequences of the action steps employed.

Analytical framework for the evaluation of online communities in the context of TAO:

Problem Statement
Purpose / Identity	Communication	Collaboration	Value Creation
Members / Target Audiences	Networking	Technical Infrastructure	Business Model
Activities	Relations with the external	Organizational Culture
Strategy	Action Steps	Recommendations	Milestones

References

Marquard, Markus (2012): Analyseraster für Online-Communities. Arbeitspapier im TAO-Projekt. (unpublished)
Marquard, Markus (2013): Internetnutzung durch weiterbildungsinteressierte Ältere als Kompetenzentwicklung.(in print)