A healthy open source community is one that demonstrates open practices, uses open infrastructure, and cultivates an open culture with the goal of becoming more sustainable. But even for the most seasoned community architects, measuring an open source community’s health is a complex, difficult, and sometimes intimidating task.
That’s because any picture of project health is actually a mosaic. Multiple factors combine to depict a community’s overall health. You’ll never find just one indicator you can point to and say, “See? The community is healthy.” Although, you can, however, find some indicators that immediately show that a community is unhealthy.
What’s more, the factors that determine the overall project and community health are always changing.
In the past, some may have pointed to the total number of downloads as a health metric. But today we understand that even millions of downloads does not reflect the fact that a community’s production and quality assurance processes might be out of sync—or that trolls are running rampant in the project’s mailing lists.
Here are twelve ways in which to measure community health, and important factors to consider when assessing an open source project’s and community’s overall health:
1) Project life cycle
An open source project’s life cycle affects all other considerations about a project’s health. Understanding the project’s place in that life cycle will help you put your assessment into proper context (for example, single-person governance can be acceptable when a project is young—but not so great when a project is more mature). Moreover, monitoring contributor trends might tell you something about a project’s short-term or long-term future.
- When was the project founded? How old is it?
- How often do new contributors join the project?
- How frequently does the project accept new contributions?
2) Target audience
Well-run open source projects demonstrate a clear understanding of their target audiences or the users (and contributors) they hope to reach through their work.
- Has the project clearly identified a target audience? Who is it?
- Is that audience the most appropriate one for this project?
- Are you part of the audience this project has identified?
- Does this audience engage with projects that either complement or compete with this project?
“Governance” refers to the rules and customs that define who gets to do what—and how they are supposed to do it—in an open source project. Successful projects thoroughly document (and continuously evolve) their governance models.
- What is the project’s governance model, and is that model documented publicly?
- How do project members make decisions, and do members seem to abide by those decisions?
- Who seems responsible for ratifying and enforcing decisions?
- Does the project’s governance model account for both the technical and the business aspects of the project?
4) Project leads
Healthy projects have visible, easily identifiable leadership. Leads often coordinate project work and establish a project’s vision. They often have extensive knowledge of project history.
- Who are the project’s leads?
- What are the project leads’ responsibilities? Are they focused more on engineering, on marketing, or on some combination of both?
5) Release manager and process
Another identifier of project health is the state of the project’s release process. The healthiest projects have formally documented their release processes, and have identified release managers to oversee those processes.
- Has the project documented its release processes?
- Has the project identified a release manager?
- How often does the project release updates?
- Do project releases occur on a steady and predictable schedule?
The tools a community uses to work on a project are almost as important as the people who comprise that community. Different projects require different technical infrastructures. The most successful projects are those that have the tools they need to do their work—and keep those tools in good working order.
- Who is responsible for maintaining project infrastructure?
- Does the project have the bare minimum infrastructure it needs? Are infrastructural shortcomings producing bottlenecks for the project?
- Is the project missing useful infrastructural components? Does it have plans to obtain these components?
7) Goals and roadmap
Healthy open source projects share their goals publicly and maintain roadmaps they will follow to reach those goals. Generally, the most successful projects are those that set attainable goals and routinely meet the deadlines they set for themselves.
- Are the project’s goals clear and public?
- Does the project have a clearly communicated roadmap, and is it also public?
- Does the project have a history of meeting its release schedule (and other deadlines)?
8) Onboarding processes
New contributors are vital to project success. Without new members, a project can lose its creativity and vitality. Successful projects feature clear, welcoming onboarding materials that assist newcomers who wish to join.
- Do the community’s resources explain precisely what the project is?
- Does the community offer useful documentation on obtaining and using the project?
- Does the project publicly document processes new contributors can follow when they want to contribute to the project?
- Does the project welcome contributions of more than one type (e.g., development, marketing, project management, event planning, etc.)?
9) Internal communication
The presence or absence of internal communication channels is a key indicator of project health, as are a project’s internal communication practices. Often, issues affecting a community will emerge first in internal channels—such as mailing lists, chat platforms, or some other medium—where contributors and users interface with each other.
- Does the project have sufficient communication channels?
- Can people easily find and use these channels?
- Are channels regularly moderated?
- Is use of the channels governed by a code of conduct?
Outreach is the process of actively promoting a project and making others aware of it. Communities use written materials (social media, blogs, whitepapers), events (meetups, conventions), and educational tactics (demos, training sessions) as part of their outreach. Healthy projects devote adequate energy and resources to outreach.
- Does the community use a clear and consistent set of outreach tools? If not, does it plan to?
- Are people writing, talking about, and promoting this project and its technologies?
Awareness is a desired outcome of a project’s outreach efforts. Measured perhaps through user and contributor surveys or general marketing analyses, it refers to how well a target audience knows the project—the audience’s awareness of the project in general, and their understanding of the problems it solves more specifically.
- Are people in the project’s target audience aware of the project?
- Are people working in an industry that would benefit from the project aware that the project exists?
- Can people in the target audience explain the project’s uses, features, and advantages over alternatives?
No project lives in a vacuum. Projects frequently depend on one another. In some cases, similar projects can be competing for similar target audiences. A community’s interactions with other projects in its ecosystem reflect the project’s health.
- What are the project’s dependencies? And what projects depend on it?
- How well integrated is this community in the project’s overall ecosystem, the project’s target industry, organizations that may use the project, etc.?
- Do members of the ecosystem view this project favorably?