Open Source Sustainability in the Age of AI
More than ten years ago, I started SOS Open Source with a rather simple idea: before adopting an Open Source project, organizations should have a way to assess whether that project was likely to remain healthy and sustainable over time.
What started as a checklist gradually evolved into a methodology. We tried to quantify aspects such as release cadence, governance, licensing, community activity, responsiveness to issues, maintainer diversity, and many other indicators that, taken together, could provide a reasonably objective picture of a project’s health. Eventually that work became part of an EU-funded research project, which in turn led to OSSEval.
Like many EU research projects, however, OSSEval reached the end of its funded life and, with it, the website and documentation eventually disappeared. That is hardly unusual. Public funding is often excellent at creating new projects, much less so at maintaining them.
Looking back, I still believe the underlying methodology was sound.
What has changed is the cost of applying it.
Ten years ago, collecting the necessary information was by far the most expensive part of the exercise. Looking at mailing lists, issue trackers, release history, governance documents and community discussions required a considerable amount of manual work. Having a structured methodology helped make that effort repeatable.
Today, AI changes the equation.
A modern AI assistant can perform much of that investigative work in a matter of minutes. It can analyse release history, identify governance changes, compare activity across competing projects, highlight maintainer concentration, and often provide a fairly balanced picture of strengths and weaknesses.
Suppose you wanted to analyze Redis. You could ask an AI assistant to compare the project’s sustainability before the move from BSD licensing, during the SSPL period, and after the return to AGPLv3. In a few minutes you would likely obtain a structured comparison covering governance, community reaction, fork activity, release cadence and maintainer ecosystem. Ten years ago that same analysis would have required hours.
That does not mean AI has solved the problem, though.
The limitations are remarkably similar to those we encountered while designing SOS Open Source. AI can only analyze what it can see. If a project’s real activity happens inside a private Git repository, on Slack, Discord, Teams or other non-public communication channels, there is simply no evidence to analyze. Sometimes the assistant will acknowledge those blind spots. Sometimes it will not.
Human judgement therefore remains indispensable.
The difference is that our role has changed.
Instead of spending most of our time gathering information, we now spend most of our time validating it.
In practice, the questions themselves have become more important than the methodology behind them.
Rather than filling lengthy assessment forms, today I would simply ask an AI assistant questions such as:
How active is the project compared to similar alternatives?
How concentrated is development among maintainers?
How responsive is the project to security reports?
Have there been significant governance or licensing changes?
Are there signs of declining community participation?
What realistic alternatives exist should the project become unmaintained?
The answers are rarely perfect, but they provide an excellent starting point for further investigation.
There is, however, one recommendation I would add.
Never ask AI for opinions. Ask it for evidence.
Request sources. Ask where the conclusions come from. Challenge assumptions. Most importantly, understand what information may be missing because it simply isn’t publicly available.
Ironically, after spending years developing methodologies and tools for assessing Open Source sustainability, I have come to the conclusion that the methodology itself is no longer the difficult part.
The difficult part is asking good questions.
The future of Open Source sustainability assessment may therefore not lie in another specialized platform, but in teaching people how to interrogate increasingly capable AI systems while maintaining enough critical thinking to verify the answers they receive.
Open Source Clouds Comparison: Eucalyptus, OpeNebula and OpenStack
(Author: Davide Galletti) OSSEval (formerly SOS Open Source) has been used to analyze three different open source cloud solutions, namely Eucalyptus, OpenNebula and OpenStack, respectively maintained by the homonymous US-based company Eucalyptus recently acquired by HP, the Spanish C12G Labs company and the North-American OpenStack Foundation. »
SOS Open Source is Now Open Source!
We are glad to inform you that the European funded initiative PROSE eventually enhanced the SOS Open Source methodology, re-engineering the tools and making them available to the general public as open source. Now everyone can run an educated open source software procurement process by using SOS Open Source tools and methodology.
PROSE has been looking at a number of methodologies aimed at providing a solution for above mentioned issues, and they decided that SOS Open Source was a very valid starting point to make educated choices. SOS Open Source methodology and tools have been integrated within the opensourceplatform.eu under the OSSEval name.
IT Decision Makers now can perform procurement activities by using our methodology at their own premise, at no cost.
Unless you are going for the obvious names or your roots are deep in the open source world – either if you are a user interested in using open source or a maker willing to build software using it – you better take your time to ponder before selecting, and to try before buying or implementing.
