Open Source Sustainability in the Age of AI

More than ten years ago, I started SOS Open Source with a rather simple idea: before adopting an Open Source project, organizations should have a way to assess whether that project was likely to remain healthy and sustainable over time.

What started as a checklist gradually evolved into a methodology. We tried to quantify aspects such as release cadence, governance, licensing, community activity, responsiveness to issues, maintainer diversity, and many other indicators that, taken together, could provide a reasonably objective picture of a project’s health. Eventually that work became part of an EU-funded research project, which in turn led to OSSEval.

Like many EU research projects, however, OSSEval reached the end of its funded life and, with it, the website and documentation eventually disappeared. That is hardly unusual. Public funding is often excellent at creating new projects, much less so at maintaining them.

Looking back, I still believe the underlying methodology was sound.

What has changed is the cost of applying it.

Ten years ago, collecting the necessary information was by far the most expensive part of the exercise. Looking at mailing lists, issue trackers, release history, governance documents and community discussions required a considerable amount of manual work. Having a structured methodology helped make that effort repeatable.

Today, AI changes the equation.

A modern AI assistant can perform much of that investigative work in a matter of minutes. It can analyse release history, identify governance changes, compare activity across competing projects, highlight maintainer concentration, and often provide a fairly balanced picture of strengths and weaknesses.

Suppose you wanted to analyze Redis. You could ask an AI assistant to compare the project’s sustainability before the move from BSD licensing, during the SSPL period, and after the return to AGPLv3. In a few minutes you would likely obtain a structured comparison covering governance, community reaction, fork activity, release cadence and maintainer ecosystem. Ten years ago that same analysis would have required hours.

That does not mean AI has solved the problem, though.

The limitations are remarkably similar to those we encountered while designing SOS Open Source. AI can only analyze what it can see. If a project’s real activity happens inside a private Git repository, on Slack, Discord, Teams or other non-public communication channels, there is simply no evidence to analyze. Sometimes the assistant will acknowledge those blind spots. Sometimes it will not.

Human judgement therefore remains indispensable.

The difference is that our role has changed.

Instead of spending most of our time gathering information, we now spend most of our time validating it.

In practice, the questions themselves have become more important than the methodology behind them.

Rather than filling lengthy assessment forms, today I would simply ask an AI assistant questions such as:

How active is the project compared to similar alternatives?
How concentrated is development among maintainers?
How responsive is the project to security reports?
Have there been significant governance or licensing changes?
Are there signs of declining community participation?
What realistic alternatives exist should the project become unmaintained?

The answers are rarely perfect, but they provide an excellent starting point for further investigation.

There is, however, one recommendation I would add.

Never ask AI for opinions. Ask it for evidence.

Request sources. Ask where the conclusions come from. Challenge assumptions. Most importantly, understand what information may be missing because it simply isn’t publicly available.

Ironically, after spending years developing methodologies and tools for assessing Open Source sustainability, I have come to the conclusion that the methodology itself is no longer the difficult part.

The difficult part is asking good questions.

The future of Open Source sustainability assessment may therefore not lie in another specialized platform, but in teaching people how to interrogate increasingly capable AI systems while maintaining enough critical thinking to verify the answers they receive.