Where can I find requirements specification for Open Source software?

Question

I can hardly imagine a software project where the things to do are not somehow collected in a formal/informal document, be it a SRS or a simple note. So I was searching for these requirements documents for some well-known Open Source applications (e.g. Signal, GIMP, VLC, ...). As the Source is open, I guess engineering documentations can be as well, be it according to Waterfall, Agile or other methodologies. But I have not found any. So where are requirements captured for open source projects?

Issue/bug trackers is common. Look at github issues for any major repo, and you'll get examples of how open source projects manage this. Alternatively RFC processes are something you see in larger more formal projects. — user1937198, Jan 19 '21 at 10:51
One thing to realise is that unless the project is a major one, most open source projects run on far more informal lines that don't nessicarily fit into traditional methodologies and generates a lot less formal documentation. Instead informal communication channels and mutual understanding of the goal are the more common approach. — user1937198, Jan 19 '21 at 10:55
I would also challenge the premise of the question even for _private_ projects: while _initial_ requirements will probably have been captured somehow, and _required changes_ will have been tracked somehow, it's extremely common for these not to be archived in any consistent form, let alone updated to integrate changes. — IMSoP, Jan 19 '21 at 12:48
Forget private projects, my workplace (federal contract) doesn't even have this stuff. We were working on version 3 of a project when one of my coworkers asked me why X was done a certain, specific way in version 2... I have no idea, it was that way when I got here, so we went and asked another person (on another team now but fortunately in the same building as us)... "It was that way in version 1 and we weren't sure why so we didn't dare change it." Now we're working on version 4 and why change it now? (Version 1, ~1995, version 2 ~2008, version 3 ~2018, version 4 probably 2025...) — user3067860, Jan 19 '21 at 19:44
Some use JIRA. See [this question](https://opensource.stackexchange.com/questions/8346/opensource-projects-using-jira) for some examples — sevensevens, Jan 19 '21 at 19:48
@sevensevens to me this seems more relating to issue tracker rather than requirements management — kuma, Jan 19 '21 at 20:22
I remember years ago seeing the SRS for OpenOffice or LibreOffice available publicly though from a quick search I can't find it. — tschumann, Jan 20 '21 at 04:46
@kuma Using an issue tracker to track requirements is common both in open source and closed source projects. A very common approach (especially in agile projects, which tend not to document any more than needed) is to create a ticket when a new requirement is uncovered (with details of the requirements being added to the ticket), and track the progress by tracking tickets. — James_pic, Jan 20 '21 at 09:56
One of the four core values of agile is "Working software over comprehensive documentation." Another way to write this is "SRS? We don't need no stinking SRS!" — David Hammen, Jan 20 '21 at 12:08
Quite often the requirements spec for a new piece of free/open software X is "replacement for existing nonfree software Y"; the GNU project started out cloning System V UNIX (whose specification is pretty well written down in POSIX), and Linux started out by cloning MINIX (the book "Operating Systems Design and Implementation", Tanenbaum) — pjc50, Jan 20 '21 at 13:46
_"I can hardly imagine a software project where the things to do are not somehow collected in a formal/informal document, be it a SRS or a simple note."_ Hah! Oh my sweet summer child.. ;) — Asteroids With Wings, Jan 20 '21 at 14:34
_"I can hardly imagine a software project where the things to do are not somehow collected in a formal/informal document"_ I can hardly imagine an elephant doing a pom pom dance, but guess what I saw last weekend when going to the circus. Not being able to imagine something is in no way a justification for its lack of existence. — Flater, Jan 21 '21 at 10:45
@DavidHammen I understand mentioning "Software Requirement Specification" in the title was misleading. "Working software over comprehensive documentation", right. But I was not looking for comprehensive documentation. Uncomprehensive would be ok too. — kuma, Jan 22 '21 at 19:26
@Flater _"Not being able to imagine something is in no way a justification for its lack of existence"_. I can hardly imagine a philosophical comment stating the obvious on "software **engineering**" stackexchange, giving zero contribution and not addressing any point of the question. — kuma, Jan 22 '21 at 21:31
Many open source projects (e.g. [GCC](http://gcc.gnu.org/), [RefPerSys](http://refpersys.org/), [zsh](https://zsh.org/)....) have requirements provided by entities funding them. For [RefPerSys](http://refpersys.org/) contact me by email: `basile@starynkevitch.net` — Basile Starynkevitch, Dec 23 '21 at 16:48

score 36 · Answer 1 · answered Jan 19 '21 at 12:39

36

Requirements don't need to be captured in a formal software requirements specification or any other kind of document. They can be captured in tests (primarily acceptance tests and system tests, but perhaps also integration and unit tests), issues (feature requests, bug reports), discussions (both persistent like mailing lists or ephemeral like chat rooms), or by pointing to features in other software products and cloning them.

Depending on the project type and history, you'll see different ways to capture requirements. A project that was started by one or two people to solve a personal problem but then open-sourced is different than a project started by a large company to make money. Projects that are built to someone else's needs or specifications are different than products generally made available in the marketplace. Projects with a core, dedicated team that ultimately controls the product are different than projects that are maintained by one person or a small number of people who otherwise don't work together. These differences lead to differences across the methodologies used.

I'd also point out that the vast majority of methodologies aren't designed for open-source projects. They make key assumptions about the location, size, structure, working hours or allocation, and skills of the team members that may not be true for open-source projects, especially large open-source projects.

answered Jan 19 '21 at 12:39

Thomas Owens

79,623
18
192
283

"They can be captured in tests". A test in itself only shows *how* something works, but doesn't explain *why*. So I wouldn't really count those. Agreed on the rest though. – Voo Jan 20 '21 at 10:31
5

@Voo Consider other, more traditional forms of requirements, like shall statements and use cases. They don't explain why. They only describe the intended behavior or attributes of a system. – Thomas Owens Jan 20 '21 at 10:54
Ah yes fair enough. Not a fan of of shall statements either for the same reason, but yeah it's a way of doing them. Use cases can be fine, since there you get at least some understanding of the rationale behind the feature (if the use cases are well written). +1 in any case. – Voo Jan 20 '21 at 10:58
@Voo users often can't explain why but just tell you that if X happens then do Y. ie they only know how – mmmmmm Jan 20 '21 at 12:24
@mmmm The problem with that is, that if you don't understand the rationale behind some behavior it's impossible to safely adapt the feature. Figuring out the why is what makes one a subject matter expert and gets one paid better than the average user. Raymond Chen put it nicely as [Good advice comes with a rationale so you can tell when it becomes bad advice](https://devblogs.microsoft.com/oldnewthing/20091104-00/?p=16153). – Voo Jan 20 '21 at 13:05
@Voo Yes and it is the systems analysts/developers job to work that out from user requirements. The users often do not know the rationale, they just do the work every day. – mmmmmm Jan 20 '21 at 15:16
1

@ThomasOwens To be fair, I've seen plenty of projects experience significant delays specifically because people decided not to share the *why*. Add in a dash of "There's no way I didn't cover every use case" and a pinch of "Those stupid people in IT keep coming up with weird scenarios that will never happen, except a few times they actually did happen" and all of a sudden the *why* would have been really, really useful... This doesn't change your statement, you're still absolutely right, but is some food for thought. – corsiKa Jan 21 '21 at 04:45

Theraot · Accepted Answer · 2021-12-23T15:37:18.147

20

Don't let the software prevent you from seeing the projects.

I challenge the idea that you need requirements for a software. And suggest that requirements are for software projects. And that there are many software projects in the making of a software. Of those, only one was the inception, and that one is often the worst documented.

Alright, perhaps from a more philosophical point of view, software have requirements from the moment the idea of them is conceived in the mind. But you are not getting to see those requirements. For a practical stand point, there will be projects.

Free and Open Source software has multiple ways to advance. The software is the software (across all its versions and editions), but each change is a project. Each one of those projects has requirements.

Bug Reports.

There are people who test and use the software, and submit bug reports (issues). Fixing them becomes the requirement for the project of, well, fixing them. Sometimes a change is done to address multiple bugs, in essence collecting those issues as requirements for a single project.

Ah, but let us talk about adding features…

Road-maps.

It is common that a lead programmer will have a road-map of features to add. Which could be public for everybody, could be shared only among maintainers/core developers, or could be entirety private. The core developers will work towards the what is laid out in road-map. Thus, the road-map are requirements.

Proposals.

You will also find the issue system co-opted for feature requests. In fact, the practice of writing proposals for change as issues is becoming widespread. These proposals are usually more detailed than a road-map. You can consider them RFC documents.

The issues, in the issue tracker - be them bug reports or proposals for new features - are requests for change. They bring new requirements (or remind of old ones in the case of regressions). To implement those changes, we do projects (with design, and testings, programming, etc). To reiterate, each one of those projects has requirements.

TO-DO lists.

Some of us will have private, never seen by the public, TO-DO lists. Which can be as detailed as needed. Those can be considered requirements, but they never get to the repositories.

For example, I may turn an issue into a TO-DO list, work on it, and delete the TO-DO list when I'm done. Oh, and let me tell you about moments of inspiration. I've learned to portray these ideas quickly. Formalism may take too long. And the idea can be too large to implement quickly. I find the TO-DO list format useful in those cases.

Ah, by the way, some tools can pull TO-DO items from source code comments. Keep an eye for those.

Pull requests.

Contributions may come from outsiders. As third parties can fork the repository, modify the source, and then have those modifications incorporated into the official repository (with approval of its maintainers). This process is formalized as pull requests. How did they manage the requirements for those changes? Each developer does it their own way.

In fact, you may find common that they only express the requirements after the fact. Often to explain what the change does, in order to have it approved. And that can be dealt with in a conversational manner, instead of formal document.

Milestones.

Furthermore, you will find milestones. Usually corresponding to major releases. For a milestone a set of issues (bugs to fix, or features to add) is selected. You may consider all those as requirements for the next release.

Tests.

For some folks following TDD, the tests are the requirements. Even for those of us who don't follow TDD, having the code pass all tests is often a requirement for a new release. Thus, those are requirements expressed in the very formal language of actually executable code.

I remind you that automated tests, unlike requirement documents, are very good at telling you something is broken. Ah, yes, put writing tests in your DO-DO list, and require tests to have pull requests approved.

After all, both design and tests should come from the requirements. And if I don't have to do estimation, or have my project approved by management, or negotiate requirements with a client… I could start expressing my requirements in the form of automated tests, interfaces and other forms of code contracts. So I get to writing code faster.

I'll also remind you that many open source software come from a personal need. In many cases, the founders of these project started them because they needed the software, other times the founders really wanted to create the software out of passion.

And I'll remind you that not every developer is a software engineer. Not everybody needs - and is trained for - formality in methodology.

I hope it is not hard to imagine for you that the requirements for new born software can be "it helps me in my situation". At the start it could have no formal iterations or milestones, those thing you can added to the workflow later. At the start what matters is whatever or not the software is an improvement over nothing for whatever the founder is doing.

So, no, you are not going to find a requirements document for the entire software. However, you will find requirements for releases, or for particular changes. Don't let the software prevent you from seeing the projects.

edited Dec 23 '21 at 15:37

answered Jan 19 '21 at 12:44

Theraot

8,921
2
25
35

thanks, but you described how the software advances, I was more interested in how the software starts. I don't think all open source software projects started out as one or two individuals sharing a passion. There must be open source software projects started by a team, and where there's a team there's someone saying to a developer "the software shall do X" – kuma Jan 19 '21 at 20:31
7

@kuma Most actually do start by 1 or a few people working on something that interests them. The ones that don't are generally run by companies, and then they'll have the kind of things you're looking for but they may not be public- they may be hidden in whatever document repos the company uses. – Gabe Sechan Jan 19 '21 at 20:46
1

@kuma One or two people is very common. However, true, that is not always the case. Going over your examples… The odd one out is VLC. I found this: https://wiki.videolan.org/Documentation:History/ - Don't know how that worked. GIMP history is interesting, it was started by two people, and it is probably the closest to what you are looking for. See https://www.gimp.org/about/prehistory.html and https://www.gimp.org/about/ancient_history.html. – Theraot Jan 19 '21 at 20:54
@kuma I don't know much about Signal history, but if Wikipedia is to be believed, two people created the Whisper Systems company, which made privative products, and was acquired by twitter. One of the two co-founders merged technology from the products, got founding from one co-founder of WhatsApp, and called it Signal. We are not getting requirements documents of that one. – Theraot Jan 19 '21 at 20:54
2

@kuma Besides, I challenge the idea that you need requirements for a software. And suggest that requirements are for software *projects*. And that there are many software projects in the making of a software. Of those, only one was the inception, and that one is often the worse documented. You don't see fixing an issue as a project? How about change management? - The issues, in the issue tracker, those are requests for change, they bring new requirements (or remind of old ones in the case of regressions). To implement those changes, we do projects (with design, and testings, programming, etc). – Theraot Jan 19 '21 at 21:01
1

Without requirements, you have no idea whether some piece of software is correct or needs to exist at all. – Kaz Jan 19 '21 at 22:27
1

@Kaz you know of it needs to exist by the presence of a user base. And it's correctness by communicating with said user base. An SRS document is just one formal way of communicating with a representative of said user base. If the main developer is also a user, that can be the primary shaping force for an open source project. – user1937198 Jan 20 '21 at 00:17
@user1937198 Here is the thing. The software is written down. It's written down in excruciating detail. If the specification is not written down, that specification is at a **tremendous** disadvantage compared to the software. Whenever there is any doubt about the spec (which will be most of the time) whatever the software is doing will have to be deemed to be correct. – Kaz Jan 20 '21 at 03:19
@user1937198 ... But, oops, the software will contradict itself; someone will report that version 3 did something and to their dismay version 5 does something else. Both behaviors which will be a surprise to the developers who forgot about having coded either behavior. (Currently, they are busy working on breaking things in preparation for version 8.) – Kaz Jan 20 '21 at 03:21
4

@Kaz and that is a problem because? Correct is defined by what users want, not what's written down anywhere. If users are happy with how it works, great. If they aren't then they will communicate the unhappiness. So what the software is doing on a macro scale should be assumed to be correct by default in a popular open source project in the absence of any complaints. – user1937198 Jan 20 '21 at 03:25
If 3 and 5 contradict each other, do users complain about the change? If not either they prefer the new version, or don't care. Stick with the new behaviour and you are probably fine. If they do complain, then discuss with them which is now the better behaviour. – user1937198 Jan 20 '21 at 03:28
2

Proprietary projects do this too. Windows is full of special cases that are outside the documented requirements because it turns out major program X doesn't follow the documented interface. What matters is usability, not that it matches the document written years ago. – user1937198 Jan 20 '21 at 03:34
@user1937198 I think that your view of software is here restricted to just something interactive with features and behaviors discoverable by navigating around in a GUI. Writing such programs willy nilly without specifications is fine, until users get serious and use those programs to produce content, and the maintenance threatens its handling. – Kaz Jan 20 '21 at 04:29
3

@Kaz It is true that without requirement specifications the development can go awry. However, with requirement specifications it can also go awry. It will come down to the discipline of the developer and watchful eye of supervisors of the community at large to keep it line. Anyway, I'm not suggesting to develop software without requirements, but that often the initial effort - let's say proof of concept - does not have well documented requirements, at least not publicly. And that in FLOSS they often specify requirements using using issue trackers, tests and so on. Call it informal if you will. – Theraot Jan 20 '21 at 04:36
@Kaz the idea of many eyes watching over the code has been close to the core of the open source movement for a long time (don't know if from the beginning). And yes, that does not prevent bugs. It only helps in catching them after the fact. On the other hand, if version 3 was good, version 5 is bad, and the original developers are busy with version 8... Somebody fork that poor software already. Take over the development from version 3, cherry-pick what you need, and continue forward. – Theraot Jan 20 '21 at 04:41
@kaz not just guis. All software has its users. And software like GCC, shells or servers are as much defined by the edge cases that worked in the previous version. A formal list of requirements can never catch all of the ways people depend on the behaviour. – user1937198 Jan 20 '21 at 10:01
@kaz if it matters more, you are more cautious about any change in behaviour at all. – user1937198 Jan 20 '21 at 10:02
Great answer (though I’m a bit late to the party). Concerning the aspect of tests being the requirements: when you are designing your tests with the „Given, when, then“ method those statements constitute perfect requirements. Combined with appropriate coverage this set of requirements can be even considered complete. – Hartmut Braun Dec 28 '21 at 15:34

score 18 · Answer 3 · edited Dec 27 '21 at 21:29

Some good answers here already, especially for your question where requirements are found in Open Source projects. Let me say something about the reasons why is it hard to find a formal spec document for such projects.

SRSs make most sense in projects with an economical interest between two parties. Some customer / client / employer requires some software and may be willing to invest some time and money. But to make sure they will something in return which will have enough value to balance the invested money, they are asking a potential contractor or employee for a detailed description of what they will get in return, something which is detailed enough for making a contract or to justify the expected effort. That's typically the primary motivation behind creating a requirements specification.

In most Open Source projects, especially FOSS projects, the motivation behind the development is usually different. Most often, FOSS authors do not have the primary interest of selling their programs to someone else; otherwise, they would not disclose the whole source code. Often, they are trying to develop something for a domain they are interested in for themselves, maybe for educational purposes, maybe for solving a problem they are personally interested in, maybe for research purposes, for cooperation with others, or for building some reputation.

I think it is quite obvious that in the latter situation a detailed spec is way less important than in the former situation. That does not mean each commercial project will always have a detailed spec, or FOSS projects never have one, but it means that in FOSS projects chances are way lower to find a formal spec than in closed-source commercial projects.

To learn more about the typical motivations in FOSS projects, I would recommend you look into Eric S. Raymond's book "The Cathedral and the Bazaar".

score 0 · Answer 4 · answered Jan 20 '21 at 18:51

The main way is through comments in the code, and the readme, and 'issue trackers'.

Someone usually starts the project, on their own. They have an idea where it's going, and they build it, and release it. This is now 'the specification'. Any question you have is answered in any comments, any tips in the README or in the functioning of the software.

One of the big differences between humans and current 'computers' is that humans can easily form a model of what the software is doing and should do and could do. I have no 'study' to back this up, but 'it is known' that these models will be quite consistent between different humans, and the inconsistencies our handled by simply issues on the tracker, or by actual pull requests expressing a human's vision of the project.

In fact, the book Culture Code does study this and found that often for simple collaborative tasks like 'stack the spaghetti into the tallest possible tower, kindergarteners did BETTER than adult teams who were 'commandeered' by a forceful 'leader' or where everyone 'discussed strategies to achieve the highest possible tower'. The kindergarteners just kind of worked together and piled stuff on and tried things and within the time allowed got better results.

So this is how the projects begin, and then as they're more complex, they are not managed as a 'full specification' but rather as individual features where 'humans agree it fits the mental model of the software', and where comments in code and issue trackers are added where complexity demands things being specific.

Very few OSS projects grow to a point beyond where this is not practical, and for those who do, they typically just make roadmaps and milestones and discussion groups, and each dev specializes in the 'area' and so they are trusted with the details. It is in fact a tough task when a dev leaves, to 'pass on the mantle'. Typically the 'replacement dev' will simply study the existing code/functionality until they teach themselves the concepts, and so they're then likely to add more helpful comments and maybe notes in a README, and it goes on.

score 0 · Answer 5 · answered Jan 20 '21 at 18:57

The minimal requirements are complete code that compiles and can be built, producing the usable executable. This does not need to be the a script to run (while often is) but it should be enough documentation to make this manageable.

The "open source" project you cannot really build (with some code or build scripts intentionally removed) is theoretically possible if you are the sole owner of the code, but this is very much against the spirit and may bring no benefits in comparison to just having it proprietary.

score 0 · Answer 6 · answered Jan 20 '21 at 22:25

My two cents - Github Issues

I have seen various open-source software collecting their requirement through GitHub issues, or at least when anyone posts any issue, they create a new issue or wiki with the detailed requirement(and they can link to/mentioned in other issues as well, thus creating a link). I can't think/find of any such issue, to give you an example. But here's a list of some issues, which might give you an idea.

Kubernetes - Support port ranges in services #23864
Kubernetes - Allow RegExp or wildcard for ingress domain and/or path #41881 Open
Terraform - Storing sensitive values in state files #516
Kubernetes - Is sharing GPU to multiple containers feasible? #52757

Github Action - Add functionality for allow self-hosted runner to protect workflow file and allow only execution only for collaborators for PRS #494

score 0 · Answer 7 · answered Jan 21 '21 at 12:16

As other answers note, it's unusual for formal specifications to exist. But there are noticeable exceptions. For instance, GCC tries to implement the ISO C and C++ standards. These standards are written much like requirements, and typically distinguish between requirements and recommendations.

The catch here is that the requirements are not written (solely) by the GCC team.

Where can I find requirements specification for Open Source software?

7 Answers7