Using Open Source for security and privacy protection¶
Security and privacy are complicated things. This is why open development is a key factor and a precondition for creating secure solutions. Security and privacy is getting more important every day. Also due to the development of machine learning applications many data driven solutions are poisoned with privacy related data.
When development happens in the open, you can directly verify if a vendor is actively pursuing security and privacy and watch how it treats issues. The ability to study the process followed, the source code developed makes that anyone can perform an independent audit. Not only on code, but also on process used!
So beside code, open development means that an open processes is followed. A process where you can see and check whether mandatory baselines and principles are used.
To increase and improve security and protect our privacy open source solutions are more and more seen as a very good solution. Within more and more companies worldwide we notice a trends towards adopting open source solutions for security and privacy protection. Governments worldwide cannot depend and trust on closed source software for their security infrastructure anymore. Gartner predicts that by 2016 99% of Global 2000 enterprises will use open source in mission-critical software. So open source solutions for controlling security and privacy are slowly but steady becoming the new de facto standard. As many security experts already known: Transparency and openness increase security protection levels. However there is still a lot of resistance against using open source for business use, especially when it comes down to security and privacy functionality.
This chapter covers facts and demystifies fads regarding open source security and privacy products. When discussing the use of open source products for security and privacy services two important question appear:
Why should open source be used for security and privacy functionality?
How can the quality of open source products for security and privacy be determined and judged?
OSS quality is a very popular field for PhD students and analyst companies. However we think that also technical experience of practical business use along with deep technical knowledge is required in order to give a good advice for a company.
Of course we have an opinion regarding using open source security and privacy products for serious business use. However opinions are to be discussed and challenged. Always. Within the technical software field sometimes we tend to see things as hard facts. For examples bad written code. Many measurements exist to measure the quality of software code. However does this means that the product is totally useless? When it comes down to software code, all software contains bugs and has more or less quality issues. If you ask an auditor to look at software code, he will write an audit report with findings and recommendations. Always. If you are hungry and go to McDonald they recommend a very tasty bad solution that works temporary. In the end with every problem you face try to find out the real interest of your trusted advisor. Is he biased? Prepossessed to get a certain result? Always try to get a real in depended security or privacy advisor when it comes down to questions that relate to your vital business risks. Always challenge the advice! When it comes down to business related questions real facts are hard. Advice is always biased. However be warned for fads! Especially within the field of open source software for regular business use. For decades many vendors have created fads regarding open source. Since this message is repeated over and over again sometimes we are weak and store these fads as facts.
Some famous fads regarding open source the use for business use:
Open Source software is created by communist to destroy our world.
Open Source software is made by hobbyist.
Open Source software is made by hackers and hackers are bad. Especially when it comes down to security and privacy.
Open Source software is never maintained.
Open Source software is free, so it can not have any value.
Quality of Open Source software is dramatic. Do does hackers known how to spell quality at all?
Using Open Source makes you depended of the good will of hackers.
Using Open Source for security or privacy protection gives unacceptable high risk, since the whole world can hack me now instantaneously.
Using Open Source is an extra thread for my security or privacy.
Unfortunately, the list is endless long. Fighting fads is hard. Fads are most of the time a perception based on incorrect information. In this chapter we will not discuss these fads or other misunderstandings concerning OSS. However we will endorse you in this chapter with solid arguments that can help you when you are faced with fads regarding the use of open source solutions for security and privacy.
Some people are keen on ready to use list of good practices. However the context of security and privacy is very complex (organization, processes, people, technology). So we will not give a list of good practices. There are bad practices, but the list of good practices is almost unlimited, since the context for a random use case depends on various business aspects like:
IT Knowledge and experience present in an organization.
Information security knowledge present in an organization.
Maturity of IT within an organization. Especially with aspects like contracting, sourcing and IT operational management.
Legal and regulatory aspects.
Depending on the exact needs and problems of an organization the way quality aspects for security and privacy solutions should be approached differs.
The following sections of this chapter covers the following questions:
What is open source?
Why should open source products be used for security and privacy solutions?
What quality levels are needed for open source security and privacy products?
What aspects are important when selecting security or privacy products for a solution architecture or within use in an organization?
What is open Source Software (OSS)?¶
Before even considering using open source products for security and privacy applications, it is strongly recommended that a good solid knowledge exist what open source really is.
In brief open source software is computer software for which the source code is available.
More in depth it is recommended to read the full definition of open source as provided by Open Source Initiative (http://opensource.org/osd):
Free Redistribution: The license shall not restrict any party from selling or giving away the software as a component of an aggregate software distribution containing programs from several different sources. The license shall not require a royalty or other fee for such sale.
Source Code: The program must include source code, and must allow distribution in source code as well as compiled form. Where some form of a product is not distributed with source code, there must be a well-publicized means of obtaining the source code for no more than a reasonable reproduction cost preferably, downloading via the Internet without charge. The source code must be the preferred form in which a programmer would modify the program. Deliberately obfuscated source code is not allowed. Intermediate forms such as the output of a pre-processor or translator are not allowed.
Derived Works: The license must allow modifications and derived works, and must allow them to be distributed under the same terms as the license of the original software.
Integrity of The Author’s Source Code: The license may restrict source-code from being distributed in modified form only if the license allows the distribution of “patch files” with the source code for the purpose of modifying the program at build time. The license must explicitly permit distribution of software built from modified source code. The license may require derived works to carry a different name or version number from the original software.
No Discrimination Against Persons or Groups: The license must not discriminate against any person or group of persons.
No Discrimination Against Fields of Endeavour: The license must not restrict anyone from making use of the program in a specific field of endeavour. For example, it may not restrict the program from being used in a business, or from being used for genetic research.
Distribution of License: The rights attached to the program must apply to all to whom the program is redistributed without the need for execution of an additional license by those parties.
License Must Not Be Specific to a Product: The rights attached to the program must not depend on the program’s being part of a particular software distribution. If the program is extracted from that distribution and used or distributed within the terms of the program’s license, all parties to whom the program is redistributed should have the same rights as those that are granted in conjunction with the original software distribution.
License Must Not Restrict Other Software: The license must not place restrictions on other software that is distributed along with the licensed software. For example, the license must not insist that all other programs distributed on the same medium must be open-source software.
License Must Be Technology-Neutral: No provision of the license may be predicated on any individual technology or style of interface.
Reading this long definition can you make confused. Especially when you need a shorter definition to explain to senior management the benefits of what open source is all about.
Open source is based on three concepts:
A development methodology that defines a community approach to developing software, meritocracy of developers, and quality based on peer review.
A licensing approach that provides free access to source code, conforms to one or more “Open Source Initiative” licenses, and prioritizes the rights of users and committers.
A community of users and developers with open participation.
Currently open source software is software that is licensed under one of several accepted free software or open source licenses approved by the Open Source Initiative that:
do not restrict your ability to run the software, for any purpose,
provide one with access to the source code,
permit one to modify the software,
permit one to share verbatim copies of the software with others, and
under certain conditions, allow one to share one’s modifications with others.
“Open source software” is sometimes also called “Free software”, “libre software”, “Free/open source software (FOSS or F/OSS)”, and “Free/Libre/Open Source Software (FLOSS)”. The term “Free software” predates the term “open source software”, but the term “Free software” has been sometimes misinterpreted as meaning “no cost”, which is not the intended meaning in this context. (“Free” in “Free software” refers to freedom, not price.) So e.g. the free antivirus software AVG (http://www.avg.com ) is no OSS software. In September 2015 Security firm AVG announced it will sell search and browser history data of users to advertisers in order to “make money” from its free antivirus software. Due to the fact that AVG is no OSS software, users who care about their privacy have no other choice than to look for an alternative antivirus package. If AVG was OSS software, presumable a software fork was created.
“Free software” means software that respects users’ freedom and community. Roughly, it means that the users have the freedom to run, copy, distribute, study, change and improve the software. Thus, “free software” is a matter of liberty, not price.
The word “free” has many different meanings, and these different meanings often make it harder to understand OSS. The term “Free software” (as used in literature) is based on the word “freedom” (the word “libre” is used in some other languages). However, “free” can also mean “no cost”, and sometimes “no cost” products come with a “catch” that in fact is the opposite of freedom. A catch everyone in the IT knows as vendor lock in or (unhealthy) dependency.
To understand the concept of free, one should think of “free” as in “free speech,” not as in “free beer”. Sometimes OSS is called ‘libre software’ to show we do not mean it is gratis. A LinuxToday posting found a simple way to express these different meanings of the word free, which I’ll slightly paraphrase here:
Free can mean various things:
Free, as in free speech.
Free, as in free beer.
Free, as no cost.
Free, as high on drugs
They are not all the same.
Free software(FOSS): Richard Stallman’s Free Software Definition, adopted by the Free Software Foundation (FSF), defines free software as a matter of liberty, not price.
So summarized: Open source software (OSS) has nothing to do with no cost or no value. The initial cost structure for acquiring OSS based solutions is different. A license fee for the software use is absent. However to keep your solution supported by a vendor most companies pay a regular maintained fee to keep quality ask risk as low as possible. This is equal as with closed software solutions.
The power of open for security and privacy¶
To make improve security and privacy within digital worlds a number of aspects are of crucial importance:
Open collaboration: This means that everyone can reuse and/or improve security and privacy related material (e.g. documentation).
Use of open solutions: This means the application of OSS products for more and more security and privacy services. Many papers and books are written of the business advantage of using OSS software. When it comes down to security the main principle to go for OSS is openness. Using open solutions makes the solutions in the end more resistant against vulnerabilities. In the end it is about transparent facts and quality criteria everyone can evaluate if needed. With closed source solution validation of quality statements is often not for all stakeholders possible. Think about the use of simple encryption software: We have more trust in an open encryption solutions that one that is claimed by a company that is unbreakable.
Learn from each other and from our mistakes. People make mistakes. We make bad designs that increases security problems instead of solving them. OSS projects are not always managed as they should be when they produce critical security software. Learning in an open collaborative way without any direct or indirect commercial interest is crucial to get security and privacy aspects in IT where they should be: Just some quality criteria within the whole range of important aspects. In future the emphasis on security and privacy is equal as on safety, usability and business continuity. Currently only for safety aspects mandatory policies exist for companies to prevent people dying from software bugs. But today security and privacy aspects are not handled in the same way as safety aspects. A different approach is taken when it comes to designing IT systems on which human lives depend compared to designing information and privacy aspects in (business) information systems.
Openness. Full Transparency is needed when it comes to privacy. OSS and especially OSS software that can be hosted on premise is well positioned to ensure privacy when it comes to your digital footprints. Free Software is probably the only way to ensure that.
Improvements will not come overnight and a paradigm shift is needed for many companies to be more open and transparent regarding their security and privacy designs. Since privacy data is a core asset of customers of all companies, in future customers will demand a full transparent view on how a company protects the value assets given by customers.
Open security can be defined as an approach to use existing open knowledge in combination with the application of open source software (OSS) to help solve cyber security problems. OSS approaches collaboratively develop and maintain intellectual works (including software and documentation) by enabling us to use them for any purpose, as well as study, create, change, and redistribute them (in whole or in part).
Cyber security problems are created by starting with bad architecture or design or simply by a lack of knowledge and experience. Using an open security approach the security can be improved through collaboration.
So why use open source software for security and privacy applications? Open source software provides additional trust by allowing people to look into the source code whereas good OSS projects are completely transparent on all their SDLC and quality processes. When using OSS adjustments or improvements are easily made providing you with a flexible solution for your business.
Summary: Open source for use in the field of security and privacy means easy reuse (code or ideas), to improve what is already there. Reuse would be in a way so everyone can benefit. That way the quality gets better and better.
Determining quality of OSS for security and privacy applications¶
What quality really is or not has been a long running debate in many (scientific) management books. So it is only logic that quality in open source has been also a long running debate. However the fads regarding OSS made these discussions even harder to get a clear view on what should be defined as quality in relation to OSS security and privacy products. If you are planning to join these discussions, we would like to warn you to beware that these discussions are biased with many fads and unproven facts. Also many opinions in this field take an almost religious turn. General statements and general discussion seldom lead to weighted balanced judgment. IT for business use or security is not only the field of scientific computer science. Social sciences play a great role within IT security and privacy (think of the many awareness campaigns), and the field of risk management is not only the field of statisticians and mathematicians, but also psychology plays a role.
In essence the definition of quality and good OSS quality largely depends on the goal and context of the specific use case.
Quality and trust are for security and privacy products one of the most important aspects. This section will give guidelines on how quality of open source software for security and privacy can be easily measured and judged depending on your goal and use case.
Determination of the quality of security and privacy for a specific use case is complex. Besides an approved OSS licensed (see http://opensource.org) an OSS security products requires far more quality aspects. A license alone is not enough. This section describes a checklist to assist in evaluating the quality of an OSS products targeted on security and privacy. OSS products should always be evaluated on quality before use for real. But security and privacy OSS products have the following points that make evaluating a bit different:
Security (Unfortunately many security products are insecure and require insecure configuration to be usable!)
Maintenance. Due to the SSL Heartbleed bug (http://heartbleed.com/) maintenance of OSS security products has grown in importance.
Safety aspects can be compromised if security and privacy aspects are not handled well. Recent examples are car-hacking and plane-hacking. Due to security flaws, the safety can be compromised if intruders get into a system. Also personal safety (where do people live that …) can be harmed if for example web shops are sloppy with personal data and order records. Criminals like list of persons who buy very expensive paintings online.
The use of Open Source Software (OSS) components is a viable alternative to Commercial Off-The-Shelf (COTS) security and privacy components. Since the quality of OSS products varies widely, both industry and the research community have reported several OSS evaluation methods that are tailored to the specific characteristics of OSS. We have performed a systematic identification and evaluation of many of these methods, and present in this section the factors that really make sense with respects to:
The endless types of context specific organizations that potentially use OSS security and privacy products.
Protect (very)small and large security and privacy OSS projects who have very high product quality, but score less on (visible) process quality aspects.
The variety in which security and privacy OSS products can be used within a SDLC.
The latest and most promising project for potential users to get a fast insight in OSS security projects is the “Core Infrastructure Initiative Best Practices Badge” project of the linuxfoundation.org. Badge will hopefully give in future some indication on some quality aspects regarding OSS security products. However the badges project has a specific scope and not all value reusable OSS software and projects are able to gain a badge. But also if an OSS has a badge, it still is important to evaluate the use and risks for your use case.
A good security and privacy product should at least be evaluated on:
Product quality aspects;
Process quality aspects and
Quality control system used to preserve product and process quality
In order to cut the complexity and not write endless notes on what quality is and how it can be measured we will focus in this section on given ready-to-use evaluation criteria. Use, reuse , or improve them. We will also try to collaborate with the badges project and similar OWASP projects to get one open evaluation list in future that is easy to use.
Note that some evaluation criteria are more important than others, but since quality is always context related evaluating the many different aspects further in depth should be done in a context specific solution architecture, not in this (general) reference architecture.
To keep things organized we use:
ISO 25010 standard for software product quality (successor of the ISO 9126 standard)
ISO/IEC15288 System Lifecycle Processes.
Note that ISO 25010 lacks attention for aspects like:
Compliance (e.g. with laws, standards) requirements
Documentation, Support and Training requirements
To overcome these aspects, we use our security and privacy principles in order to get an in-depth list of criteria that can be used for evaluation.
The following evaluation model is used:
Our main goal is to present in this reference architecture a list of evaluation criteria as simple as possible. So we enriched the criteria with simple (example) questions.
In the following paragraph key questions are given that can be used to evaluate an OSS security or privacy application for your use case.
Architecture and design¶
OSS projects that produce security or privacy software, solutions, libraries etc. should have:
Make reuse of e.g. good security and privacy standards to avoid reinventing the wheel.
Readable architecture or design. So also people who are not programmers can understand the design. At least all design decision should be documented.
Unfortunately good security or privacy architectures and designs are rare for IT projects. This does not only account to large governmental projects, but also for large OSS security projects. Mind also that a big OSS security or privacy project can mean different things:
Large number of users of a product or
Enormous amount of source code
Significant number of full time maintainers (over 10 is already a huge amount)
Enormous number of contributors to a project.
E.g. the OpenSSL project has many users worldwide, however since the number of active project members was dramatically small, large is no guarantee for sustainable good quality.
When using OSS software you must have a strategy and a process that handles the maintenance of the software. Maintenance is essential for security and privacy related software products.
Maintenance has many aspects. For a healthy OSS security and privacy application you can divide maintenance in:
Maintenance on the OSS software product itself;
Maintenance on the quality system built around the eco system (processes, organization, financial s, control procedures, contributors and maintainers stability, etc.)
Maintenance process required for using the product.
Since this section only covers guidelines for evaluation of quality aspects of OSS security and privacy products we will only deal with the maintenance aspects directly related to the OSS product and organization surrounding it. But please beware: The maintenance required to be organized by you or your organization can differ significantly per OSS product. Some OSS security and privacy projects are aimed at making maintenance processes needed within your organization as simple as possible where other projects require more effort. Critical evaluation questions are:
Is there a transparent way for (new)requirements adoption?
Is there a strict change management process?
Is there a tough release scheme? (A release early, release often (sometimes abbreviated RERO) approach). E.g. every month a new release.
Is there a stable release and an alfa or beta release with new features?
Is there an active community of developers?
Are security vulnerabilities fixed in a structured way?
Is there a source code release and a binary code release?
What is the frequency of updates for the OSS project?
Does the project use a build system that can automatically rebuild the software?
Is there an automated test suite available?
Are new tests always added for new functionality? (E.g. due to a internal policy?)
Maintainability plays a special role for open source cryptographic software algorithms. Cryptographic software requires next to excellent programming skills deep knowledge of cryptography. To be able to maintain cryptographic software finding the right resources is very hard. Within the security principle section some principles can be found that relate to quality aspects formulated for cryptographic software.
Whenever you use an OSS security or privacy product you rely on protection or functionality. Reliability is a core aspect when evaluating OSS security and privacy products. Critical evaluation questions for reliability are:
Is there an automatic test suite for the product?
Does the testing methodology include (automatic) regression tests?
Are interfaces with other products and platforms tested?
Is there a written test plan along with documented test results?
Are test reports published on the website?
Is the software tested (when relevant) against OWASP top 10 vulnerabilities?
Is the OWASP Application Security Verification Standard (ASVS) met?
Is there a public accessible defects database?
Is there a process organized around defects management?
Is there a standard procedure followed before release new software in stable versions?
Is the quality process documented?
Is an endurance test under stress load performed with the public released version? Is this test public so everyone can (re)use it? (note: Not for all applications relevant)
Has the project a website with a static URL?
Is it clear who are project members, contributors and committers?
Does a written procedure exist on how one can get commit rights on the core repository?
Is there a public audit log available of changes on the core repository? (Subversion, Git and many other SSCM systems provide this crucial feature.)
Are the SANS Securing Web Application Technologies (SWAT) criteria met?
When using a FOSS product you must verify if it is secure. Security is of course about trust, but when you use OSS security and privacy tools you must evaluate some crucial security aspects.
One of the promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. However, whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into ostensibly secure software during the various compilation and distribution processes.
As with commercial software, also FOSS security products exist that decrease your security. Only the advantage of FOSS is that inspection and fixing can be easier. Software that requires insecure configurations for example or many nonstandard network sockets is not a good example of decent security.
Even if you are only testing a product or evaluating, you must have some criteria in place to prevent downloading malware or worse.
Some critical questions to determine some security aspects are:
Are security vulnerabilities fixed according to a described process?
Does the project have its own security officer or security team?
Does a procedure exist and is it followed for performing a static and dynamic security code review on every major release? Are results of the secure code reviews available?
A dynamic analysis tool for the code is used before releasing a major version (e.g. the project may use a fuzzing tool (e.g., American Fuzzy Lop) or a web application scanner (e.g., OWASP ZAP or w3af).
What kind of socket connections and protocols have been used? Standard sockets connections (22,443,80,445) and standard protocols used (e.g. HTTPS, SSH, SSL, LDAP, LDAPs )?
Are product vulnerabilities mentioned in the CVE database?
Is it clear how many vulnerabilities (open and fixed) are mentioned in the CVE database? (Use the http://web.nvd.nist.gov/view/vuln/search?execution=e2s1 ) and search on product name. Note that vulnerabilities can be reported on the core product, but also on additional contributed modules if you are investigating a large OSS project.
Is there a process to deal with vulnerabilities (e.g. release of fixed/patches in a controlled manner).
Has the project created its own cryptographic libraries? Note that writing cryptographic algorithms is very hard and should be prevented by using already available good OSS algorithms.
If cryptographic protocols or libraries are used, have these algorithms been published and reviewed by experts?
Security and privacy principles and requirements are defined for the project and within the design and implementation it is clear how these are covered.
All vulnerabilities are reported on the project site and are accessible without limitation by the public.
It is clearly documented what process must be followed to obtain change rights on the main software repository.
Procedures and policies exist to protect the code base from vandalism.
Is a software release signed by a hash (minimal sha1 or stronger)?
Are reproducible builds available? (https://reproducible-builds.org/)
When you use an OSS security or privacy product you should not be required to register your name and organization in a database if it only serves a marketing purpose. All OSS licenses are very clear on what is allowed regarding distribution. People may sell OSS software. Even the GPL allows this. But since privacy aspects are becoming more and more important you must be aware on critical aspects that can harm your privacy when using OSS security or privacy tools.
Some critical questions to determine and evaluate privacy aspects are:
Tracking cookies and other finger printing techniques are not used on the project core community website.
The OSS security and privacy project respects the privacy of its users and contributors in all possible ways.
Project maintainers and contributors are allowed to participate under an alias since not all governments allow working on OSS privacy or security products.
The project is clear on measurements for handling contributors’ personal identifiable data.
Privacy of users or companies using the product is neither exposed nor stored.
No privacy related data is stored and used by the project.
There can be no progress without change and if change is not taking place the bit rot will start. For security and privacy OSS projects some change control LCM aspects are of crucial importance. To make implementation of changes easy more and more projects enable an automatic update service that automatically implements changes on all running software instances. However implementing such a mechanism requires a very high level of internal change and governance processes.
Some questions to determine and evaluate change control aspects are:
Has the project a version-controlled source repository that is publicly readable?
Does the OSS project uses tools and principles to make builds reproducible? preferred is that the OSS software is build using the https://reproducible-builds.org/ standards.
Is issue tracking for defects in place? (For reporting bugs or feature request).
Is tracking of requirements or enhancement on requirements request in place?
Does the project release software with unique version numbering?
Is a change log in human readable format for each release available?
Does a clear documented SLCM process for the project exist?
Is it clear how automatically built CI environments are configured and maintained?
Does the change control process allow roll backs of releases?
Whilst anyone can inspect the source code of free and open source software for malicious flaws, most software is distributed pre-compiled with no method to confirm whether they correspond. So for change control it is crucial that software builds are fully reproducible. Creating real reproducible software build is a complicated tasks. However the tools and infrastructure offered by the https://reproducible-builds.org/ makes this transparent and a bit simpler.
Some questions to determine and evaluate documentation aspects to investigate the quality of an OSS security and privacy application are:
Is documentation for new developers available for free on the website?
Is the source code documented?
Is documentation maintained?
Does a structured written procedure exist on how the documentation is maintained?
Are documentation processes embedded in the CI pipeline?
Are the user manuals provided by the project?
Is it directly clear what the status of the documentation is? Programmers usually do not write the user documentation. But it is crucial that the documentation keeps in sync with every release.
Are there (many) books (besides the one published by the project itself) available?
Is commercial documentation available (e.g. books on Amazone)?
Can everyone participate in improving the documentation?
Is the documentation published under a Creative Commons licenses (CC) license?
All solid OSS security and privacy projects have a strong and stable community. By evaluating community aspects one can get an indication on how the project deals with all kind of crucial quality aspects on product level and on process level. A community does not have to be large and very active. Many good security projects exist with 2-3 community members who manage to perform all crucial processes on a periodic basis. Stability is often more important than size. An OSS project that has too many forks can be an indication of a strong vision of the leader or a lack of leadership on dealing with crucial issues regarding the project health. A fork is most of the time a good sign. It means the software is used in many different ways and some people are building other communities to support their future vision for the project. But some research on why a project is forked should be done when you are evaluating OSS security products that offer exact the same functionality and share the same code base.
Some questions that can assist you in evaluation community related aspects:
How big is the community of core developers?
Is the process of joining the OSS project transparent?
Is it clear how one can become a code submitter?
Is the process around the core community open and transparent?
Are commercial books of the project available?
What is the number of available commercial books of the project?
Are many books available? (E.g. on amazone.com or O’Reilly)
Are mailing lists of the core developers open and transparent?
Is it clear how decisions are made within the project?
Can everyone attend to all project discussions (e.g. mailing list, slack channels, IRC)?
Average number of people active on IRC or slack?
The project has a written policy to stay active and healthy (e.g. the C4, see zmq)
Using OSS security or privacy software is always done in a specific context. You already have other software building blocks, you need your own reports, or you want to use another identity manager to use the product. Integration aspects on business and technical level are crucial for healthy OSS security and privacy projects. Too often projects fall victim to scope creep and are creating a one-size-fits-all solution. Logging, auditing and encryption e.g. are services are a world of themselves. The same goes for great responsive GUI’s. You cannot create an excellent CMS when you are focusing on a dedicated security or privacy function.
Some questions that can help you evaluate integration aspects of OSS security and privacy products:
Can the software easily be integrated with non OSS or other OSS projects?
Does the software allow an easy way to extend its functionality?
Is the software modular built?
Are REST interfaces available?
Are all interfaces for external use stateless by design?
Are API’s well documented?
Does the OSS license have impact on building your own library or module against the core product? E.g. the GPL is very clear on integration.
Is it clear how security or privacy aspects are impacted when third party integration modules are used?
Every organization using OSS security and privacy products sooner or later needs some professional support to maintain the product, to adjust configuration settings or to implement new versions. Within many businesses support on software is crucial and it is often written down in lengthy support contracts with many sentences that must make clear what kind of support is requested. In general, when you have a good relationship with a company that supports some (OSS) software for you, the contract should be based more on trust. Lengthy contracts are usually the result of little confidence or expensive mistakes made in the past. The great advantage of using OSS security and privacy products is that you can be very flexible in how you organize crucial support issues for a product. Of course when you rebuild the product it is hard to find people who can easily resolve problems. Some OSS security and privacy products have a commercial version for which you can get paid support. But when the commercial version differs from the OSS version you are not dealing with a healthy OSS project any more.
A large and well known OSS security and privacy project has many excellent people within the community who are willing to provide support.
Some questions that help you evaluate support aspects regarding OSS security and privacy products:
Is paid support possible?
Is there a strong community support?
Can questions on usage, configuration or problems be posted somewhere?
Has the project an active open forum or mailing list for support questions?
Does a mailing list exist for paid support or contracting work corporate users of the product?
Is it possible to contact one of the core developer(s) working on the product directly (e.g. email?)
Security and privacy application can have many legal aspects. This applies not only to the usage, but also to the possession and creation of security and privacy related software. Many governments suspect people who use encryption software for private use. In some countries the use of privacy protection tools is prohibited. When using OSS security and privacy products it can be crucial to evaluate the legal aspects first, before using the product. Many security or privacy OSS products are great tools for criminals. This cannot be avoided. When someone uses a tablet to smash people on the head Apple cannot be accused of creating a murder weapon. However responsible projects are aware of possible trivial misuse.
Some questions that can help you evaluate legal aspects of OSS security and privacy products:
Which OSS license is used?
Is the license approved by the OSI foundation?
Is the OSS license a widely used license?
All functionality must meet the **Open Standards Requirement for Software** by the Open Source Initiative
Is the OSS project aware of any possible misuse of the product? E.g. does a notice exist on what the intention for correct usage is of the product?
Can you be held responsible for damage or lawbreaking when you use the product on the open internet? Does the project warn you for this kind of aspects?