Talk:CII Badge Program Checklist
Regarding the questions in those links:
The project MUST have a general policy (formal or not) that as major new functionality is added, tests of that functionality SHOULD be added to an automated test suite.
Ideally there should be tests for all functionality that gets shipped in a release. Many products has been exploited in the past through the use of unsupported features that were present in shipped code so we are seeking a general policy that all new code should have some tests included.
The project MUST have evidence that such tests are being added in the most recent major changes to the project.
If your tests are grouped together by the feature set being tested and the new features ar tracked in a change log then this should sufficient, at least for the current badge criteria.
It is SUGGESTED that this policy on adding tests be documented in the instructions for change proposals.
If the policy you discussed was adopted then that would be great.
It is SUGGESTED that at least one dynamic analysis tool be applied to any proposed major production release of the software before its release. I am not quite sure about this: aka does a fuzzer count here? If it does we can argue that XenRT being run on RC's is sufficient
Yes, currently a fuzzer counts as dynamic analysis. We might have a more strict definition in the future.
It is SUGGESTED that if the software is application-level software written using a memory-unsafe language (e.g., C or C++) then at least one dynamic tool (e.g., a fuzzer or web application scanner) be routinely used with a mechanism to detect memory safety problems such as buffer overwrites. I am not quite sure about this: what does routinely mean? Does running it on RCs qualify?
Running on RCs is fine. Following up on the output of any sort of dynamic analysis tool, and especially fuzz testers, can be very time consuming so we are not looking for this to be done on nightly builds. Testing on RCs so that new functions are tested before final release, with time to fix things that get found, is a good solution.
