Re: ISO 9000-3 meets the Bazaar

[ Follow Ups ] [ Post Followup ] [ Discussions at the Paneris philosophy node ]

Re: ISO 9000-3 meets the Bazaar

Posted by William Chesters on Thu Sep 10 5:08:48 1998

In Reply to: ISO 9000-3 meets the Bazaar posted by TimP on August 29, 1998 at 15:30:40:

Distribution:	paneris@i-way.co.uk	lurker@paneris.co.uk

There's an important point about CVS buried in here! See if you can
find it ...
multiboard@paneris.co.uk writes:
> New message at http://www.paneris.co.uk/philosophy/messages/4.htm
>
> Subject: ISO 9000-3 meets the Bazaar
> From: TimP (tim@paneris.co.uk) (194.112.47.112)
>
> The combination of the Bazaar methodology with
> automated quality assuance seems to me to be unbeatable.
>
> As you know I am very keen on ISO9000-3 registration and
> would be interested in any thoughts you have on obtaining
> it.
I don't know what 9000-3 is, but if it's anything like 9000-2 then the
big problem is the outlay. The "examiners" do not come cheap.
Otherwise all they are looking for is a sensible-looking system which
provides things like this:

having designated roles for everyone, including someone in charge
specification, reviewed at least once and signed by customer and
project manager

requirements spec: what it's for

functional spec: how it seems to the user

project plan, reviewed at least once signed by project manager

resources (people, plant, euros, ...) needed for development

one of those pretty timetable/dependency charts :)

risk analysis

test plan

acceptance plan

documentation on the actual architecture and code, reviewed and
signed
versioning/build system with identifiable and recoverable
releases

and system for making backups, with someone responsible,
checklists, signatures ...

fault report/feature request system
change notice

each generated in response to a fault/feature request

contains a description of what files will be changed and how:
all work has to be justified/explained in this way

signed off by project manager

when the files concerned are checked out of the version
control system, a pointer is left to the change notice

checklisted procedure for building a release, since that can
be quite complicated
testing results of each proposed release, signed by test person
change document for each release, signed by project manager

summary of change notices made since last release

procedure for ensuring that all interested parties get the
change document drawn to their attention

acceptance of each tested release, signed by customer
(go back to fault report/feature request generation)
In addition you need a detailed description of the whole process, all
the documents and all stages (inputs/outputs from each)---and this
description must itself be under version control and subject to
periodic revision.
Signatures (or their computerised equivalent) are great because they
focus responsibility; things mustn't be allowed to fall into the gaps.
Checklists (or their computerised workflow equivalent) are great
because they provide a simple net for catching some of the more stupid
mistakes.
Authorisation and justification of every piece of work is great
because it provides an audit trail in case things go wrong, and helps
stop the classic mistakes of duplicating work, messing other people's
work up, and doing work which is not really needed in the first place.
> Some jobs will not require the full weight of QA, so our QA system
> must be scalable to the job
A system like this is in fact surprisingly unintrusive, because you
can adjust the granularity of the records over a wide range to suit
the conditions. Generally if you only have a very few people working
on the code then "change notice CN6782: add spellcheck feature; will
modify [27 files] in whatever way seems appropriate", followed by
"release 1.4: spellcheck feature; see CN6782" will do fine.
Some projects even start and end with "release 1.0: initial release;
see CN0001".
Also it's perfectly OK to do things in a different order from what the
above suggests. For instance you can go away and make changes to your
local copies of source files in response to a bug report, then make up
a change notice and get it signed off, check out officially and
immediately check your changes back in. Of course, you have to be
confident that your project manager will agree with your idea and
hence pay you ...
Yes this means lying and forging the documentation :).
> Get the Bazaar methodology working
> ==================================
The Bazaar methodology interacts badly with serious QA in two ways:
firstly, all this signing-off and reviewing requires constant
communication (I don't know what to about that except give everyone a
mobile); and secondly, the version control system can become a
bottleneck.
Basically the whole check out/check in procedure becomes an enormous
pain, because the source file is generally too coarse-grained a unit
to support multiple developers working on multiple problems. Everyone
wants to check everything out and the result is chaos.
That is why virtually all the well-known bazaar projects (Wine, Gnome,
KDE, egcs, Linux kernel, ...), and many traditional software shops,
use CVS --- http://www.loria.fr/~molli/cvs-index.html .
CVS doesn't (by default) bother with locking your sources. Instead,
if Mr X tries to commit changes on a file which Mr Y has in the
meantime already committed changes of his own, CVS uses `diff' to show
X what Y has done; 95% of the time it is obvious that nothing bad will
happen and X can leave CVS to merge the two sets of changes
automatically. Otherwise X has to fix up his changes to fit with Ys.
The CVS client and, I think, the server run on both Win32 and Unix
over TCP (e.g. Internet).
One of the nice features is that you can give everyone read access,
e.g. for code readthroughs, while restricting write access to active
developers. Of course the aforementioned projects still run into
problems with those developers messing things up because of
misunderstandings---which is why a commercial cooperative would do
better not to trust them quite as much, and instead insist that they
go through a proper change notice procedure.
Another splendid thing about CVS is that because it was originally
just a collection of shell scripts wrapped around GNU RCS, it is still
extremely extensible and customisable and could be integrated into
whatever project system one wanted.
> To test our products we need as many people as possible,
> on as many platforms as possible, to visit a site and record
> problems and what platforms used.
You need to write up beforehand, in what you agree with the customer,
what exactly is going to be tested and on what platforms.
> For code reviews things get a bit trickier.
> Code Reviews have a number of functions.
> They are intended to give the customer and ISO an warm feeling,
Hmmm, maybe the QA people are generally more interested in
specification and testing---unless you are talking about
safety-critical code in which case they want to see code reviews by a
separate third party.
> but their real import is education and cohesion.
Yes, they are a fantastic way of *achieving* good results which pass
the QA tests whose function is to *demonstrate* that you have achieved
a quality product.
> This is traditionally achieved by having a 'house style' enforced
> by reviews and a little automated style checking.
Just as long as the style points are left very general. I don't think
there's any point in trying to enforce particular indentation rules on
people, or silly complicated naming conventions (except pseudo-package
prefixes in languages that lack real packages, like C).
> Finally, of course, the major problem with our Bazaar at the moment is that
> very few people are prepared to use the Message Boards for emails.
Run a news server and gateway it to/from a set of mailing lists.
That's the only way you will get people to really use it.
> We should have a mechanism by which scripts/programs can be
> held at the project node,
This was the original idea behind the doc system ...
As far as supporting a work structure like this is concerned ... I
think it's sufficiently complicated that it would be worth
implementing it as a set of high-level rules, maybe even diagrams.
It will obviously have similarities with expensive products from
Rational, as well as with humbler things like Notes, and there are
bound to be homebrewed things already out there.
Finally, it's incredibly important that the system should be flexible.
It should be obvious how to short-circuit it. This is easy to achieve
with a paper-based system, perhaps an obvious banana skin for a
computer-based one.

Post a Followup

Name:

E-Mail:

Subject:

Circulation:

alexh	colinr	jackm	jennyp
keithh	krugerd	mikec	mylesc
nicr	outcaa	paneris	patickg
programmers	timj	timp	williamc
williams

Message:

>There's an important point about CVS buried in here!  See if you can
>find it ...

>multiboard@paneris.co.uk writes:
> > New message at http://www.paneris.co.uk/philosophy/messages/4.htm
> > 
> > Subject: ISO 9000-3 meets the Bazaar
> > From:    TimP (tim@paneris.co.uk) (194.112.47.112)
> > 
> > The combination of the Bazaar methodology with 
> > automated quality assuance seems to me to be unbeatable. 
> > 
> > As you know I am very keen on ISO9000-3 registration and 
> > would be interested in any thoughts you have on obtaining 
> > it.

>I don't know what 9000-3 is, but if it's anything like 9000-2 then the
>big problem is the outlay.  The "examiners" do not come cheap.

>Otherwise all they are looking for is a sensible-looking system which
>provides things like this:

><UL>
><LI> having designated roles for everyone, including someone in charge
><LI> specification, reviewed at least once and signed by customer and
>     project manager
>  <UL>
>  <LI> requirements spec: what it's for
>  <LI> functional spec: how it seems to the user 
>  </UL>
><LI> project plan, reviewed at least once signed by project manager
>  <UL>
>  <LI> resources (people, plant, euros, ...) needed for development
>  <LI> one of those pretty timetable/dependency charts <TT>:)</TT>
>  <LI> risk analysis
>  <LI> test plan
>  <LI> acceptance plan
>  </UL>
><LI> documentation on the actual architecture and code, reviewed and
>     signed
><LI> versioning/build system with identifiable and recoverable
>     releases
>  <UL>
>    <LI> and system for making backups, with someone responsible,
>         checklists, signatures ...
>  </UL>
><LI> fault report/feature request system
><LI> change notice
>  <UL>
>  <LI> each generated in response to a fault/feature request
>  <LI> contains a description of what files will be changed and how:
>       all work has to be justified/explained in this way
>  <LI> signed off by project manager
>  <LI> when the files concerned are checked out of the version
>       control system, a pointer is left to the change notice
>  </UL>
><LI> checklisted procedure for building a release, since that can
>     be quite complicated
><LI> testing results of each proposed release, signed by test person
><LI> change document for each release, signed by project manager
>  <UL>
>  <LI> summary of change notices made since last release
>  <LI> procedure for ensuring that all interested parties get the
>       change document drawn to their attention
>  </UL>
><LI> acceptance of each tested release, signed by customer
><LI> (go back to fault report/feature request generation)
></UL>

>In addition you need a detailed description of the whole process, all
>the documents and all stages (inputs/outputs from each)---and this
>description must itself be under version control and subject to
>periodic revision.

>Signatures (or their computerised equivalent) are great because they
>focus responsibility; things mustn't be allowed to fall into the gaps.

>Checklists (or their computerised workflow equivalent) are great
>because they provide a simple net for catching some of the more stupid
>mistakes.

>Authorisation and justification of every piece of work is great
>because it provides an audit trail in case things go wrong, and helps
>stop the classic mistakes of duplicating work, messing other people's
>work up, and doing work which is not really needed in the first place.

> > Some jobs will not require the full weight of QA, so our QA system
> > must be scalable to the job

>A system like this is in fact surprisingly unintrusive, because you
>can adjust the granularity of the records over a wide range to suit
>the conditions.  Generally if you only have a very few people working
>on the code then "change notice CN6782: add spellcheck feature; will
>modify [27 files] in whatever way seems appropriate", followed by
>"release 1.4: spellcheck feature; see CN6782" will do fine.

>Some projects even start and end with "release 1.0: initial release;
>see CN0001".

>Also it's perfectly OK to do things in a different order from what the
>above suggests.  For instance you can go away and make changes to your
>local copies of source files in response to a bug report, then make up
>a change notice and get it signed off, check out officially and
>immediately check your changes back in.  Of course, you have to be
>confident that your project manager will agree with your idea and
>hence pay you ...

>Yes this means lying and forging the documentation :).

> > Get the Bazaar methodology working
> > ==================================

>The Bazaar methodology interacts badly with serious QA in two ways:
>firstly, all this signing-off and reviewing requires constant
>communication (I don't know what to about that except give everyone a
>mobile); and secondly, the version control system can become a
>bottleneck.

>Basically the whole check out/check in procedure becomes an enormous
>pain, because the source file is generally too coarse-grained a unit
>to support multiple developers working on multiple problems.  Everyone
>wants to check everything out and the result is chaos.

>That is why virtually all the well-known bazaar projects (Wine, Gnome,
>KDE, egcs, Linux kernel, ...), and many traditional software shops,
>use CVS --- http://www.loria.fr/~molli/cvs-index.html .

>CVS doesn't (by default) bother with locking your sources.  Instead,
>if Mr X tries to commit changes on a file which Mr Y has in the
>meantime already committed changes of his own, CVS uses `diff' to show
>X what Y has done; 95% of the time it is obvious that nothing bad will
>happen and X can leave CVS to merge the two sets of changes
>automatically.  Otherwise X has to fix up his changes to fit with Ys.

>The CVS client and, I think, the server run on both Win32 and Unix
>over TCP (e.g. Internet).

>One of the nice features is that you can give everyone read access,
>e.g. for code readthroughs, while restricting write access to active
>developers.  Of course the aforementioned projects still run into
>problems with those developers messing things up because of
>misunderstandings---which is why a commercial cooperative would do
>better not to trust them quite as much, and instead insist that they
>go through a proper change notice procedure.

>Another splendid thing about CVS is that because it was originally
>just a collection of shell scripts wrapped around GNU RCS, it is still
>extremely extensible and customisable and could be integrated into
>whatever project system one wanted.

> > To test our products we need as many people as possible, 
> > on as many platforms as possible, to visit a site and record 
> > problems and what platforms used.

>You need to write up beforehand, in what you agree with the customer,
>what exactly is going to be tested and on what platforms.

> > For code reviews things get a bit trickier. 
> > Code Reviews have a number of functions. 
> > They are intended to give the customer and ISO an warm feeling,

>Hmmm, maybe the QA people are generally more interested in
>specification and testing---unless you are talking about
>safety-critical code in which case they want to see code reviews by a
>separate third party.

> > but their real import is education and cohesion.

>Yes, they are a fantastic way of *achieving* good results which pass
>the QA tests whose function is to *demonstrate* that you have achieved
>a quality product.

> > This is traditionally achieved by having a 'house style' enforced
> > by reviews and a little automated style checking.

>Just as long as the style points are left very general.  I don't think
>there's any point in trying to enforce particular indentation rules on
>people, or silly complicated naming conventions (except pseudo-package
>prefixes in languages that lack real packages, like C).

> > Finally, of course, the major problem with our Bazaar at the moment is that 
> > very few people are prepared to use the Message Boards for emails.

>Run a news server and gateway it to/from a set of mailing lists.
>That's the only way you will get people to really use it.

> > We should have a mechanism by which scripts/programs can be 
> > held at the project node,

>This was the original idea behind the doc system ...

>As far as supporting a work structure like this is concerned ... I
>think it's sufficiently complicated that it would be worth
>implementing it as a set of high-level rules, maybe even diagrams.

>It will obviously have similarities with expensive products from
>Rational, as well as with humbler things like Notes, and there are
>bound to be homebrewed things already out there.

>Finally, it's incredibly important that the system should be flexible.
>It should be obvious how to short-circuit it.  This is easy to achieve
>with a paper-based system, perhaps an obvious banana skin for a
>computer-based one.

[ Follow Ups ] [ Post a Followup ] [ Discussions at the Paneris philosophy node ]

Re: ISO 9000-3 meets the Bazaar

Follow Ups:

Post a Followup