To: paneris@i-way.co.uk, lurker@paneris.co.uk
Subject: Sys Admin - where do we put projects [/system/newbox/ by Mylesc]
X-Mailer: VM 6.34 under Emacs 20.2.1Sorry about this long message ... the gist should be clear in amongst
the rambling.
multiboard@paneris.co.uk writes:
> For this reason it seems sensible to organise all files for a
> particular project together under one directory. This will allow us
> to checkout out one CVS module per project.
God yes, that's critical.
> Note, however, that I think the project documentation should
> live somewhere else (inside the PanerisII project) since we
> should only really be concerned about interacting with that
> via the web interface.
No, it should go along with the code. The quality docs, code
documentation docs, code drafts, scribbled notes, code, templates,
test data, even the live data should all be a seamless,
cross-referring whole with the integrity of the references maintained
by CVS checkpointing.
Everything, but everything, relating to a project goes under that
project's directory. It makes version control and backup and copying
and deletion and everything far easier.
> This just leaves the templates. At present we can only have one
> template directory for each instance of webmacro.
This is a technical problem and there is bound to be a solution,
whether it involves symbolic links or hacking webmacro to behave
differently or whatever. As far as I can see, if webmacro has a
single template directory then it's entirely natural to call the
templates "sugar/foo..." since otherwise you are collapsing namespaces
and making a great big scrambled pudding of a directory.
> We could have a whether it involves symlinks or
> jserv zone for each webmacro application, but the JVM that webmacro
> launches is relatively large (I'm not sure if it's large enough to
> worry about).
The executable and shared libraries will be shared between processes.
In fact there is something to be said in favour of splitting
essentially separate tasks between separate JVMs: garbage collection
takes time essentially proportional to the size of your live data set.
> I think that that is a fairly sensible organisation for
> projects. Where should we put each project's directory? I suggest
> that we put them under: /home where at the minute
> we have /home/httpd. Thoughts?
Fine, if projects really do correspond to Unix users (as httpd is a
Unix user). Otherwise something under /usr/local is a fair choice.
For what it's worth I have always found that the best global directory
structure on your machine is one that makes the cleanest possible
separation between
o OS
o generic installed software
o generic software distributions (ie sources you compile)
o master copies of your own specific packages
o stuff specific to the functioning of the particular machine
For example that's in increasing order of backup-worthiness. If you
have a convention that everything under /usr/local/packages is
something you have installed onto the machine on top of what came on
the CD, then the backup problem reduces to (1) a list of the installed
RPMs, which you can get from rpm -qa and (2) a backup of
/usr/local/packages and /home, plus whatever symlinks are needed from
/usr/local/bin, /usr/local/lib and /usr/local/man to activate the
packages. Similarly installing your whole environment on a new
machine just means ensuring (1) and copying (2).
I would argue for
/usr/local/packages/master/cvsroot/projects/bloggs-plc/sugar as the
CVS repository,
/usr/local/packages/inst/projects/bloggs-plc/sugar-1.06 as the
installed, currently running version and
/usr/local/packages/dist/projects/bloggs-plc/sugar-1.06 as the
canonical distribution (only needed if there is compilation or
equivalent to be done).
> What about code common to a number of projects? This includes:
>
> - administration handlers
> - common java (e.g. datadictionary stuff, utils, messageboards)
> - common perl
>
That goes in /usr/local/packages/master/cvsroot/dbadmin. Under
there goes everything to do with the datadictionary library. That
means Java, Perl and ML versions of the code (with the Java probably
under src/java/org/paneris/dbadmin), all the documentation, and
so on.
> We can get access to generic handlers so long as the
> classpath includes the central repostitory of generic
> code.
Yeah, or what I do is make symlinks out of /usr/local/lib/java. For
some reason I don't like PATHs, CLASSPATHs, Perl INCs, C -Is ...
symlinks let you have these relationships explicit in the filesystem,
not hidden in some stupid .bash_profile.
> If we need to, we can copy the code into each project (but then we
> would have to worry about multiple versions - ugh).
Yes but if you don't do that, then you have to be very disciplined
about versioning on the libraries and effectively making a new release
of *all* the libraries every time you make a snapshot of one of the
projects they depend on. Otherwise things break annoyingly.
In summary, I believe strongly that these are the right ways to
organise our directories:
o according to functionality
and these are the wrong ways:
o "it's source" versus "it's documentation"
o "it's HTML" versus "it's code"
o "it's Java" versus "it's Perl"
o for the convenience of our server software
Here's a tree for classifying all the files on the computer by function:
OS
...
machine config
...
users' files
...
installed packages
package distributions (e.g. sources to be compiled)
...
package installations
ours
...
...
third party
...
our master sources/docs
general libraries
...
specific projects
...
...
That suggests the following directory structure:
/usr/local everything installed specially on this machine
/usr/local/packages the bodies of the installed software
/usr/local/packages/dist ... their distributions, where necessary
/usr/local/packages/inst ... their actual installations
/usr/local/packages
/usr/local/packages/master master copies of our software
/usr/local/packages/master/cvsroot
our CVS repository
/usr/local/packages/master/cvsroot/
our libraries (source + docs)
/usr/local/packages/master/cvsroot/projects//
our projects (source + docs)
/usr/local/packages/inst/paneris//-#.##
installed binaries etc. of a project/usr/local/bin links to the appropriate files in packages/inst
/usr/local/lib ditto, where necessary
/usr/local/include etc.
/etc this machine's configuration
/home stuff specific to particular users
in particular:
/home/httpd links into packages/inst, or
working copies in developers
directories
Operations this scheme supports: foolproof, low-effort and prioritised
backup; simple package deletion; hassle-free copying of packages onto
other machines. It's also transparent and consistent, which means
easier understanding for us and incomers and fewer silly mistakes.
The working practices go as follows:
You do development work on a personal copy of absolutely everything
(code, HTML, data, docs ...), whether that's on your own machine or on
a common dev machine. That's under /home/you/whateveryouwant,
completely informal and outside the above framework.
When you are happy that you have done the work to implement a
particular thing, you commit to the CVS repository.
During early development you maybe just do a cvs update into the
common development site directory.
As we start to have more to show the customer and the thing gets
bigger, we make things a bit more orderly by following this procedure:
o work in your personal dir
o cvs commit
o get thumbs up from from project leader to ...
o cvs tag a new "release"
o cvs update it into the new directory
.../packages/inst/paneris//-0.4.37
o perhaps test it a bit, informally
o move the link
.../packages/inst/paneris//
to point to ...-0.4.37, thus making it the site pointed
to by the web server---but we can still switch back
very easilyFor post-release maintenance, when the site is live, there are more
stringent tests and signoffs to go through.