This document is designed as a broad overview of the Datadictionary System that I have written using Webmacro Technology. It includes API docs for the org.paneris.util package. This document is far from perfect, please do not hesitate to tell me how to maker it better. It should qt least get you started.
Here are some notes on where we have put things on Ganesh (www.paneris.org), they should allow you to easily install and run a Webmacro based application.
Webmacro is installed in /usr/local/webmacro. To configure webmacro, edit /usr/local/webmacro/examples/WebMacro.properties. This file should really be in /usr/local/apache/etc
There are only 2 parameters to be set in here:
TemplateDirectory = /usr/local/dev/templates
LogFile = /usr/local/apache/var/log/wm.log
You should not change these.
This is found in /usr/local/apache/etc/ along with other httpd configuration files. jserv.properties sets up the environment for Webmacro applications to work, so to add a new application you will need to change the Classpath to include the directory with your new Handlers (eg. wrapper.classpath=/usr/local/dev/handlers/sugar ). I understand that in webmacro 0.80 handlers will subclass HttpServlet and therefor be configured as normal servlets. this should mean we do not have to piss arround with Classpaths for each new application
This is found in /usr/local/apache/etc/.
Confusingly named, this is really part of jserv.properties . In here, you configure your handlers eg:
servlet.Upload.code=org.webmacro.servlet.Reactor
You need to do this for each of the Handlers you have. Again, with webmacro 0.80 this should become a lot simpler.
This contains information used for initialising the db connection pool. It specifies a JDBC driver to use and the URL, username and password for each datasource. I have added an 'upload' directory property, which is presently generic to all databases, but need not be.
You will need to amend this file for each application you build.
We have decided that all httpd related logs should live in /usr/local/apache/var/log/
Currently we have the following logs available:
| access_log | apache httpd log |
| db.log | log generated by our database connection pool. This will let us know when we don't return connections to the pool etc. |
| error_log | apache httpd error log |
| jserv.log | servlet runner log |
| jserv.trace | servlet runner trace (tends to be more useful than jserv.log) |
| wm.log | Webmacro log file (this is where you will find all exceptions generated by your application) |
We have created a directory /usr/local/dev/ where we can build our applications. /usr/local/dev/ splits down as follows:
| data | This directory holds data (not databases). It is currently used to store csv files containing data for the sugar application. (I tried putting the data in /home/timj/data - but my java app could not see the directory). This data should probably not be here, so please suggest where to put it. |
| handlers | This directory contains the webmacro handlers. |
| java | This directory contains our own Java resources, currently in a package org.paneris.util |
| templates | Contains Webmacro templates |
Whenever you change your application, you will need to restart Apache. To do this, run
/usr/local/apache/sbin/apachectl restart
webmacro 0.80 promises to spot when a class has changed and to therefor dynamically reload it, but this will not extend to changes to java resources. But wuth dynamic reloading, simply touching a Handler should force everyting to reload, avoiding the need to restart apache so often during development.
When messsing about in Postgres, use su - postgres to log in. This will put you in /home/postgres.
To create a new database type createdb yourdbname.
Go into psql (use psql
If anyone can find any docs for the psql editor, they would be most useful.
In psql, remember to terminate all commands with a ;
The datadictionary is a set of Java classes and Postgres database tables that provide access to meta data that is useful when building a database based system. This is the 3rd incarnation of the datadictionary, the first was written in perl and used for the pm system (http://panerisdev.i-way.co.uk/pm/), the second was also written in perl and used for iglu (http://www.iglu.com/) this remains the most sophisticated. We have now ported the datadictionary to Webmacro and Java.
These is still work to be done to bring the java version up to where we got to with iglu (paticularly wrt javascript validation), but we are nearly there.
It cannot be stressed strongly enough that the datadictionary only deals in meta data, it does not manipulate the underlying database in any way. To do this requires more thought than I have had time to apply. Because we do not manipulate the database, it is easy for the datadictionary and database to get "out of line". For iglu, we wrote a "datadictionary reconciliation report" to find such problems, this has yet to be ported to Java / Postgres.
Because the datadictionary stores it's meta data in the database, you need to set up and populate the database before the datadictionary will work. This is very simple to do, simply run:
/var/lib/psql/scripts/createdatadictionary.sql
in psql, for your new database. Now you have a basic datadictionary set up, and can do everything else via the web interface.
Please not that it is by no means clear to me how to write a webmacro application. I have not seen any other webmacro or even servlet applications. This suite therefor has a tendency to be structured much like a Perl/CGI application. Appologies.
This section is not intended to replace the API docs, but rather give a quick intro to what has been done so far.
The handlers for manipulating the datadictionary via a web interface can be found in:
/usr/local/dev/handlers/administration
There are 4 classes:
This is an entry point for manipulating all datadictionary based applications on the system. It is called without parameters ie:
http://www.paneris.org/webmacro/AdministrationDatasources
It is driven from the db.properties file.
3.1.2 Administration
This is the entry point for manipulating a particular datadictionary. It requires the datasource to be specified ie:
http://www.paneris.org/webmacro/Administration?db=sugar
Which will display a list of Tables in this datadictionary, if called with a table parameter, it will list the contents of that table:
http://www.paneris.org/webmacro/Administration?db=sugar&table=categories
This handler is the only one I have written that can return a choice of templates. I suspect that this handler should be split, so that we have a single handler responsible for preparing each screen.
This displays the edit screen for amending a database field. It requires an id. An id of 0 indicates a new record, any other number will amend that record. ie;
http://www.paneris.org/webmacro/Administration?db=sugar&table=categories&id=2
Takes the Post info from AdministrationEdit and hits the database. A 'refresh' screen is still used to return the user whence they came. A return URL can be specified on the QueryString if required.
This package contains Classes used to service the Handlers (and other database applications). It is very likely that I have put too much code in the Handlers and not enough into these utility classes. Also my OOD/OOP leaves a little to be desired.
You can get hold of the api docs here
The Sugar database has been supplied for Datatext. It is a very simple database of 3 tables:
You can check out the system at www.paneris.org/sugar.html
Create and drop database scripts have been written:
/var/lib/psql/scripts/createsugar.sql
/var/lib/psql/scripts/dropsugar.sql
The sugar specific fields in the datadictionary are populated using a CSV import facility written expressly for Sugar. These files can be found in:
/usr/local/dev/data/sugar/dbadmin
The handlers for the sugar database can be found in:
/usr/local/dev/handlers/sugar
There are 4 classes:
This is an entry point for the system, it simply displays a list of categories starting with 'A'. Links are given for other members of the alphabet
This lists companies that match a particular category.
This displays company company information.
This provides the ability to import .CSV files. It locates the files by looking in the directory specified in the db.properties file. The upload can optionally clobber all data in the table before loading in the new stuff.
Due to a lack of a clear definition of what a CSV file is, I have decided on the following:
1) A CSV files is something generated from M$ Access. Access simply ignores embedded quotes, so the import does not attempt to cope with them.
2) The 1st line of the CSV file should contain the field names.
None of the work detailed here is yet on the CVS system. Chippy and I will attempt to sort this out in the near future.
Document Dated: Fri Apr 16 14:53:01 1999 by TimP
Modify this document
Previous Version