English Yacs handbook Content management

How to create YACS pages out of a Lotus Notes database?

Learn about the methodology we have used to transfer some records from a Lotus Notes database into YACS

As a network consultant we were using Lotus Notes for some years to record our professional services assignments. In order to switch to open and efficient web technologies we have developped an overlay script to describe our assignments under YACS. This has been achieved roughly in May, but concerns only new assignments.

This page describes the several steps we have used for the migration of legacy information that has just taken place. You should be able to use this information for any such kind of migration.

All you have to do is to prepare a valid XML file, and for us this meant an overall process made of three steps, as depicted in the diagram below:

1. Export data - In our case, from Lotus Notes to a flat ASCII file

2. Shape it to XML - In our case, using some specific AWK script. But PHP or Java would have do as well

3. import into YACS - Just upload the XML file to the adequate YACS script

[title]Step 1 - Export data from Lotus Notes[/title]

We have selected to keep things as simple as possible. Basically, Lotus Notes had just to dump the textual content of the database. The result is a flat text file, with structured content as demonstrated in the following excerpt (actual names have been changed):

ProjectResources:  
ProjectKeywords:
DateCreated: 03-09-1998 16:27:46
TimeCreated: 03-09-1998 16:27:46
ProjectOwner: CG
ProjectEntryDate: 03-09-1998 16:27:46
ProjectStartDate:
ProjectDuration: 1 month
DateModified: 25-01-1999 15:34:20
TimeModified: 25-01-1999 15:34:20
$Revisions: 03-09-1998 16:33:14,03-09-1998 16:33:41,03-09-1998 16:34:19,03-09-1998 16:36:58,25-01-1999 11:32:23,25-01-1999 14:31:02,25-01-1999 14:41:51,25-01-1999 14:44:53,25-01-1999 14:45:49

Assist xxxx to audit existing network and prepare RFP for
outsourcing of domestic network. Value xxx KUSD (base yy days @ zzz USD/d)


[title]Step 2 - Build a XML file[/title]

Then a specific AWK script has been created to transform the input file and to produce a valid XML file, according to YACS XML grammar for articles (see articles/article.dtd.

<article>

<description>[table]
Status|inactive / completed
Leader|CB
Workload|20 man.day
Start Date|1998-09-01
Contacts|Mike Dump LAXSG
Resources|CB, Roberto Alfredo
[/table]

<pre>initial ASPAC SOW sent to SYD Account team in June 98

Fact-finding missions:

Weeks 42 (US: SFO, DAL), 43 (ASPAC: SYD, SIN) and 44 (Europe: LON)
Collected documents (mostly paper) are in 'yyyy' binder in my office

Architecture redesign recommendation:

Project presentation at december service meeting in SFO:

Migration Guidelines (11-jan-99):

SPS from VO database:
IP addressing plan from EIS Sydney:

</pre></description>
<create_name>CB</create_name>
<create_date>1998-07-17 11:30:17</create_date>
<publish_name>CB</publish_name>
<publish_date>1998-07-17 11:30:17</publish_date>
<edit_name>CB</edit_name>
<edit_date>2000-10-06 16:12:12</edit_date>
</article>


[title]Step 3 - Import data in YACS[/title]

As you may imagine, the XML file has been uploaded to our corporate web site, and the YACS importation script (articles/import.php) has been asked to create pages in the section for assignments.

[title]Lessons learned[/title]

Of course we have lost a lot of useful information during the migration, and most notably files attached to assignments records.

Another annoying thing is that we have lost the 'folding' capability of Lotus Notes, meaning that event lists may be cluttered with copies of mail messages.

However we have now a simple system to handle most important pieces of information gathered so far. Through the search box provided by YACS, we are able to dig in the past when necessary.

The entire process required about 4 hours of work, meaning about 40 seconds per migrated record. Most of this time has been devoted to the development and test of the AWK script. If we assume that it would take about half a day to build such a script, the operation is cost-effective as soon as more than 200 records have to be transferred.

If you are interested into migrating some Lotus Notes database to a YACS-based server, do not hesitate to comment this page or to submit a query form.