ITworks » DB2

Starting DB2 instances on boot

csak — Thu, 06 Aug 2009 08:32:27 +0000

After migrating our hosted application to a virtual server I realized, that as usual the system is a “maintenance free” Linux, meaning we set it up as it were and left it running for ages without touching it. On the occasion of power failure the applications were started manually.

As I don’t want to restart everything by hand, whenever our system is restarted I decided to iron these glitches out. OK, the system is only restarted about twice a year, but I tend to forget to restart things manually, so that’s the real reason.

The hosted application runs on a WebSphere Application Server Community Edtition 2.0 (meaning an IBM branded Apache Geronimo 2.0) which had an init script hacked together from an init script originally for Apache Tomcat. Talking about code reuse is one thing, doing it is another. It had a few quirks, like the JAVA_HOME setup, that were easily ironed.

The application stores it’s data in an IBM DB2 Universal Database 9.5, which performs beautifully by the way. It’s running on Debian, and was installed using the factory installer, which created the startup scripts as well. However the DB2 instances are not automatically started, and that’s a problem.

I was going to write a startup script which starts the db using the instance user, but it turned out it’s easier than that. DB2 instances can be auto started, only they are not created that way. Changing the flag is easy, as documented here, you just have to issue the db2auto command for the instance.

The only thing left is to add some script to auto start the VirtualBox instance, and preferably stop it as well on shutdown.

DB2 Express-C 9.5 on Debian

csak — Tue, 13 Nov 2007 19:23:23 +0000

For our current pet project we’ve decided to use DB2 Express-C as a database back-end. The choice is made, so our junior developers could practice their art on databases that resemble enterprise databases more than MySQL or Postgres. (We have already ruled out SAPDB/MaxDB on previous occasions.)

First off the system I’m trying to install the server is remotely located, and doesn’t have any kind of X on it. This rules out all kind of graphical installs, but I never was fond of those anyway.

The system is running a Debian testing, with pretty strict firewall rules.

The install went easy, but with a few glitches. First off I’ve downloaded the installer from IBM. This required my PartnerWorld password, so I guess it’s not as easy to get as it seems. Then I’ve unpacked the installer and went on to install as root (this is usual for most installs, but almost mandatory for IBM products) This turned out to be a mistake later, as Express-C is different from other enterprise DB2-s. It’s preferred to be installed as a DB user, unless you want to run several instances on a given machine (not common for a single-shot pet-project).

The install went on fine with the console mode, and as a nice feature, even the text-mode installer shows a time estimate for each step.

After installing the application several times, I went to read some documentation, as I couldn’t find any of the tools I was used to on DB2 EE 8.2. The IBM Infocenter is always a good idea to look up such informations, see the one I looked at here.

Unfortunately all manuals described the startup as a simple process, all starting with the usual loading of DB2 profile, and then starting up the db with the db2start command. Unfortunately it always threw an error message, which turned out to be a problem with the default shared memory size. The simple solution is to add the lines:
kernel.msgmni = 1024 kernel.shmmax = 268435456

to the /etc/sysctl.conf file and then applying the changes with the

sysctl -p

command.

The db then started up fine, but as it turned out I should not be running the non-root installation, as that only permits local “APP” level connections.

So I’ve created the system users dasadm, db2inst for later use.

mkdir /data/db2 adduser --system dasadm --home /data/db2/dasadm touch/data/db2/dasadm/.profile chown dasadm.nogroup/data/db2/dasadm/.profile
adduser --system db2inst --home /data/db2/db2insttouch /data/db2/db2inst/.profile chowndb2inst.nogroup /data/db2/db2inst/.profile

Then the entire DB was reinstalled using the root user.

Creating an admin server

The admin server is used to monitor, and manage your DB2 instances on a system. To create and start service, issue the commands:

chsh dasadm -s/bin/sh DB2_HOME/instance/dascrt dasadm

There seems to be some error on the current version of DB2, the admin server keeps sending messages like:
The state of instance 'dasadm' gcf module '/opt/ibm/db2/V9.5/das/lib/libdb2dasgcf.so.1' is OPERABLE (OFFLINE).
I could not yet locate the actual reason, but to disable this message it’s great to turn the db2fmd off. That can be achieved by commenting out the respectable lines in the inittab, and then reloading the inittab with:

init q

I’ll keep this off until I find some way to work around the current problem.

Creating a db instance

The DB2 instance was created and started using the following commands, as root:
DB2_HOME/instance/db2icrt -u db2inst db2instDB2_HOME/instance/db2istrt -u db2inst db2inst

command. When using the command one must make sure the provided ID actually HAS A SHELL! Of course it took me quite a while to find that out.

Getting the Control Center to work

First off I thought it would be a good idea to just forward the DB2CC X11 output to my computer.

To get X forwarding on the server I tried doing what I read here. After installing the xauth package I was able to forward my favorite X application (xteddy) through to my desktop. There still seems to be a problem, that the window content is not forwarded, even thought the bear shaped window appears. However when I switch over to another user (root and then to db2inst) it simply fails to forward the X. It turned out that the xauth must be copied over to the user that is to be authorized using a command like:
xauth -f /home/MYUSER/.Xauthority extract /root/.Xauthority :0

I could not however use this, to extract the xauth from my user directly to the db2inst user.

Cutting the crap, I’ve got fed up with the xauth, so I just thrown in a shell for the db2inst user, and authorized my user to log-in remotely. Then X11 forwarding worked straight away, and I was able to forward the db2cc without any problem. It’s quite slow, but what the heck?

The speed problem can be helped a bit by adding a -C (compress) parameter to the ssh. It speeds the connection up almost to the level of running it on the local network through my ADSL connection.

Summarizing

After wasting hours on trying to set things up I got some problem starting up the administrator instance, being unable to trace it down to the root of the problem, I uninstalled the entire installation, wiped the install directory and started up the install in graphical mode as root with copying the X authorization as described above.

The installer was running for about 5 minutes, and created and configured the basic DB instance.

Quick and dirty way to create DB2 update scripts

csak — Tue, 29 May 2007 20:52:32 +0000

In on of our major projects we store some of our business logic data as database metadata, but provide GUI interface for updates. Since clicking through the project’s GUI is much easier, than creating and verifying scripts for each tiny modification, our developers got lazy and we had to find a way to create SQL scripts to bring releases up to date. This basically means, we had to create a script generator that compares the data in source and destination databases and creates SQL INSERT, UPDATE and DELETE statements that can be used to modify the destination to source.

For the script we didn’t need to be very precise, as the generated statements are always validated manually to avoid problems. This means that there was no need to care about table constraints, but the table content comparison is enough. Still there were situations where some fields were to be ignored, and situations where the tables had composite keys.

As usual this was not a task that was planed in the project in advance, so we had to hack together a code that didn’t require too much programming. This is usually the case when one takes two strong coffees and switches to script writing mode.

The databases in question are DB2 8.x patched up to a version equivalent to DB2 9.x, the environment the script must run in was GNU Linux, Cygwin and possibly even AIX, I had about 4 hours to write the original version.

On the end I whipped up something like this.

After this script was done I just had to add the table specific invocations to another script and voila it was able to create the scripts we needed to create the deploy scripts.

The simplest case of invoking the function, when the tables are to be compared with all their fields.
compare_content $SRC_DB_ALIAS $SRC_DB_USER $SRC_DB_PASSWORD $DST_DB_ALIAS $DST_DB_USER $DST_DB_PASSWORD 'MY_SCHEMA' 'MY_TABLE' 'ID';

But the script is also good for cases, where you would like to ignore some fields for the comparison would like to conditionally compare the dbs (MY_FILTER IS NOT NULL), use composite alternate key (MY_ALTERNATE_KEY1,MY_ALTERNATE_KEY2,MY_ALTERNATE_KEY3) for identifying a row and want to use a synthetic key as primary key (NEXT VALUE FOR MY_COMPLEX_TABLE_ID_SEQUENCE) on inserts.
compare_content $SRC_DB_ALIAS $SRC_DB_USER $SRC_DB_PASSWORD $DST_DB_ALIAS $DST_DB_USER $DST_DB_PASSWORD 'MY_SCHEMA' 'MY_COMPLEX_TABLE' 'MY_ALTERNATE_KEY1,MY_ALTERNATE_KEY2,MY_ALTERNATE_KEY3' 'MY_FILTER IS NOT NULL' 'ID' 'NEXT VALUE FOR MY_COMPLEX_TABLE_ID_SEQUENCE';
For this the script will automatically create the required statements, where the updates will omit the field ID, the inserts will insert the next value for the sequence in ID, but all comparison will be performed among values that have the same alternate key.

I’m sure this is a script that has a lot to improve, but still this is a good example to show the power of script languages when it comes to hacking together a utility that saves plenty of time, and would be a real pain to create using conventional programming techniques.

If you replace the db2 commands with the equivalents of your favorite DBMS’s command line tool you might see that this can be easily adapted to your needs as well.

NB. There are some other aspects in the original script I didn’t include. Namely it’s able to compare the database structures of the DB’s using the meta data tables of DB2. Since these modifications are quite rare in our project and it’s hard to compare data for some cases, that part of the script is not finished.

Off-line SWT application

csak — Wed, 09 May 2007 23:02:05 +0000

One of our current Customers have a very interesting requirement for an upcoming project. The nature of the project requires mobile clients and frequent (or even instant) database updates, but there is no guarantee to have constant connectivity. We more or less have full control on the choice of tools to use for this application, but since our developer resources are limited, it would be great if we could use the architecture from a previous project. We’ve been working with András on this for quite a while and it seems we found a pretty interesting way to do this.

The current architecture looks like this:

The RCP client connects to the application server using Axis based web services that uses our framework to call the business logic which accesses the database through Hibernate.

As the application currently uses web services to access the framework containing the business logic and this is already separated in the application it’s relatively straightforward to modify this dispatcher to access the business logic.

When online the modified application works just like the one above, and during the online period the application will synchronize its database with the server using DB2 Everyplace.

When the application is off-line the actions will perform the actions on the locally synchronized database, and send MQ messages of these actions through the local MQ Everyplace. Thus the application will see the local database as it’s own and perform the actions on it’s local database. The requirements allow for this, as the data is distributed among the users, (it’s more of a data collector, than business application) and there is only a very little chance of simultaneous updates.

When the client goes on-line again first the MQ messages will be sent to the server, the client will then wait for all the sent messages to be consumed by a message bean on the application server. After the messages are consumed the local database can be synchronized with the server so the data modified by other clients will be reflected on the client.

We’ve also considered using the features of the Lotus Expeditor platform, which works similarly, but decided against it. Our two main concerns were, that firstly we have limited resources and learning a new platform is time consuming, especially for such a new framework, the other was that Lotus Expeditor’s works offline like the original Lotus Notes platform (this is our assumption, based on a very short research) has the preconception, that noone is allowed to use the same database at the same time. If someone does the records are marked as save conflicts that must be reconciled manually.

Our way would permit the business logic to handle these updates gracefully, by either simply applying the updates on the given record in sequence, or refusing the update using either a version- or timestamp, or in workflow like cases statuses.

I’m sure this is not the only, and very likely not the best way to do this, but it’s most certainly the one that would require the smallest effort.

The images suck, I know, but I’m not in a drawing mood right now.