The Case of the Disappearing Rows

August 5th, 2011

I just lost a week of my life trying to track down a problem with a program that did a mass insert of records into a SQL Server database.  The program is written in Delphi 2007 and uses dbExpress components for access to SQL Server.

The problem first manifested after an 8 hour run when, lo and behold, the database only had 11K rows rather then the expected 884K.  I knew that the program was sound because I had just run the same program to load the same file into a Firebird database without a problem.  (That only took 3 hours by the way).

Now, the source file had some bogus data in it.  I knew this and the program made allowances for it by trapping the resulting exceptions, writing them to a log file and dropping the offending record.

It turns out that these data conversion errors fall into a class of SQL Server error known as “batch aborting” errors.  Whenever a batch aborting error occurs the current transaction is rolled back without so much as a “by your leave”.  You would think that this would be documented somewhere, in BIG BOLD LETTERS.  If it is, I can’t find it.  If it weren’t for Google I would still be scratching my head.

I found this table in an article on another web site:

Error Aborts
Duplicate primary key. Statement
NOT NULL violation. Statement
Violation of CHECK or FOREIGN KEY constraint. Statement
Most conversion errors, for instance conversion of non-numeric string to a numeric value. BATCH
Attempt to execute non-existing stored procedure. Statement
Missing or superfluous parameter to stored procedure to a procedure with parameters. Statement
Superfluous parameter to a parameterless stored procedure. BATCH
Exceeding the maximum nesting-level of stored procedures, triggers and functions. BATCH
Being selected as a deadlock victim. BATCH
Permission denied to table or stored procedure. Statement
ROLLBACK or COMMIT without any active transaction. Statement
Mismatch in number of columns in INSERT-EXEC. BATCH
Declaration of an existing cursor Statement
Column mismatch between cursor declaration and FETCH statement. Statement.
Running out of space for data file or transaction log. BATCH

The entire article, which is a good review of SQL Server error handling techniques, can be found here.

Using 32-bit ODBC drivers on 64-bit Windows

June 28th, 2011

When you start the ODBC configuration tool from the menu or the control panel on 64-bit Windows, it starts the 64-bit version of the tool.  All well and good.  Unfortunately, this only shows the 64-bit ODBC drivers that are installed on your system.  What if you need to use a 32-bit ODBC driver?

It turns out the quite simple once you know the trick.  You simply need to run the 32-bit version of the ODBC configuration tool.  To do this just browse to c:\windows\syswow64 and double click odbcad32.exe.  This will allow you to configure the 32-bit ODBC data sources that you need.

TFrame Displaying as TForm in Designer

June 15th, 2011

I came across an interesting quirk in the Delphi 2010 IDE today.  I was working in a project that uses a lot of TFrame components.  To make life easy on myself, or so I thought, I created a custom TFrame descendant and used that in place of the default TFrame.

Everything was working well until I displayed one of the frames as text, made some silly change and then redisplayed the frame’s visual designer.  At this point, the IDE started treating my TFrame as a TForm.  It displayed as a TForm in the visual designer, all of TForm’s properties appeared in the object inspector, all of TForm’s properties were being saved in the .DFM file.  I didn’t notice this right away.  Not until I recompiled everything and tried to run the application.  At this point I started to get ‘ClientHeight property not found’ errors when I tried to instantiate the TFrame.

This little gotcha cost me most of an afternoon.  I guess the rule is:

If you are designing a TFrame descendant use the visual designer only.  Do not try to edit the .DFM file as text.

Converting Native Fujitsu Cobol Numeric Fields to / from Decimal

June 13th, 2011

There are any number of bizarre syntax rules concerning the interaction of native Cobol data types and .Net data types.  One of the stranger ones is this:  If you want to convert a .Net Decimal value to a native Cobol numeric field, that field must be signed.  Oddly enough, the inverse is not true.  You can convert unsigned Cobol fields to .Net Decimal values.  So, to convert Decimal to Cobol you must do something like this:

01  WS-TEMP PIC S9(5)V99.
01  WS-VALUE PIC 9(5)V99.
01  WS-DECIMAL OBJECT REFERENCE CLASS-DECIMAL.

SET WS-TEMP TO WS-DECIMAL.
MOVE WS-TEMP TO WS-VALUE.

What at PITA!

Global Compiler Options for Fujitsu Cobol

June 13th, 2011

There are certain compiler options that must be applied to Fujitsu Cobol projects that must be the same for all projects if things are to work properly.  For example, there is an option that controls the internal representation of signed USAGE DISPLAY numbers.  If different programs use different options then they are going to misinterpret each other’s numeric fields. Fortunately, it is possible to specify options to the Cobol compiler that will be used by all projects.

There is a file named cobol.rsp in the Net Cobol for .Net installation directory that holds the global compiler options.  Simply add a new line to this file for each compiler option that you wish to specify.  For example, to specify 88 Consortium signed numeric field handling, add a line with /wc:”Decimal(88)” to cobol.rsp.

Using RM/Cobol Indexed Files with Fujitsu Cobol

June 3rd, 2011

We are migrating an application from RM/Cobol to Fujitsu Net Cobol for .Net.  As part of this, we want to be able to access indexed files from RM and Fujitsu programs concurrently.  Since the Fujitsu indexed file handler appears to have been licensed from RM back in the misty depths of time, this would appear to be a no-brainer.  Unfortunately, this turns to not be the case.  While RM files are compatible with Fujitsu programs, there are restrictions.  Here is what I have been able to find out so far: Read the rest of this entry »

Visual Studio 2010 Very Slow

May 6th, 2011

One morning I fired up Visual Studio 2010, opened one of my solutions and found that the Visual Studio user interface was so slow as to be completely unusable.  It took 5 seconds+ for each key stroke to appear, or to switch tabs or to do just about anything in fact.  But only for this one solution.  All of my other solutions seemed to be fine.

After spending a considerable amount of time in Google I finally found the solution to the problem.  I deleted the .SUO file in the main solution directory and the speed returned to normal.  No idea why.

Apparently, the .SUO files contain user dependent information, like the state of the solution explorer (collapse all nodes, close the solution, delete the file and  you’ll notice that the nodes in the solution explorer are expanded again).

Imagelib PutTifFile Function Hangs

April 25th, 2011

I got hung up for a while recently trying to figure out why the Skyline Tools Imagelib puttiffile function was hanging when I tried to use it to create a multi-page TIFF file from a series of bitmaps.  It took me a long time to figure out that appending to an empty (zero byte) file causes puttiffile to enter an endless loop, while appending to a non-existent file does not.

Go figure!

Anyway, I have bundled up a small test application and sent it to Imagelib technical support.  I will post here when I get a response.

Black Bars When Displaying TIFF Images Using Windows Image Viewer

April 25th, 2011

I recently discovered that certain TIFF images generated by our document imaging application were showing ugly horizontal black bars when viewed using the Microsoft Image Viewer on Windows 7 and Windows 2008.  Thinking that this just had to be a problem with our imaging system I spent many hours trying this and that to see if I could figure out what the problem was.  I even bundled up the bad images and sent them off to technical support for the imaging tool kit that we use.  All to no avail.

I finally turned to a developer’s best friend (Google) and spent a couple of hours on that before I found out that the black bars are caused by a known bug in the Microsoft Image Viewer.  There is even a hot fix for it. This is all described in KB 2459492.  Downloading and installing the hot fix solved the problem.

One wonders why Microsoft didn’t release this as part of the Windows Update process.

Delphi with … do Gotcha

February 10th, 2011

I got bitten by something the other day and it had me tearing my hair trying to figure out what was going on.  An application that had been working for years starting throwing access violations.  Stepping through it with the debugger, everything looked good.  No nil pointers, nothing obviously wrong.

Finally, I realized that I was freeing and reallocating the object referenced by a with … do block while I was inside the block.  Very simplified, the code looked something like this:


with myObject do
begin
myObject.Free;
myObject := TmyObject.Create;
end;

Pretty stupid when you think about it, but frustrating all the same.