Failure To Do Data Analysis

October 27, 2010

Databases are all about, data.

So it’s totally bizarre how so many people working with databases never look at the data!

It’s really so fundamental, it’s hard to make a good analogy.  Imagine a tailor making your clothes, but never looked at the fabrics!  Can you think of a better analogy?

I’ve met programmers who have charged ahead and written one thousand lines of code, and have not even looked at either the data in the tables, the input, or the output!

Read the rest of this entry »

System Performance: Cobol Versus Database

October 7, 2010

More Architecture Mistakes To Avoid, Database Design Mistakes To Avoid, Tuning

When I started working with Oracle, I helped to replace a Cobol system. For a while, the company systems ran half on Cobol, and half on Oracle.  The Oracle 7 system was pretty slow then.  After the cold backups had run overnight, it was not unusual in the morning to see a query against a 700K row table take about 30 seconds.

I was asked to write a C program that only interfaced with the Cobol flat files, and read Cobol transaction records.  Each transaction was stored in multiple rows, and given repeating groups of records, could vary in size from a few rows, to dozens of rows.  The files contained about 700K to 800K rows altogether.  The program grabbed information out of the multiple lines, tallied the numbers up, and output one line of summary. Lots of IF statements, and loops.  About 800 lines of code.

I expected this program to take at least 30 seconds, given that database queries to similar size tables took that long, and that there was so much logic and so many lines of code.

In fact, the program ran in about 5 seconds!  I was really surprised.

But when you think of it, it should be obvious that Read the rest of this entry »

The Meaningless Data Model

October 6, 2010

More Database Design Mistakes To Avoid
Architecture Mistakes to Avoid

The saying goes, a picture is worth a thousand words.

If an organization has a good database model, with meaningful table and field names, a look at the model should give you a pretty good idea what the system does.

Like a picture, when you look at a model, it’s supposed to communicate.  Consider a model car, or airplane, or a scale model of building.

Without a model or picture, can you communicate how to build the special research car, XOF1?  How it runs?  Or what it looks like?

Similarly, can you look at an ERD (Entity Relationship Diagram) diagram your of database, and have a pretty good idea of the system does?  Or not?

At so many places, I’ve have to look in many places to understand the system.
– Many queries to the Oracle Data dictionary
– Values in database fields.
– Rows in database tables.
– Extensive data analysis.
– Requirements documents, that don’t give any specifications, most always outdated. Read the rest of this entry »

Cars, License Plates, And Using The Wrong Identifier

September 29, 2010

More Database Design Mistakes To Avoid
Architecture Mistakes To Avoid

I’ve now moved to a few different states, and registered my car in the new state.

Even though I was only a customer, and never saw the back end database, I could quickly tell which identifier the states were using for the car.   Each identifier had huge implications on the whole process, the consumer, the insurance companies, and the DMV.

In Connecticut, the process (in 1999) I followed was as follows.
Read the rest of this entry »

Creating Cobol Systems Inside the Database

September 29, 2010

More Database Design Mistakes To Avoid
Architecture Mistakes To Avoid

I haven’t worked a whole lot with Cobol, but I did do enough.

Coming from a background where the first thing I really learned well on the job was how to do good database design, there were some odd things that I noticed about Cobol.
Read the rest of this entry »

Ways To Get Duplicate Data

September 23, 2010

More Database Design Mistakes To Avoid

Duplicate data.  Designed into the system.  How can I count the ways?
Read the rest of this entry »

Splitting a Table Vertically, AKA, Duplicate Tables!

September 23, 2010

More Database Design Mistakes to Avoid

There is more than one way to split a 1:1 relationship into multiple tables.  In the first example, the table was split horizontally, and different columns went into different tables.  But the number of rows stayed the same in all of them.

The stranger way is to split the table vertically.

In other words, to have duplicate tables!
Read the rest of this entry »

%d bloggers like this: