Detailed Design Documentation

Today I had to present my detailed design document for an OBIEE project.

The response was...not great. Not even good. So bad, in fact, my boss had to call me afterwards. Fortunately for me, he's a good boss and remembers the good with the bad.

I'm not sure if it is evident, but I'm not afraid to fail. I've done it before, I did it today, I'll do it again in the future. It's just life. As much as I would like to believe (and as much as I tell my wife and kids), I am not perfect. Never have been, never will be.

What should be evident though, through 686 posts here and the untold pages of documentation I have written for wikis, email, groups, blog comments, or some other medium, I don't have a problem with documentation. I like it. I find a lot of value in it. I usually take the lead on it if nothing exists.

Not too long ago I talked about Design Documentation, but that was more of the high level kind. The kind I was expected to deliver today was to be much more in depth. How much more so? Well...

Physical Layer
Each table and the columns that were to be used. This includes aliasing these tables and then their respective joins.

No, that's nothing terrible to ask. I don't argue that.

BMM (Business Model Mapping) Layer

1. I was to have every column defined (both sourced and calculated).
2. Each and every dimension and fact table, including all the Logical Table Sources (LTS) for each of those, including their joins (INNER/OUTER/FULL OUTER), Content, Fragmentation and Leveling defined.
3. The heirarchies (aka dimensions, but not of the table kind) defined by level with all the accompanying columns at each level.

This section gave me the most problems today. While I appreciate the high level design, I felt this encroached a bit on actual development. The more I think about it though, the more reasonable it sounds.

It is a departure for me though.

I tend to develop and design in tandem. Documenting everything up front feels...dirty to me.

Perhaps it's the fact that I was using Word? I mean, seriously, it's 2011. Why are we using binary files? Data should be free!

I prefer wikis. I prefer email over Word. I've used Atlassian products before and think they are great.

Where I have a problem is this (seeming) waterfall approach. How do you know what you don't know?

Then I'm back at the format of the documentation, Word, stored in...ugh...SharePoint. I don't believe I have ever met a less desirable solution. SharePoint is the suck.

Documentation should live and breathe. I have never seen that happen with a Word document. Never. I consider myself somewhat disciplined, and I couldn't fathom maintaining this. Let's make it more narrative (wiki). Let's have a place where the latest and greatest lives, with a possible story of how it got that way. The decisions were made because of A, B and C. You miss most, if not all, of that in the design documents.

Perhaps it's not even Word, or SharePoint. Perhaps it is the process. I'm probably not asking the right questions yet.

To sum it up, I failed today. I have to follow the guidelines set forth...but I can try to influence them...and of course you know I will.

Why I Virtualize

I made the permanent switch to Ubuntu almost 2 years ago. I haven't looked back.

While I don't believe it is quite ready for prime time, it gets close with every release. By prime time, I of course mean, will I have to support it. I have tried to talk my parents into it, but only half-heartedly; I know I'll have to do more than I already do.

Since I started using Ubuntu I have also been using VirtualBox extensively. Originally, it was just for work, i.e. no one seems to allow the easy integration of linux distros onto their network. I say that based on just my experience, which is not extensive. There are also certain work related tools that only run on Windows.

Here's the short list of tools that I use that require Windows:
- GoToMeeting - I love the tool, from a collaboration perspective. I hate that I have to use Windows to use it. Fortunately, I always have 1 or more Windows VMs up and running.
- Neoview Management Tools - this doesn't matter much anymore, but it was one of the original reasons.
- Quicken - I haven't moved to the cloud version yet. I probably will. Soon.

Wow. I think that might be it. Cisco VPN used to be a reason until I found vpnc. That's really not too bad. I could be down to 1, GoToMeeting. I did find that Adobe has a product that is OS independent, pretty sure it uses Flash. Definitely a future consideration...

I don't game, at all, so I have no need for Windows there.

The reason I virtualize? Compartmentalization.

Here's why:

1. Last year, while on site at a client, my Windows VM got corrupted. I was back up and running within 2 hours. If that had been my host system, I am sure it would have taken much, much longer. I only put the software on the VMs that I need. Nothing more, nothing less. Windows Office. Notepad++. WinMerge. SQL Developer. Maybe a few others specific to the client.
   
   I don't store any data on the VMs, I use Shared Folders and write to my host disks. If I lose one, no big deal.
2. Single Unit of Work. I can create a VM with an Oracle database, one with OBIEE, one with Subversion and a Wiki (ok, not a single unit of work, whatever). Since the purchase of my new computer, this is my reality. If I lose one of those VMs, I don't lose all the work I put into them. I just rebuild the one I need. Annoying, yet. Catastrophic, no.
   
   If I built a single system with all of these components, tools, etc, the loss of my computer would be catastrophic, even with the best backup policy in the world.
3. Finally, there was last night's incident. I have a large VM (12 GB RAM allocated, 4 processors) for this medicaid database I am putting together. I had a 20 GB OS partition. At some point, VirtualBox decides to write that RAM to disk, all 12 GB of it. Space available: 0 (zero) bytes. I couldn't do anything. OpenOffice kept telling me I had no more space to auto-save, had to kill it. This even after I pointed 99% of the settings files away from the OS partition.
   
   I tried symbolic links. I was successful with the Desktop, but nothing else. I tried, very unsuccessfully to point /dev/shm, /dev/pts, /var/run and /var/lock to a different location. If I had a better understanding of how the OS works, it might not have been a big deal, but for me...
   
   Since you can't expand a partition (please don't tell me you can), I decided to just wipe the host OS and re-partition my drives appropriately. Total time* to get everything back up and configured? 3 hours. VirtualBox was easy, pull in the vdi files, create a new VM and voila, everything is back to normal.

Those are my reasons for virtualizing. I suspect those of you who use VMWare, VirtualBox or any other virtualization tool for a living might have different or better reasons. Please share.

* Total Time = Does not include the fact that the first USB Flash Drive I was using didn't work properly. I spent 4+ hours trying to get GParted to work to no avail. Research pointed to the possibility of using a different one. I went to BestBuy, bought a new one and everything went smooth. By the way, to add to this frustrating day, my battery died at BestBuy. Awesome.

OBIEE: Gotcha #2

This one bit me hard today and it shouldn't have. I should have followed my instincts...instead I trusted the tool. ID10T.

So here we go.

We use the Filters in the Security tool which can apply predicates based on the Group.

First, open up the Security Manager



You should see this


Then click on Permissions


I then, like an ID10T used the Expression Builder to build my predicate. This was a BETWEEN, so I went to Operators and found the BETWEEN and double clicked on it. Can you see what it did?


How about now?


That's right, it says "Upper Bound" first followed by "Lower Bound"

I knew that wasn't right...but I went with it anyway.

After the "mistake", I verified in SQL.

SQL>EXEC :C := 20;

PL/SQL procedure successfully completed.


         C
----------
        20

SQL>SELECT * FROM DUAL WHERE :C BETWEEN 40 AND 10;

no rows selected

Elapsed: 00:00:00.18
SQL>SELECT * FROM DUAL WHERE :C BETWEEN 10 AND 40;

D
-
X

1 row selected.

Live and learn I guess. I'll never trust that thing again. One more reason to dislike GUIs.

OEL, OBIEE, VirtualBox and Shared Folders

I spent the week installing OBIEE in Windows (Vista Ultimate) and OEL 5, Update 4. I've always tried to install the software I'm working with, it helps to give me a better understanding of what's going on. I don't always understand at first, but after prolonged exposure, I can more easily put the pieces together.

Digression
I can remember trying to install OAS on a Windows server back in 2002, multiple times. I had absolutely no idea what I was doing...just clicking buttons and stuff. In particular, there was the was this one night...err, morning, 4 AM to be exact where I was installing the software for what felt like the 200th time. Did I mention I was onsite...and in the server room? The only reason I remember that particular time was the justification I was using...LC was T minus 1 week.

Post
In the OEL/OBIEE article, I mentioned some problems that I had, those problems are the reason for this post.

I'll recall the setup:

Guest
Oracle Enterprise Linux 5, Update 4
Oracle Business Intelligence Enterprise Edition, 10.1.3.4
Java 6 (1.6_0.17)
Memory:  1280M
Diskspace:  30 GB

Host
Ubuntu 9.04
VirtualBox 3.0.10
--VirtualBox Guest Addtions (i.e. Shared Folders)
Dell XPS M1530
3 GB RAM
150 GB Hard Drive

I believe that's all that is relevant. If I left something off, let me know and I'll add it in.

This is a fresh install of OEL. After installation of the OS, I installed the Guest Additions so I could 1, get a higher screen resolution and 2, I could use Shared Folders.

From the host, I unzipped the OBIEE software into /opt/software/oracle/obiee/linux, I then un-tarred (?) the resultant .tar file.

From the guest, I logged in as root and issued:

mount -t vboxsf software /mnt/software

* You can see the full set of instructions on the Shared Folder post

I then navigated to the directory via the mounted path

/mnt/software/oracle/obiee/linux/RH_Linux/Server/Oracle_Business_Intelligence

From there I followed the instructions.

[oracle@localhost Oracle_Business_Intelligence]$ ./setup.sh -console
InstallShield Wizard

Initializing InstallShield Wizard...

Preparing Java(tm) Virtual Machine...
...................................
...................................
...................................
...................................
...................................
...................................
...................................
...................................
...................................
...................................
...................................
...................................
...................................
...................................
...................................
...................................
...................................
...................................
...................................
...................................
...................................
............

It would just hang there. For the longest time.

I poured over the documentation to see if I missed a required library or something. Nothing. I ran and reran the UnixChk.sh script multiple times (the script checks to see if OBIEE can be installed and outputs SUCCESS or FAILURE). I would start and stop this installation over and over. Going so far as to let it run overnight (which for me is about 5 hours).

I could get past this part, given sufficient time, typically down to the point where it is actually writing the software to disk (presumably anyway).

Installing $L(com.siebel.analytics.install.AnyStringConstants,
InstallConfig.ProductName) $P(SiebelAnalyticsProductBean.productNumber). Please
wait...


|-----------|-----------|-----------|------------|
0%         25%         50%         75%        100%
|||||||||||||||||||||||||||||||||||||||||||||||||||

Installing Oracle Business Intelligence 10.1.3.4.1. Please wait...
||||||||||||||||||||

Again, it would just sit there.

Finally, on Wednesday evening I saw a friend of mine (Tom Roach, former President of SOUG) online (or did he see me?). Anyway, I told him about the situation and, since he expressed interest in my problem, I offered to show him.

We then spent the next 30 minutes trying to get him access to my computer. Suffice it to say, I can do that easily on Windows. Ultimately, he talked me through enabling SSH and he was in.

That's when the magic began.

As I watched him type up all these commands, looking through scripts...and starting and stopping the installation. I had my own personal (*nix) trainer. First, I believe he ran

[oracle@localhost Oracle_Business_Intelligence]$sh -x ./setup.sh -console

Which then output all kinds of stuff I didn't recognize. I asked him later what it was, the sh command let's you debug the shell script to get a better idea of what is going on. When he killed it, I noticed an error, "'run' class cannot be found" or something along those lines. Before Tom showed up, I had suspected there was something going on with my Java environment. I tried using jcontrol to change the memory allocated to Java upon start with no luck. I began installation, then ran jconsole to see what was going on with the JVM. I saw "things" that must have been the problem...none of this proved out in the end. In fact, I think I told Tom that originally, that I suspected that the Java environment wasn't set up correctly.

He then noticed that it was calling Linux.jar in the same directory. So he ran that directly (to eliminate a possible error in the main script I'm guessing):

[oracle@localhost Oracle_Business_Intelligence]$exec Linux.jar -console

or something very close to that.

Occasionally he'd run this "history" command. Wow! You can see all the commands you've entered. That's helpful.

Next up he tried to run "strace." "File or directory not found" Tom, prefix it with /sbin or /bin. No go. It's not installed. I then began to look for it on 1 of the 4 ISO images. Found it and loaded it.

ENTER STRACE COMMAND

He then began to pour through the when he notices lines like this:

wait4(5171, 0xbfa23aac, WNOHANG|WSTOPPED, NULL) = 0
read(3, 0x8a39cd0, 4096)                = -1 EAGAIN (Resource temporarily unavailable)
read(5, 0x8a39cd0, 4096)                = -1 EAGAIN (Resource temporarily unavailable)
nanosleep({0, 10000000}, NULL)          = 0
wait4(5171, 0xbfa23aac, WNOHANG|WSTOPPED, NULL) = 0
read(3, 0x8a39cd0, 4096)                = -1 EAGAIN (Resource temporarily unavailable)
read(5, 0x8a39cd0, 4096)                = -1 EAGAIN (Resource temporarily unavailable)
nanosleep({0, 10000000}, NULL)          = 0
wait4(5171, 0xbfa23aac, WNOHANG|WSTOPPED, NULL) = 0
read(3, 0x8a39cd0, 4096)                = -1 EAGAIN (Resource temporarily unavailable)
read(5, 0x8a39cd0, 4096)                = -1 EAGAIN (Resource temporarily unavailable)
nanosleep({0, 10000000}, 0)             = ? ERESTART_RESTARTBLOCK (To be restarted)

Googled those, but I believe he had his suspicions. He also found the following useful, but I'm not sure why. I think he mentioned something about the fork() and trying to change directories but being unable to.

getcwd("/home/oracle"..., 1024)         = 13
chdir("/mnt/software/.../Oracle_Business_Intelligence") = 0
pipe([3, 4])                            = 0
pipe([5, 6])                            = 0
fork()                                  = 5171
chdir("/home/oracle")                   = 0
close(4)                                = 0
close(6)                                = 0
fcntl64(3, F_GETFL)                     = 0 (flags O_RDONLY)

If I can't recall very clearly it's because we started this IM at about 11:30 PM EST. It was now approaching 1 AM.

Based on what he said and the problems Jake had previously, I began to seriously think it might have something to do with the Shared Folders. I wondered aloud on IM, "Could this be due to the Shared Folders and the OS's inability to write there?" Tom, "Perhaps"

I asked if I should just move the software to the OS and run it from there.

By then Tom was giving me a great tutorial on screen.

Tom didn't get to installing the software successfully, but he gave me sh, screen, history, and strace. By reviewing the output with him and explaining what I was trying to do, Tom got me going in the right direction.

After 2 hours, now 1:30 AM, Tom finally succombed. That was pure awesome...thanks again Tom for your help!

I wasn't ready for sleep yet (besides being a bit of an insomniac), I had new life.

First copied the files over to the guest OS. Ran the UnixChk.sh script then ran the setup.sh script. After about a minute, I was prompted for my first response. Sweet! I answered all the questions and went down to watch TV for an hour or so. I came back, answered a few more questions and it was done.

Only to be destroyed later that morning and re-installed.

"They're just not working out..."

Yup, let's stop the clock.

189 days isn't so bad is it?



Read the timeline for more info on my brief stint in Tampa.

No OpenWorld For Me: Part II

To say it has been a crazy week or so would be a gross understatement.

Last week, I bowed out from the race to attend Oracle OpenWorld.

Two days later, Justin then Jake then Billy (and here) started a movement to get me there.

All of this in conjunction with 100+ tweets and re-tweets on Twitter. Bradd even got his shirt today. He was one of the first of 15 so far to make the purchase...ostensibly to get my fat ass (Bradd didn't say that) to San Francisco and OOW.

Here's some of the backstory, which Jake alluded to in his post.

The night after Part I, I went to my parents begging for money. Our mortgage was nearly a month late. When I got home that evening, I saw Justin's post and subsequent tweet...I easily separated the 2 events in my head. Borrow money from parents != Community supports my efforts to go to OOW.

Easy right?

Then it came down...How could you do that when you just borrowed money?

I think you know where this is headed.

Unless I win the lottery so that I can pay back all those I owe, I will not be attending OOW this year.

While I think my T-Shirts are really cool, I know many of you (all?) bought them to help me get there. I could not take that money in good faith.

I'd like to donate it to something or someone. It's not a lot, approximately $100 as of right now.

I would like your help in deciding.

I'll repeat myself, to say that I've been humbled by these events would be a gross understatement. Thank you so much for your support...

Oracle: Day 1

I've been wanting to translate some of the concepts of Oracle to pictures for quite some time. Pictures have a way of telling a story that words cannot express.

So, inspired by Michael and his Eternal Sunshine series, I'm going to give it a go.

My First Day With Oracle


Perhaps I should have made myself much smaller? I remember what an overwhelming experience it was, "Here's your tnsnames.ora and SQL*Plus, get started..."

Learning From Failure

I think I began reading The Daily WTF about 4 years ago. I don't miss or skip a post.

I remember this one time, probably about the time I began reading the site, I had to automate a process to move files from one server to another. Originally, I had tried to create a network drive (yes, it was Windows) on the database server so that I could just use a simple Java class to read the directory and then load the files via DBMS_LOB.

I had ultimately decided on a service, but I didn't know how to write one for Windows. Then I found the Java Service Wrapper which would allow me to write the guts in Java and then install it on Windows as a service. Perfect.

Now that I had that settled, I had to figure out how to detect when a file was read to be moved. I decided on a looping mechanism, to check every minute or so, to see if a file was available. It looked something like this (I'm a tad rusty, so bear with me):

package project1;

import java.util.Date;

public class Class1
{
  public static void main(String[] args)
  {
    Class1 class1 = new Class1();
    Date d = new Date();
    long l = d.getTime() + 1000000000;
    String s = String.valueOf( l );
    
    for ( int i = 0; i < l; i++ )
    {
      //some sort of MOD "wait" here, then check for the file
    }
  }
}

It wasn't pretty, but it seemed to work.

Then I got a call from the server admin.

SA: "You've got something running on this machine that's spiking the CPU."

Me: "Really? I can't think of anything."

SA: "Well, take a look and see if you can find anything."

Me: "10-4"

Sure enough, go into Task Manager and there's java.exe hogging up all the CPU. WTF?

I just ran this on my machine and you can see the CPU start to spike:


Off to Google to see what I can find. During my research, I found mention of a small method called Thread.sleep(long). So I replaced my brilliant add 1000000 to the current date with Thread.sleep(6000), or whatever equals 60 seconds. Problem solved.

A short time later I read a post on The Daily WTF about the same exact problem (I can't find the exact post for the life of me). The "victim" did the exact same thing I did. The solution was the Thread.sleep() call.

Me = FAIL

One more short example.

Over beers, a friend (see last entry) of mine and I were discussing the failure of the North Korean missile launch. I said, "Idiots." He said, "They're going to learn a lot more from that failure than they would have had it suceeded."

Spoken like a true engineer I guess.

The point? You learn by trying. You learn my doing. You learn by failing. Whether you realize it or not, you learn. (Well, some people don't, but that's another post). If you're reading here though, that probably means you have a passion for what you do. That means that you are trying. You are learning (maybe not here specifically ;).

Here's to trying and failing and hopefully trying and succeeding.

Failed Deployments: An Index

Since there seem to be so many now, I'm creating this index page to track them. Enjoy!

The Countdown Timer - a brief not on the origin of the countdown timer adorning my site.

Good Day to Worse Day - This is the day that the countdown timer was first started.

DELETEng an entire production table.

Blowing up the Rate Application.

Another Failed Deployment...

Just before I went on vacation I (we) deployed an application so that our finance group could maintain their own rates.

We were doing between 5 and 8 critical change requests a month, unacceptable. I fought months ago to bring the rates into the data warehouse so that we could own them and eventually build this application (in APEX of course).

So what happened?

From my perspective:
1. I was rushing so that I could go on vacation without worry.
2. I was doing support work while trying to build the application.
3. I added create_date into the composite unique key without at the very least truncating SYSDATE. This created duplicate records. This was the impetus behind my post the other day named "How do you audit?"

We performed the official root cause analysis today and here's what came out of it:
1. There is no defined or official design process.
2. No official or formal design review was performed.
3. Chet sucks!

It was a good exercise. I believe I am always open and upfront about mistakes. I welcome criticism.

Length between failed deployments: 5 days

I guess it's better to get them out of the way in that short time span. It's never fun and it's always embarrassing. Ultimately, I just have to accept it and move on.