ORACLENERD twitter/oraclenerd view chet justice's profile on LinkedIn feed oraclenerd t-shirts Unemployment Clock:

  Code Comment WTF? Part 209
Found this in a snippet today:
-- ********************************
-- End of Package Body

END package_pkg ;
/
Seriously? Was that necessary? Could I possibly be under the illusion that it is not the end of the package?

Stop it.

Now.

Labels: , , ,

  3/09/2010 01:38:00 AM   6 Comments Links to this post
  EBS Oracle VM Templates
OK, now I am angry.

The other day I emailed the Steven Chan, he's the Senior Director at Oracle in Applications Technology Integration, Oracle E-Business Suite Development, about The EBS Challenge. Just a courtesy thing, I wasn't sure if he had seen it or not.

Well, today he wrote me back...

And what did he tell me?

He told me about two recent, and by recent I mean December...you know...about the time I started this whole saga of installing EBS, posts of his.

Actually, I'm not sure who I am more made at, him or John. I suppose I could excuse Steven since we have never interacted before...but John? Are you telling me John didn't know about this? John knew the pain I was going through just to create the stupid 300 GB VDI. John knew!

I can come to no other conclusion than John purposely tortured me. I hate you now John.

Alright...that might be a little harsh. Wait...no, it's not harsh at all. Yes it is.

If you don't feel like going through this exercise yourself, here is the easy way out:

--E-Business Suite 12.1.1 Templates for Oracle VM Now Available
--Using Oracle VM with Oracle E-Business Suite Virtualization Kit

Thanks John...we're not friends anymore. :)

Labels: , ,

  1/07/2010 11:25:00 PM   14 Comments Links to this post
  My EBS Install Saga - Part II
Finally!

Finally!

After just a little over a month, I've finally gotten this stupid thing installed. Read a bit of my prior trials and tribulations here.

Cost
In total, I believe this cost me about $250.
  • $80 - 320 GB Hard Drive which I didn't use for this install (but meant to)
  • $100 - 1 TB hard drive which I did use for this exercise.
  • $70 - Amazon EC2 usage. Most of the $70 is because I used a 26 ECU machine (8 dual core CPUs, 64 GB RAM)
The EC2 expenses are high mostly because I spent more time connecting and such than actually doing any work. When you have a powerful machine, work is easy (and justifiable). I could download a 1.5 GB file in 3 seconds...well, it seemed like 3 seconds. Plus, I was learning how it all worked.

As far as disk space goes, I didn't have any. Maybe a total of 300 GB to start spread on 3 different disks.

Now I need one of those big ass workstations so I can actually work with this software. Sadly, as I have priced them out, I can't get under $4K. Anyone from HP or Dell (or Apple) reading? Wanna give me one of your workstations? I'll blog about it...a lot.

Instructions
I followed John's instructions to the letter. I received the same error screen as he did

error screen

Checked the logs as instructed, found the problem
error while loading shared libraries: 
libdb.so.2: cannot open shared object file: No such file or directory
The only errata I've found is that mine was a 32 bit system and not a 64 bit system...but who cares? It works.

I followed his instructions to create the symbolic link...I would never in a million years been able to figure that out.

I'm fortunate to have used the 9i and 10g Oracle Application Server and OC4J extensively in the past, I just need to figure out where everything is located.

Resources
My goal is 2 fold
  1. Create a test environment with EBS, OBIEE and OBIA
  2. Prove (or disprove) a theory about integration with APEX I have
That means I need to have room (memory) to run another VM. Not for APEX, but for the OBIEE/DW space.

Very first thing I did was go into the database init file. I changed the sga_target from 1G to 512M and I changed the pga_aggregate_target from 1G to 512M.

Next up, I'll be trimming some of the memory allocation for the OC4J instances...but not yet. I'm just going to bask in the glow of finally getting this thing done.

I'll also read up Part 3 of John's Install Guide and Part 4 will be coming out this week, probably tomorrow afternoon.

John, you are the man! Thanks!

Labels: , ,

  1/03/2010 12:27:00 PM   0 Comments Links to this post
  My EBS Install Saga
(Yes, it is sad that I am writing this on New Year's Eve...I'm on a mission at this point and nothing will stop me).

So I've been trying to follow along with John's EBS Install series...and by trying to follow along I mean I'm still on step 0 (zero).

First, it was a lack of space. 300 GB to be exact. So I went out and bought a 320 Seagate expansion drive. I can't remember that far back, but I think other things came up and I had to requisition that space for other projects.

A couple of weeks ago, I found a 1 TB external drive (USB 2.0) for $100. Couldn't pass it up. Sweet...now I'm all set to do this thing.

Somewhere, somehow, I removed all the carefully managed installation files. Couldn't find them. So I started over. A quick reminder, it's 46.25 GB of download, which takes a total of about 8 hours...even with 25 Mbps down. I am really thankful that Kris Rice doesn't do EBS...he'd be in for a world of hurt.

OK, I'm downloading the files and unzipping them simultaneously...to save a little bit of time. Get everything downloaded and unzipped...this might be where the fun really begins.

If this is your first time, I'm using VirtualBox. The rest of the setup is like this:
-- Dell XPS M1530
-- Intel Duo Core T1555 1.83 Ghz
-- 3 GB RAM
-- 1 TB Seagate External Drive (USB 2.0)
-- Ubuntu Karmic Koala (9.10)

Not the most powerful machine in the world...but decent enough for 99% of everything. Of course by everything I mean everything but EBS.

The first time I created a 300 GB Dynamic HDD. That's the one I downloaded and unzipped all the files too. Upon completion, it had grown to 68.5 GB. OK, let's run it. Kick off the rapidwiz script, run through the first couple of screens and then it does a system check. Oops...there's not enough space. John did say that 350 GB would be preferable...I obviously didn't listen.

OK, so I could either expand that HD or shrink it and add another. Expansion is not possible (yet) in VirtualBox. I tried cloning (not sure why) and still ended up with a 300 GB disk of which 68.5 GB was used.

I then tried to use zerofree so that I could compact it (again, I have no idea why I was trying to do this). Those attempts were very unsuccessful because I couldn't figure out how to make the HD read-only. I did figure out a way later...just add a bad line in /etc/fstab and then you can run zerofree. What was the poing of compacting it though? No idea.

In one of the more recent versions of VirtualBox (I think 3.0), you could add a second HD. At least that's when I noticed that ability. So I created a new Dynamically-expanding 300 GB HD and attached it through the VirtualBox GUI.

Now I was stuck trying to figure out how to mount the stupid thing. Mr. Brad Tumy to the rescue.

Kick off the rapidwiz script, get through all the setup screens and it's installing...I left it over night...and one more night...78 GB had been written to disk.

I killed it.

How about just creating a 300 GB Fixed HD? Tried it...ran for 24 hours...about 60 GB had been created.

I killed it.

Maybe I could use Shared Folders. I consulting The Twitter and was reminded of my recent fun with Shared Folders.

Hmmm....How am I going to do this?

Then an IM with a colleague where he asked about The Cloud. It would be nice to have a test environment, as we are all remote, that we could share. I gave him 2 suggestions, I could buy a beefy machine and host it here at home or I (we) could try to do something in The Cloud.

That got me thinking...The Cloud? Hmmm...let's see if I can install OBIEE there.

I was immediately sold when I downloaded the OBIEE software (1.5 GB) in like 3 seconds (14 or 15 Mbps). I got OBIEE installed and running on Windows...but when I was creating the Instance, I noticed some pretty beefy boxes.

Maybe I could create the 300 GB VDI in The Cloud?

Yes, absolutely!

So I fired up a 20 ECU Windows Server Datacenter (2008?), with 8 dual core CPUs and 7.5 GB of memory. It took 2 hours to create the VDI.

Sweet!

Now...how in the hell am I going to get that 300 GB down to my machine? Since my local drive was attached, perhaps I could just drag and drop it there. Nope...didn't like that.

I'll zip it up!

Using 7zip, I was able to compress it to a very modest 360.9 MB. That's got to be close to a million percent compression right?

Downloaded that via IIS (had to set that up as well) and began to unzip it. First 5 GB took about 5 minutes, so I'm thinking 5 hours. Then it started to crawl. I was at 75 GB after 10 hours which was going to put me at 40 total hours. But it looked like it was exponential...that 40 hours probably would have ended up around 1 million hours. I don't have that kind of time.

I killed it.

Consulted The Twitter again.

@TedC and @jtdavies offered up some suggestions. @TedC told me to use Amazon S3. @jtdavies told me to split it up and use rsync.

Being incredibly impatient and quite possibly very stupid, I "split" the difference. Using my 400 GB attached Volume, I decided to use The File Splitter to split the files. 10 MB chunks. 3000 some odd files. No worries, I'll just zip up every 100 or so, download and unzip.

Until I ran out of room.

*%##@$!!~!@#$%#$#$**&%(*##@@!!!!!!!

Shoot me...now!

I created a second (well, third) volume, 315 GB, which would be the target for the split files. Attached it and began the split...again. This time, 10 GB chunks. I wasn't worried about the size as they compress very nicely. Two of those equals about 24 MB of download.

So I'm almost done with that...about 95% complete. I just have to download 2 more zipped files.

The funner part will be getting those put back together...if it's even possible. I've read up on cat, I just hope it comes through.

If you have any suggestions...or you want to come over to my house or remote into my computer and help me...that would be awesome! :)

Labels: , ,

  12/31/2009 11:30:00 PM   0 Comments Links to this post
  Here's My Recommended Technology Stack

Labels:

  12/04/2009 04:40:00 PM   0 Comments Links to this post
  RTFM Minimalism
Just found this at @rnm1978's blog and found it quite amusing...and poignant.



I'm not sure who that is, but it looks like Kathy Sierra, the sentiment expressed in the slide sounds like her as well.

Labels: ,

  12/04/2009 03:39:00 PM   1 Comments Links to this post
  Dilbert: You're Fired!

Labels: ,

  11/16/2009 11:40:00 AM   0 Comments Links to this post
  Security Tools Malware
My parent's neighbor called me this afternoon...his computer had a virus.

There was no virus software installed.

I told him to try AVG (which makes me want to put parenthesis after it with a GROUP BY clause), scan the computer and then call me back.

He did call back, but he was unable to download the software. He kept getting messages that his computer was infected and that his credit card details were being delivered to the 'tubes. Just bring it over and I'll take a look.

I tried opening up Task Manager, no deal, Security Tools identified it as a virus. WTF?

I tried opening up a shell window. Ditto.

Here's what "Security Tool" looks like:



Hmmm...I could use the Ubuntu Live CD and try to remove it...you know, since I'm an expert and all that.

So I opened up a terminal and started poking around the file system (/media/disk). Nothing jumped out at me. So I googled and found this entry. It suggested installing AVG for Linux and then scan the Windows disk. So I did. Took about 2 hours to complete and it came back with nothing.

I gave up on trying to solve the problem with Linux. I really thought I would be cool. Oh well.

So I did some more research and found this entry on "Security Tool." It suggested downloading MalwareBytes’ Anti-Malware, but it was impossible to do anything. I then found this post on the Symantec forum which suggested opening up Task Manager as soon as you login. That seemed to work. Found the process and killed it immediately.

I then installed MalwareBytes', scanned the computer and removed the infected files. Voila! It only took about 6 hours from start to finish. I will get a free lunch out of it though...I'll take all the free I can get these days. I also updated Windows, installed Firefox and avast! so we can hopefully avoid these same issues in the future.

On a side-but-related note, I ran across an infected site last week and it looked like this:



I found it quite amusing...but probably authentic enough to fool a number of people.

Hopefully some poor soul out there will get good use out of this.

Labels: ,

  11/11/2009 11:52:00 PM   2 Comments Links to this post
  Database Changes

Labels:

  11/02/2009 12:00:00 AM   0 Comments Links to this post
  Calculating on the Fly
From the mind of John Parades, author of The Multidimensional Data Modeling Toolkit.

Labels:

  10/19/2009 12:27:00 AM   0 Comments Links to this post
  Learning By Breaking
I've spent the last couple of days building and rebuilding my sandbox database.

I can't get it just right.

Last night while running my scripts, none of the foreign key clauses would work. Primary key doesn't exist? WTF?

Do a quick check on the count of indexes...it's about 10% of what it should have been.

Oh yeah, when using REMAP_TABLESPACE, it helps to include the index tablespaces as well. $#*@~!

So I queried all the index tablespaces and put them into my parameter file for impdp.

Only to have my database freeze about 70% of the time. Being just a dba and not a DBA, I figured I'd just add a datafile. OK, so the first time I created it is was 1GB. I didn't bother to resize it.

Then it hung up again. (Of course I'm not bothering to find the root cause of the issue, I'm just brute forcing it). Somewhere in the index creation (reading from the console output anyway).

Killed it again.

Cleaned out the schema again.

Rinse and repeat a couple more times.

Then I decided to drop a datafile, the small one I created up above. Yes, I could have resized it...but I'm just a rookie. Used the OFFLINE DROP, shutdown the database and deleted the datafile.

Reran the impdp.
Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options
ORA-39001: invalid argument value
ORA-00376: file 6 cannot be read at this time
ORA-01110: data file 6: 'USERS02.DBF'
Oops.

OK, let's drop the tablespace and try again.
DROP TABLESPACE users INCLUDING CONTENTS;

ERROR at line 1:
ORA-12919: Can not drop the default permanent tablespace
OK, let's create a new tablespace
CREATE TABLESPACE USERS2 DATAFILE 'USERS2_01.DBF' SIZE 10G AUTOEXTEND ON;
Now I'll make that one the default
ALTER DATABASE DEFAULT TABLESPACE USERS2;
Now I'll drop the USERS tablespace
DROP TABLESPACE users INCLUDING CONTENTS;

ERROR at line 1:
ORA-22868: table with LOBs contains segments in different tablespaces
Killing me...

So I dropped all the users (just a reminder, this is just a sandbox) and tried it again.
SYS@TESTING>DROP TABLESPACE users INCLUDING CONTENTS;

Tablespace dropped.
Finally!

For whatever reason, I'm a glutton for punishment. I don't like the look of "USERS2" so I want to go back to "USERS." Let's go through this again.
ALTER DATABASE DEFAULT TABLESPACE USERS;

Database altered.

DROP TABLESPACE users2 INCLUDING CONTENTS;

Tablespace dropped.
Add a second datafile just for fun:
ALTER TABLESPACE USERS ADD DATAFILE 'USERS_02.DBF' SIZE 10G AUTOEXTEND ON;
Done.

Obviously no one should let me do this sort of thing anywhere near a production database. I know just enough to be really, really dangerous. It is find having to go through this (painful) exercise once in awhile in a closed environment. Helps to understand a little bit about how the database works. I'm not sure what exactly I did learn, other than I am a moron...but it was...fun.

Labels: ,

  9/21/2009 06:37:00 PM   0 Comments Links to this post
  Tri-lingual?
A friend of mine, a couple of jobs back, told me this great joke.

We were walking out to our cars after a fun day at work and he proceeds to tell the joke.

Srini: "What do you call someone who speaks 3 languages?"

Me: "Tri-lingual?"

Srini: "What do you call someone who speaks 2 languages?"

Me: "Bi-linqual?"

Srini: "What do you call someone who speaks 1 language?"

Me: "I don't know, mono-lingual?"

Srini: "American."

Srini was not American born, nor is that is full name. He also spoke 3 or 4 languages and he was giving me a hard time for being "mono-lingual."

His full name consists of a whopping 26 letters. I went to great effort to learn how to pronounce his name prior to his interview. Took about 4 weeks to get it right.

Not really sure why I recalled that particular joke today...but it was a good one. I miss Srini...such a good sport and a fun guy. If you ever run into him, ask him, "Can I tie that for you?" and see the different shades of red he turns.

Labels:

  9/08/2009 12:25:00 PM   2 Comments Links to this post
  The Bit Bucket

Labels: ,

  9/03/2009 12:51:00 AM   0 Comments Links to this post
  Oracle: Day 1
I've been wanting to translate some of the concepts of Oracle to pictures for quite some time. Pictures have a way of telling a story that words cannot express.

So, inspired by Michael and his Eternal Sunshine series, I'm going to give it a go.

My First Day With Oracle


Perhaps I should have made myself much smaller? I remember what an overwhelming experience it was, "Here's your tnsnames.ora and SQL*Plus, get started..."

Labels: ,

  8/31/2009 02:48:00 AM   2 Comments Links to this post
  Stories of LC
LC being Little Chet. And no that is not some perverted reference...

He said something pretty funny the other day, which reminded me of a bunch of little stories about him. Since I post so many stories about Kate, I figured it's about time I do one on him.

Policy
I get a cup of ice cream the other night and he immediately wants some too. Mom asks if he has had any that day, and he replies, "Yes."

"But there is no policy on ice cream."

Me: "No What?"

LC: "No policy."

Kris and I look at each other and just start laughing. He's 6 years old and he's talking about policy? He's been hanging out with his mother way too much.

To make sure we knew that he knew what he was talking about we asked him what he meant.

LC: "Well, I can have 2 popsicles a day. But there's no limit on ice cream."

Wow.

The Letter R
In Pre-K 4 a couple of years ago, his teacher asked the class to say words that began with the letter R.

The teacher started them off, "Rapunzel."

"Rabbit!"

"Rainbow."

"Chet, do you have one?"

"Craphead"

Sea World
After a day at Sea World with my parents and 2 small kids, we naturally gravitated to the Hospitality Suite. For those who don't know, Anheuser Busch used to own Busch Gardens and Sea World. Both parks, or both companies as there are more than one park for each, serve free beer. There is a two beer limit.

Anyway, Kris takes this opportunity to spend some time with Chet and she walks him over to the Clydesdales (while I drink her beer).

Mom and son, holding hands...a perfect picture.

"Mom, that's the biggest penis I've ever seen."

All the people around them began laughing...OMG.

Jackass
I believe this happened in 2006, while in Gainesville. I would drive Chet to school, St. Patrick's Catholic school. This was always a concern of mine because I have never been too careful with my choice of words around him.

My old boss had once shared a story with me about her daughter, at this same school, saying JFC (Jesus F$#%*&% Christ). I always thought the same would happen to me.

It never did. He had amazing control (besides the craphead comment above) at school.

At school.

Not at Publix though.

We're shopping one day with him sitting in the basket when this woman comes out of the aisle and gets in front of us.

"Move that F$#%*&% Cart!"

It was about all I could do to maintain composure. He was 3, maybe 4 at the time. Thankfully, the woman did not hear him. If she had, I probably would have fallen down from laughing so hard.

However, I managed to compose myself and give him a stern lecture about how inappropriate that was.

I couldn't wait to get home to tell Kris.

The First Word
I've probably talked about this before. Yes, I have. Here. The similarity between my name and shit. I've heard it all before...I actually used to introduce myself referencing Weird Science.

Me: "Hi, I'm Chet"

Them: "Ted?"

Me: "Chet"

Them: "Jeff"

Repeat this a few more times.

Finally I'd say, "You know, "Chet" like the guy from Weird Science?"

Them: "Ohhhhh...yeah, the big pile of shit!"

Me:...

That reference is only useful with people of a certain age. Twenty-somethings don't really get it.

Anyway, I always thought I could get away with teaching Chet that for his first word. Chet. Shit. Pretty close right? I can't remember if he actually said it for his first word...it's a funny story none-the-less.

I know I'm not the only one with funny child stories. Anyone care to share?

Labels: , ,

  7/08/2009 11:18:00 PM   4 Comments Links to this post
  Classic: Application Developers vs. Database Developers II
The original (with all the fun comments) can be found here. Originally posted on December 9, 2008. This is the "followup" to yesterday's post.

You can read the first article here. My application developer friend, Mr. M, emailed me and another fine gentleman this little blurb recently:

Mr. M:
OH YEAH BABY!!! TEN TIMES FASTER!!!! YEAH!!!!!!!!

Hey seriously, what a tub of shit Oracle is. Where does this myth come from that it's such a great platform? Their client tools suck balls and it's generally just a pain in the ass to work with from a developer's point of view. But devs for some reason are under this impression that from thew server perspective it's rock solid and performant. Well, it may be solid, but it's a fucking turd. Our dba here - definitely an Oracle guy - has been tasked with looking into moving us onto an oss db. He basically refuses to even look at MySQL, stating that it's a mickey mouse worthless pile of shit (can't really argue with him there lol), so that basically leaves Postgres. So it's been a few weeks now, and he will basically admit now that Postgres completely waxes Oracle as far as performance goes. LOL We run RAC in production too. He's looking at failover, replication, blah blah blah now, we'll see what the verdict on that is. Oh, and Oracle AQ? That's a worthless pile of shit too. Why do they tack a fucking message queue that doesn't play nice with jms onto the fucking database, which is probably already overworked? Oh wait, that's right, they're in the business of selling per cpu licenses! Cocksuckers.
This was prompted by a recent Oracle email blast about the Exadata storage system/Warehouse.

As I did before, I'll just put the email here.

Me:
Agreed, their client tools aren't all the great. Which ones are you using?

I use SQL*Plus (naturally), SQL Developer and JDeveloper. The latter 2 tie in very nicely with Subversion. With JDeveloper, when I want to try out Java, it's got a pretty good suite of tools.

Oracle starting out catering to businesses, Microsoft started with the consumer. Oracle has made pretty good strides in making their software more usable while Microsoft has made theirs more scalable.

I haven't used AQ a whole lot and definitely not with Java. I do know that it's billed as JMS compliant.

Postgres has it's place and so does Oracle. It is a great platform if you know how to leverage it. PL/SQL is a 3GL (or is it 4) and you can write entire applications using it...and if you do that right, an incredibly robust and scalable application at that.
Mr. M:

"It is a great platform if you know how to leverage it. PL/SQL is a 3GL (or is it 4) and you can write entire applications using it...and if you do that right, an incredibly robust and scalable application at that."

NO!!! NO!!! NOOOOO!!!

I want to beat people like you who say this with a ball pean hammer. There are only a select few individuals on this earth who can write and read application logic written in SQL. AVOID THIS ANTI-PATTERN AT ALL COSTS! What is it with you f_cking database guys??? Why do you always want to stuff as much crap into the db as possible?

DUDE! SQL IS A PILE OF SHIT FOR EXPRESSING APPLICATION LOGIC!!!

It's fine for querying and manipulating sets of data, in a relational database. But it is a worthless sack of shit for expressing application logic!

I'm having to dig through this f_cking abortion right now because some Oracle f_ckhead thought "you can write entire applications using it!" Blog that, mofo!
This was followed by a package he had been working on. I wouldn't say it was the greatest, but it wasn't all bad either.

Me:
goodness gracious.

"DUDE! SQL IS A PILE OF SHIT FOR EXPRESSING APPLICATION LOGIC!!!"

disagree (naturally). It's incredibly easy to do, you just don't know how yet...and it seems even the Oracle professionals out there don't either.

I'll tell you this, the crazier the SQL or PL/SQL needed to derive and manipulate data the poorer the design. Start with a good design and it all becomes very simple.
Of course note the use of "naturally" in my lexicon. Thanks Jake.

Mr. M:
well dude, we are back to our old discussion - you arguing that procedural sql code is perfectly fine for building apps, and by extension, that the last 20 years of computer science have basically been a misguided lost journey down the meandering, fruitless trail of oop. um.....no. select this from that. otherwise keep that sql crap caged up where it belongs.

btw, do the db guys here suck? seriously. i'm not competent enough to judge. (to be fair, apparently that crap i sent you is fairly old stuff, but still....)
Me:
I would say, based on limited knowledge of software development, that the OOP movement was started because the database (specifically Oracle) was not mature enough to do what was needed. Plus, I seem to recall that the OOP movement was supposed to have solved all the world's problems by now.

It's further propogated due to the needs you specified in our discussion that day at WellCare (i.e. performance). I still believe that if application developers better understood how a DB works (Oracle, MySQL, SQLServer, etc) they would end up writing less code. Database constraints alone force you to write less (and better) code simultaneously ensuring good data.

The code that I did look at (first 1000 lines or so) isn't great.
1. With all those IF THEN ELSE statements it's telling me that there's probably a better way to store the data. Looks like they're missing an attribute that should be contained with a track.
2. using Object (PL/SQL) types to store data in the way they seem to be doing it is not the best way. Again, probably a design issue.
3. When you do something like this:

   UPDATE pb_album_metadata
      SET primary_digital_flag = 0
    WHERE album_id IN (SELECT b.album_id
                       FROM (SELECT a.album_id AS album_id,
                                MAX(a.album_id) OVER (PARTITION BY a.standard_upc) AS latest_album_id
                             FROM pb_album_metadata a
                             WHERE a.standard_upc = g_album_tab(1).standard_upc ) b
                       WHERE b.album_id <> b.latest_album_id )
         AND primary_digital_flag <> 0;

They should probably have considered end_date as an attribute of the album metadata. While analytic functions are pretty cool, they're more for analyzing (OLAP) and not OLTP environments.

That's for starters and without table definitions...
Me (again):
oh yeah...and PL/SQL is/was built on top of ADA, FYI.
Mr. M:
"I still believe that if application developers better understood how a DB works (Oracle, MySQL, SQLServer, etc) they would end up writing less code. Database constraints alone force you to write less (and better) code simultaneously ensuring good data."

Huh? What are we not understanding? What would be an example of a constraint that would force us to write less and better code?
Me:
CHECK, NOT NULL (same as CHECK) and FOREIGN KEY constraints all fit into that category.

Instead of having to check if a value is NULL when submitting it to the database, just submit and let the database throw an error, which you'll conveniently capture in your Exception block. Same goes for CHECK constraints, columnA should be either one of three values (test, test1, test3), you won't have to check that in your application code, just catch the exception. FKs insure good data (proper relationships).

A different perspective. If you are going to pay for it, use that damn thing. If you don't care, don't do it. But over the course of an "enterprisey" application, you'll end up writing more code and make maintenance that much more difficult (did I do it there?). Just those constraints will force you and the teams of application developers to put good data into the database.

You can still do it in the application of course (form validation is a good place)...
Mr. M:
Ahh, jeez dude, I wasn't sure if you were referring to the literal "constraint" or not.

Dude, even f_cksticks like redacted I think have a decent enough understanding of when and how to use db constraints. It's when you get into things like cursors or cost plans of subselects and anonymous tables (i think that's the name for it - where I say select something from (select some other crap). Then we defer to db gurus like yourself.

But dude....."you won't have to check that in your application code, just catch the exception".......uh, don't ever repeat that off our little email group. And definitely don't go posting that on your blog. F_ck me man, it's a damn good thing we keep you db folks caged up in that rdbms box....
Me:
So we've had this discussion at work...for a high transaction system, do Java/C/etc handle exceptions well or what?

Why is it bad to deal with exceptions rather than coding to avoid them?

I highly doubt even redacted understood database constraints...him and his cohorts believed all database software would be commoditized and MySQL would be king in short order.
Mr. M:
"for a high transaction system"

Or for any system really....

To use your example of check constraints (is this value one of....) or not null constraints, checking these rules in the Java code and in the database code would seem to violate DRY. But we do that alot, and it is acceptable in certain cases. For instance, we also probably violate DRY if we're checking this same rule say in Javascript on the front end. But we accept this tiny violation of DRY because it tends to make for a better user experience and as a performance gain too, for we avoid the round trip to the server. Now, what your advocating here is close to the same thing. You're basically saying, don't check for the not null constraint in Java code, just go ahead and hit the database, let the database throw back an exception to the Java code, presumably correct the problem, and then make another roundtrip to the database again. Dude, what are you thinking?!? This to say nothing of the fact that this also could be considered a violation of Fail Fast, and a violation of Defensive Programming - what happens if the dba forgot to add the not null constraint in production?

Dude, listen to this guy. For a "high transaction system" basically you want to treat the database, as much as you can, as just a dumb data holder. A f_cking dumpster, that you just throw sh_t into and pull shit back out, using no vendor-specific features if at all possible.

Again, for we've had this discussion, but even in everyday apps, not just on Wall Street, the database is the bottleneck. And the database is the hardest layer to scale. So given those facts, you should design your app to do as little work in the database as possible.
I was laughing at this point because the link above points to one of our consulting architects (I'm not really sure what his role is at this point).
Me:
i agree in any application that you want to minimize the number of round trips...

shocker...he's one of our architects. he's spot on in many instances, but...

database is the bottleneck because people don't know how to write SQL. I'll certainly concede the wall street applications (for the time being anyway), but the rest of us with what we do on a daily basis...Oracle will scale far beyond the demands they place. When that bottleneck shows up, 9 times out of 10 some dumb-ass c#/java guy thought he could write a better query than i. besides, what's the idiot doing anything but a simple join anyway? probably poor data model to start with...and we're right back where we started (sort of).
Mr. M:
"database is the bottleneck because people don't know how to write SQL.....some dumb-ass c#/java guy thought he could write a better query than i."

Dude, I'll grant you, people don't know how to write SQL, myself included. But that's not always why the database is the bottleneck. I think it's safe to say that's not even the majority of the reason. Yes, there are apps written by people who were just idiots, and they needlessly pummel the database, but that's definitely not the majority of scenarios. At my work the database is the bottleneck, and we run RAC in production. It's my understanding that even with RAC, there is a limit to how much you can scale that out. But any layer up from the database we are basically unlimited in how much we can scale that out. So it's very easy to stick another Apache box in front, or fire up another Weblogic server. But I can't do that with the database. We have 24 Weblogic servers for redacted. The database is the bottleneck. And we don't have shitty sql code in the app. In fact, we have very few hand-written queries anywhere in the app. Persisting something to a database is really a low-level concern that as an application developer I shouldn't even have to be bothered with, except for the rare corner cases where the persistence abstraction I'm using is just a little bit too generic to handle things effectively. And we don't use these ORMs because we don't know how to write sql. To effectively use an ORM tool requires a deeper understanding of sql and databases than simply being able to write solid SQL code. (Not saying Java devs who use ORMs know SQL better than a dba, just that it requires better sql skills than is required of a Java dev to simply write JDBC/SQL.) Now, before you try to tell me that my ORM library isn't able to write as efficient of sql code as a dba of your caliber, keep in mind that ORM tools are pretty advanced. They're able to intelligently do things like batch sql updates, and let you apply transactional semantics much easier than with raw jdbc/sql. But the overwhelming reason developers have so thoroughly adopted ORM is because Structured Query Language is such a nasty piece of shit for expressing application logic. SQL is a declarative, procedural language. It's totally unsuited for writing application logic! This, more than anything else, is why organizations and dev teams should seek to restrict what is handled within a relational database as much as possible - because the programming interface to it is a fucking ancient backward dying dinosaur.
Mr. V (note, not Mr. M):
My 2 canadian cents:
The polyglot approach "... use different languages for different domain ..."
Database was developed to manipulate data and should remain there.
General purpose language was developed to encapsulate logic and should remain in that domain.
You should not use DB to encapsulate business logic (in my opinion) no more than you would use HTML to create complex logic.

While Java, C#, Python, etc are described as General Purpose languages, they, too, are really domain-constrained. Their expressiveness are confined (and should be) to express low-level, tersed, explicit, verbose, and repetive logic (if that makes any sense). Languages such as these are more suitable for low-level abstraction on top of which, richer, more expressive languages can be built. We are seeing this now with the emergence of languages on the JVM (and arguably on the .Net's CLR).

I think SQL as we know will take a back seat and a smart developer somewhere will create a new domain-specific language that lives on the VM and will push the SQL expression out of the RDBMS and closer to the code that needs it. We are not there yet, but Microsfot is trying (see LINQ and all ORM papers). This is not to say that there won't be isntances where tuning in the SQL-native language won't be necessary. However, 80 to 90% of simple CRUD cases will be handled closer to the logic code that uses the data code.

Again, that's my 2 canadian cents... I could go on. But I have a meeting with redacted.
Mr. V, I believe, is a little bit more sensible. Mr. M on the other hand is just trying to rile (sp?) me up.

Me:
Someone will probably create something like that, but it still gets at the heart of one of my arguments, many developers don't know how to use a database thus will go to any means to circumvent it. Embrace it I say.

Ultimately for me, it comes down to simplicity. I can write an application using PL/SQL that will scale very well for 90% of the solutions. Of course that doesn't include the "fancy" javascript/Ajax stuff needed for the UI. In my ever so humble opinion, this is a much better solution for a business in the long run.
1. You're more likely to change the middle tier than the database tier. Java, asp, Ruby, what's next?
2. Fewer moving parts means lower cost of ownership, even with the CPU costs. One person, one, was able to build and maintain a 350 page website. Be hardpressed to do that with the more expressive languages.

I think all of us are purists and very passionate about what we do. It's probably the main reason we get along. I thoroughly enjoy these conversations because it does force me to think...and that's always a good thing.
Mr. V:
Haaa chet.
You sorta made my point than diverged away. Building an app in all PL/SQL is dangerous. It's no safer than me building an app in all Java. I can build very limited app in all Java. The moment I need to interact with other domain (UI, data, low-level native code, etc), I have to switch to something that is closer to what I am trying to do. If I need to create UI, I will pick a ui-centric environment, when I need to talk to DB, I will pass connect to a db and send in my SQL, and so forth. I will use Java as the glue to bring it all togher.

In the end, we may be saying the same thing, but using different accent. O well.
And that's where it ended. I must say it's always fun. Mr. M and Mr. V are both very smart individuals and I highly respect what they do. We have different perspectives...but I think they listen, if only a little, as I listen to them. Their voices creep up on me especially now...which is definitely a good thing.

Labels: , , , , , ,

  7/07/2009 05:15:00 PM   1 Comments Links to this post
  Have You Ever Referred To Yourself As An Oracle Nerd?
<shameless self promotion>
Now's your chance. For a limited time only you can join the ORACLENERD group on LinkedIn and have the ORACLENERD logo display on your profile!
Recruiters will love you. Future employers will understand your dedication to you craft. Friends will envy you.

Just send $19.95 to...act now and it's only $9.95! Be one of the first callers and...
</shameless self promotion>

Labels:

  4/24/2009 12:03:00 PM   6 Comments Links to this post
  projectEUREKA
An old friend of mine who I worked with on Oracle a few years back has, with some friends, just launched a new website, projectEUREKA
...is a collaboratively edited site for problem solvers/inventors - regardless of problem or field.

We are a bunch of math enthusiasts who decided to create a framework for submitting and solving problems. However, the framework is not limited to math problems, any problem, puzzle, or trivia question can be submitted to project Eureka. Similarly, for people who prepare for various tests project Eureka offers the possibility of grouping multiple problems in a test
It sounded eerily familiar to something Chen Shapira linked up here awhile back, Project Euler.

If you get a chance check it out.

Labels: ,

  4/13/2009 11:35:00 PM   0 Comments Links to this post
  April Fools!
Suffering from a bout of writer's block I impulsively decided to post that I had lost my job again. I figured it would be somewhat plausible given my recent past. I think it worked, a little. I realized that I did tag it "funny" but left it on there. Notably, Gary caught it.

I did email Lewis and Dom privately to tell them I was being funny. Bradd caught it himself going so far as to delete his original post.

I started to worry about jinxing myself, you know, the self-fulfilling prophecy sort of stuff? One way or another, it's out of my hands. It was funny to me at least. I think Jake chuckled a little bit but didn't want to acknowledge my small prank.

I was paid back by LC when I got home...he popped out of the bushes and scared me. Followed by another sneak attack shortly there-after.

Happy April Fool's Day!

Labels: ,

  4/01/2009 10:06:00 PM   8 Comments Links to this post
  Not Again...
Seriously...could I have a worse year?

I've been laid off again!

I'm starting to get a complex...
Mr. Justice,

Due to the economic downturn...yada yada yada

Labels: ,

  4/01/2009 10:40:00 AM   5 Comments Links to this post
  The Breakup
No, I didn't lose my job.

Funny cartoon though:



From Timo Elliot via Dain Hansen.

Labels: , ,

  3/27/2009 01:18:00 PM   0 Comments Links to this post
  Random Things
No Oracle stuff here, just feel like writing.

Kate
Kate, my little monster, decided to go to the hospital again. Penuemonia (yes, I spelled it wrong, that's how we say it around here). Last week her lungs were beautiful, probably the best doc has ever seen them. Her head was another matter, not a single black spot to be found. Diagnosis: sinus infection. Apparently it traveled down to her lungs. Hopefully it will be a short stay.

Words
"Publicly?" Why do I want to spell it "Publically?" Pub-lick-lee.

Kris and Little Chet
Kris let Little Chet watch Terminator 2 last night. Seriously? I watched most of it with him and there really isn't that much gore. A lot of bullets flying, but it's 2 robots shooting at each other right? I came down to check on him during the final scene of the movie when the Terminator is put into the super hot molten stuff. He was crying. He didn't want the Terminator to die.

Coaching Baseball
I've been "coaching" Little Chet's 5-6 year old baseball team. It's coach pitch the first 5 then bring out the tee. Hands down the best stress reliever of all time. I get to yell at a bunch of kids for and hour and a half. Is there a better way to live? I don't think so.

Me Yelling, neigh, Screaming
"Come on! Hustle of my field!"

"Quit playing in the dirt!"

"That's why you wear a cup!"

"I'll hit you with this tennis ball if you walk off that field again!"

"No wrestling!"
I think the parents enjoy the show I put on...at least I hope they do.

Thanks for listening.

Labels: , , , ,

  3/24/2009 09:04:00 PM   2 Comments Links to this post
  Fun With Linux
I write this from my first successful Linux installation, Ubuntu Intrepid Ibex. It started innocently enough after reading this tweet.
Is a bad that I uninstalled Internet Explorer from all my family members pcs in the last 2 weeks and replace d it with firefox.
It's fairly innocuous, but it got me thinking about installing some flavor of Linux on my son's old laptop running XP Pro. Maybe it was the sneaky factor?

So last Saturday I went to Wal-Mart and bought an 8 GB San Cruzer flash drive, downloaded the ISO image from ubuntu.com, unzipped it and put the files on the flash drive. Plugged in the flash drive to my son's computer and waited like 20 minutes for it to boot up. It's an old laptop with 256 MB Ram, and I'm not sure what the processor is...but it's slow.

When it finally did come up, I tried clicking on the Install icon on the desktop and it just sat there...for hours...and hours. Finally coming up with some gnome applet error. By the way, the scenery is quite nice:

From oraclenerd


So I sat there for the next 24 hours trying to figure this thing out. Finally, I just installed it from the wubi.exe file supplied in the ISO via Windows. Weird thing about that is it went out to the 'tubes to get the files when it had them right there. Oh well.

Installation went swimmingly. Had Ubuntu running just fine, a bit better than Windows ran on that machine. The update manager told me I needed had updates so I started that process. As I sat around I started to look around (very dangerous). I'm not much of an OS guru, I know enough to get by and that's it. So I'm looking at the files that I left on the system and decided to start removing them. Windows folder? Bah, don't need that. Trash. NTDETECT.com? Trash. boot.ini? Trash. The only folder I did leave was Ubuntu and Documents and Settings (for the saved files). After the update manager completed I needed to restart, so I did.

Can't Load Operating System or some such nonsense. What? I just installed Ubuntu! Effing Linux! Can't these people get it straight? No wonder no one uses it!

Naturally, I'm an idiot.

Had it running on Sunday night and Monday when it came time for my son to get on the computer (after school), it was broken. I received a few irate IMs from the wifey in that regard. Ugh.

Found the XP recovery disk that night and tried to update the boot.ini and NTDETECT files (among others) to no avail. Began re-installation of Windows. Joy. Maybe I learned my lesson this time?

I decided to install Ubuntu on my old machine at home too. That was cut short because I had a very old Graphics card (Diamond FireGL2) that didn't work with Ubuntu. Perhaps I should get a newer graphics card?

In the meantime, I decided to install it on a 3rd machine, my other old box. This has been my mainstay for the last 7 years (yikes). Dell 4300, Pentium IV, 512 MB Ram. There are two disks on this machine, the first of which has 2 partitions, one for the OS and one for Data.

I successfully install it, reboot and am then prompted with a GRUB prompt? WTF? I can't remember the details as it all seemed to happen so quickly. Messing around there I played with the HIDE/UNHIDE commands, MAKEACTIVE, etc. Reboot. Error Loading Operating System. WTF? Come on!

XP Recovery Disk to the rescue. Getting to the command line I quickly noticed F:\WINDOWS. F? How'd that happen? Oh wait...HIDE...MAKEACTIVE...great. Spent the next few hours trying to replace the boot.ini file, but it wasn't recognizing the partition so it didn't matter. I finally just decided to delete the OS partition and re-install Windows.

Finally, finally, here I am. Ubuntu installed on son's computer, check. Ubuntu installed on my (first) old computer, check (still need graphics card). Ubuntu installed on my (second) old computer, check.

I decided not to delete any windows system files this time.

Next up, I need to figure out how, on restart, to have Ubuntu the default in OS selection.

Labels: , , ,

  1/12/2009 04:49:00 PM   7 Comments Links to this post
  2009 New Year's Resolutions
I was tagged by @crisatunity here, so I'll do this. Hopefully The Curmudgeon isn't reading...

These are professional resolutions, not personal (I'm fat, so guess what #1 is on that list).

1. No brainer here. Go the entire year without losing my job. If I leave voluntarily, that doesn't count. I'm just shooting for no firings, no layoffs and no furloughs.
2. Become proficient in OBIEE. I've been given a second chance so to speak. Our DW guy decided now would be a good time to start his own consulting company and I am the lucky recipient of his job. Bonus: Datawarehousing is all the rage. OBIEE is a great tool to have in the tool box so I consider myself lucky on more than one front.
3. Build a PL/SQL only test harness. I don't really know why, but I want to give this a go. I've used SQLUnit (and like it) and a little bit of utPLSQL, but I think there's a lot to learn here. Specifically, how to generate and run code. I've never really done that, except for a small DROP <OBJECT> script now and again.
4. Contribute something tangible to the Oracle community. So I blog, cool. I've done an ApEx presentation, sweet. I would love to be able to contribute an application (a test harness perhaps?) or some sort of library. I don't know exactly what it is yet...

Four is good enough for me this year. One will be the one I focus on most...I just hope it doesn't become a self-fulfilling prophecy. ;)

So, who shall I ask to join us? Jake has already done like 33 end-of-the-year posts...I'm sure one of them included some form of resolutions (predictions actually, it qualifies though). Here we go:

1. Bradd Piontek || (twitter, blog)
2. Don Seiler || (twitter, blog)

I've paired those 2 guys together because I'm pretty sure they know each other. Watching them on Facebook and Twitter go at it keeps me laughing all day. For instance, tonight on Facebook Bradd posted a picture of him drinking something out of a brown paper bag and asking the community to guess what he was drinking. Mr. Seiler said something along the lines of "shame and disappointment?"

3. Dan McGhan || (twitter (loser), blog)

Dan was my "co"-presenter in September on ApEx. I made him do the whole thing, mostly because he heckled me (i.e. answered all the questions) in my first presentation.

*No drinking went on during the writing of this post.

Labels: , ,

  1/02/2009 11:16:00 PM   5 Comments Links to this post
  Countdown Timer Part II
Once before I had a countdown (well, countup) timer to mark the time since my last production incident (one where I screwed something up). It was a way to poke fun at myself but also a steady reminder to be careful.

In that vain, I've created a new one marking the consecutive time I've been employed! As noted in my previous post, I've had difficulty holding on to jobs this past year. Three times in fact. Fired once, laid off twice and brought back twice (by the same company).

Here's to not having to reset this effin clock in 2009!

Cheers!

Labels: ,

  12/30/2008 08:43:00 PM   5 Comments Links to this post
  Exception Handling WTF
In homage to Oracle WTF.

Trying to remove references for a table that no longer exists (a view was created in it's place), I found this little gem:
OPEN c FOR
  SELECT 'INVALID USER'
  FROM dual;

  RETURN c;

It was nested in an IF-END IF block.

Instead of just throwing the exception or return 0 (zero) rows, this was the preferred method.

Awesome!

Labels: ,

  12/23/2008 12:43:00 PM   2 Comments Links to this post
  Application Developers vs. Database Developers: Part II
You can read the first article here. My application developer friend, Mr. M, emailed me and another fine gentleman this little blurb recently:

Mr. M:
OH YEAH BABY!!! TEN TIMES FASTER!!!! YEAH!!!!!!!!

Hey seriously, what a tub of shit Oracle is. Where does this myth come from that it's such a great platform? Their client tools suck balls and it's generally just a pain in the ass to work with from a developer's point of view. But devs for some reason are under this impression that from thew server perspective it's rock solid and performant. Well, it may be solid, but it's a fucking turd. Our dba here - definitely an Oracle guy - has been tasked with looking into moving us onto an oss db. He basically refuses to even look at MySQL, stating that it's a mickey mouse worthless pile of shit (can't really argue with him there lol), so that basically leaves Postgres. So it's been a few weeks now, and he will basically admit now that Postgres completely waxes Oracle as far as performance goes. LOL We run RAC in production too. He's looking at failover, replication, blah blah blah now, we'll see what the verdict on that is. Oh, and Oracle AQ? That's a worthless pile of shit too. Why do they tack a fucking message queue that doesn't play nice with jms onto the fucking database, which is probably already overworked? Oh wait, that's right, they're in the business of selling per cpu licenses! Cocksuckers.
This was prompted by a recent Oracle email blast about the Exadata storage system/Warehouse.

As I did before, I'll just put the email here.

Me:
Agreed, their client tools aren't all the great. Which ones are you using?

I use SQL*Plus (naturally), SQL Developer and JDeveloper. The latter 2 tie in very nicely with Subversion. With JDeveloper, when I want to try out Java, it's got a pretty good suite of tools.

Oracle starting out catering to businesses, Microsoft started with the consumer. Oracle has made pretty good strides in making their software more usable while Microsoft has made theirs more scalable.

I haven't used AQ a whole lot and definitely not with Java. I do know that it's billed as JMS compliant.

Postgres has it's place and so does Oracle. It is a great platform if you know how to leverage it. PL/SQL is a 3GL (or is it 4) and you can write entire applications using it...and if you do that right, an incredibly robust and scalable application at that.
Mr. M:

"It is a great platform if you know how to leverage it. PL/SQL is a 3GL (or is it 4) and you can write entire applications using it...and if you do that right, an incredibly robust and scalable application at that."

NO!!! NO!!! NOOOOO!!!

I want to beat people like you who say this with a ball pean hammer. There are only a select few individuals on this earth who can write and read application logic written in SQL. AVOID THIS ANTI-PATTERN AT ALL COSTS! What is it with you f_cking database guys??? Why do you always want to stuff as much crap into the db as possible?

DUDE! SQL IS A PILE OF SHIT FOR EXPRESSING APPLICATION LOGIC!!!

It's fine for querying and manipulating sets of data, in a relational database. But it is a worthless sack of shit for expressing application logic!

I'm having to dig through this f_cking abortion right now because some Oracle f_ckhead thought "you can write entire applications using it!" Blog that, mofo!
This was followed by a package he had been working on. I wouldn't say it was the greatest, but it wasn't all bad either.

Me:
goodness gracious.

"DUDE! SQL IS A PILE OF SHIT FOR EXPRESSING APPLICATION LOGIC!!!"

disagree (naturally). It's incredibly easy to do, you just don't know how yet...and it seems even the Oracle professionals out there don't either.

I'll tell you this, the crazier the SQL or PL/SQL needed to derive and manipulate data the poorer the design. Start with a good design and it all becomes very simple.
Of course note the use of "naturally" in my lexicon. Thanks Jake.

Mr. M:
well dude, we are back to our old discussion - you arguing that procedural sql code is perfectly fine for building apps, and by extension, that the last 20 years of computer science have basically been a misguided lost journey down the meandering, fruitless trail of oop. um.....no. select this from that. otherwise keep that sql crap caged up where it belongs.

btw, do the db guys here suck? seriously. i'm not competent enough to judge. (to be fair, apparently that crap i sent you is fairly old stuff, but still....)
Me:
I would say, based on limited knowledge of software development, that the OOP movement was started because the database (specifically Oracle) was not mature enough to do what was needed. Plus, I seem to recall that the OOP movement was supposed to have solved all the world's problems by now.

It's further propogated due to the needs you specified in our discussion that day at WellCare (i.e. performance). I still believe that if application developers better understood how a DB works (Oracle, MySQL, SQLServer, etc) they would end up writing less code. Database constraints alone force you to write less (and better) code simultaneously ensuring good data.

The code that I did look at (first 1000 lines or so) isn't great.
1. With all those IF THEN ELSE statements it's telling me that there's probably a better way to store the data. Looks like they're missing an attribute that should be contained with a track.
2. using Object (PL/SQL) types to store data in the way they seem to be doing it is not the best way. Again, probably a design issue.
3. When you do something like this:

   UPDATE pb_album_metadata
      SET primary_digital_flag = 0
    WHERE album_id IN (SELECT b.album_id
                       FROM (SELECT a.album_id AS album_id,
                                MAX(a.album_id) OVER (PARTITION BY a.standard_upc) AS latest_album_id
                             FROM pb_album_metadata a
                             WHERE a.standard_upc = g_album_tab(1).standard_upc ) b
                       WHERE b.album_id <> b.latest_album_id )
         AND primary_digital_flag <> 0;

They should probably have considered end_date as an attribute of the album metadata. While analytic functions are pretty cool, they're more for analyzing (OLAP) and not OLTP environments.

That's for starters and without table definitions...
Me (again):
oh yeah...and PL/SQL is/was built on top of ADA, FYI.
Mr. M:
"I still believe that if application developers better understood how a DB works (Oracle, MySQL, SQLServer, etc) they would end up writing less code. Database constraints alone force you to write less (and better) code simultaneously ensuring good data."

Huh? What are we not understanding? What would be an example of a constraint that would force us to write less and better code?
Me:
CHECK, NOT NULL (same as CHECK) and FOREIGN KEY constraints all fit into that category.

Instead of having to check if a value is NULL when submitting it to the database, just submit and let the database throw an error, which you'll conveniently capture in your Exception block. Same goes for CHECK constraints, columnA should be either one of three values (test, test1, test3), you won't have to check that in your application code, just catch the exception. FKs insure good data (proper relationships).

A different perspective. If you are going to pay for it, use that damn thing. If you don't care, don't do it. But over the course of an "enterprisey" application, you'll end up writing more code and make maintenance that much more difficult (did I do it there?). Just those constraints will force you and the teams of application developers to put good data into the database.

You can still do it in the application of course (form validation is a good place)...
Mr. M:
Ahh, jeez dude, I wasn't sure if you were referring to the literal "constraint" or not.

Dude, even f_cksticks like redacted I think have a decent enough understanding of when and how to use db constraints. It's when you get into things like cursors or cost plans of subselects and anonymous tables (i think that's the name for it - where I say select something from (select some other crap). Then we defer to db gurus like yourself.

But dude....."you won't have to check that in your application code, just catch the exception".......uh, don't ever repeat that off our little email group. And definitely don't go posting that on your blog. F_ck me man, it's a damn good thing we keep you db folks caged up in that rdbms box....
Me:
So we've had this discussion at work...for a high transaction system, do Java/C/etc handle exceptions well or what?

Why is it bad to deal with exceptions rather than coding to avoid them?

I highly doubt even redacted understood database constraints...him and his cohorts believed all database software would be commoditized and MySQL would be king in short order.
Mr. M:
"for a high transaction system"

Or for any system really....

To use your example of check constraints (is this value one of....) or not null constraints, checking these rules in the Java code and in the database code would seem to violate DRY. But we do that alot, and it is acceptable in certain cases. For instance, we also probably violate DRY if we're checking this same rule say in Javascript on the front end. But we accept this tiny violation of DRY because it tends to make for a better user experience and as a performance gain too, for we avoid the round trip to the server. Now, what your advocating here is close to the same thing. You're basically saying, don't check for the not null constraint in Java code, just go ahead and hit the database, let the database throw back an exception to the Java code, presumably correct the problem, and then make another roundtrip to the database again. Dude, what are you thinking?!? This to say nothing of the fact that this also could be considered a violation of Fail Fast, and a violation of Defensive Programming - what happens if the dba forgot to add the not null constraint in production?

Dude, listen to this guy. For a "high transaction system" basically you want to treat the database, as much as you can, as just a dumb data holder. A f_cking dumpster, that you just throw sh_t into and pull shit back out, using no vendor-specific features if at all possible.

Again, for we've had this discussion, but even in everyday apps, not just on Wall Street, the database is the bottleneck. And the database is the hardest layer to scale. So given those facts, you should design your app to do as little work in the database as possible.
I was laughing at this point because the link above points to one of our consulting architects (I'm not really sure what his role is at this point).
Me:
i agree in any application that you want to minimize the number of round trips...

shocker...he's one of our architects. he's spot on in many instances, but...

database is the bottleneck because people don't know how to write SQL. I'll certainly concede the wall street applications (for the time being anyway), but the rest of us with what we do on a daily basis...Oracle will scale far beyond the demands they place. When that bottleneck shows up, 9 times out of 10 some dumb-ass c#/java guy thought he could write a better query than i. besides, what's the idiot doing anything but a simple join anyway? probably poor data model to start with...and we're right back where we started (sort of).
Mr. M:
"database is the bottleneck because people don't know how to write SQL.....some dumb-ass c#/java guy thought he could write a better query than i."

Dude, I'll grant you, people don't know how to write SQL, myself included. But that's not always why the database is the bottleneck. I think it's safe to say that's not even the majority of the reason. Yes, there are apps written by people who were just idiots, and they needlessly pummel the database, but that's definitely not the majority of scenarios. At my work the database is the bottleneck, and we run RAC in production. It's my understanding that even with RAC, there is a limit to how much you can scale that out. But any layer up from the database we are basically unlimited in how much we can scale that out. So it's very easy to stick another Apache box in front, or fire up another Weblogic server. But I can't do that with the database. We have 24 Weblogic servers for redacted. The database is the bottleneck. And we don't have shitty sql code in the app. In fact, we have very few hand-written queries anywhere in the app. Persisting something to a database is really a low-level concern that as an application developer I shouldn't even have to be bothered with, except for the rare corner cases where the persistence abstraction I'm using is just a little bit too generic to handle things effectively. And we don't use these ORMs because we don't know how to write sql. To effectively use an ORM tool requires a deeper understanding of sql and databases than simply being able to write solid SQL code. (Not saying Java devs who use ORMs know SQL better than a dba, just that it requires better sql skills than is required of a Java dev to simply write JDBC/SQL.) Now, before you try to tell me that my ORM library isn't able to write as efficient of sql code as a dba of your caliber, keep in mind that ORM tools are pretty advanced. They're able to intelligently do things like batch sql updates, and let you apply transactional semantics much easier than with raw jdbc/sql. But the overwhelming reason developers have so thoroughly adopted ORM is because Structured Query Language is such a nasty piece of shit for expressing application logic. SQL is a declarative, procedural language. It's totally unsuited for writing application logic! This, more than anything else, is why organizations and dev teams should seek to restrict what is handled within a relational database as much as possible - because the programming interface to it is a fucking ancient backward dying dinosaur.
Mr. V (note, not Mr. M):
My 2 canadian cents:
The polyglot approach "... use different languages for different domain ..."
Database was developed to manipulate data and should remain there.
General purpose language was developed to encapsulate logic and should remain in that domain.
You should not use DB to encapsulate business logic (in my opinion) no more than you would use HTML to create complex logic.

While Java, C#, Python, etc are described as General Purpose languages, they, too, are really domain-constrained. Their expressiveness are confined (and should be) to express low-level, tersed, explicit, verbose, and repetive logic (if that makes any sense). Languages such as these are more suitable for low-level abstraction on top of which, richer, more expressive languages can be built. We are seeing this now with the emergence of languages on the JVM (and arguably on the .Net's CLR).

I think SQL as we know will take a back seat and a smart developer somewhere will create a new domain-specific language that lives on the VM and will push the SQL expression out of the RDBMS and closer to the code that needs it. We are not there yet, but Microsfot is trying (see LINQ and all ORM papers). This is not to say that there won't be isntances where tuning in the SQL-native language won't be necessary. However, 80 to 90% of simple CRUD cases will be handled closer to the logic code that uses the data code.

Again, that's my 2 canadian cents... I could go on. But I have a meeting with redacted.
Mr. V, I believe, is a little bit more sensible. Mr. M on the other hand is just trying to rile (sp?) me up.

Me:
Someone will probably create something like that, but it still gets at the heart of one of my arguments, many developers don't know how to use a database thus will go to any means to circumvent it. Embrace it I say.

Ultimately for me, it comes down to simplicity. I can write an application using PL/SQL that will scale very well for 90% of the solutions. Of course that doesn't include the "fancy" javascript/Ajax stuff needed for the UI. In my ever so humble opinion, this is a much better solution for a business in the long run.
1. You're more likely to change the middle tier than the database tier. Java, asp, Ruby, what's next?
2. Fewer moving parts means lower cost of ownership, even with the CPU costs. One person, one, was able to build and maintain a 350 page website. Be hardpressed to do that with the more expressive languages.

I think all of us are purists and very passionate about what we do. It's probably the main reason we get along. I thoroughly enjoy these conversations because it does force me to think...and that's always a good thing.
Mr. V:
Haaa chet.
You sorta made my point than diverged away. Building an app in all PL/SQL is dangerous. It's no safer than me building an app in all Java. I can build very limited app in all Java. The moment I need to interact with other domain (UI, data, low-level native code, etc), I have to switch to something that is closer to what I am trying to do. If I need to create UI, I will pick a ui-centric environment, when I need to talk to DB, I will pass connect to a db and send in my SQL, and so forth. I will use Java as the glue to bring it all togher.

In the end, we may be saying the same thing, but using different accent. O well.
And that's where it ended. I must say it's always fun. Mr. M and Mr. V are both very smart individuals and I highly respect what they do. We have different perspectives...but I think they listen, if only a little, as I listen to them. Their voices creep up on me especially now...which is definitely a good thing.

Labels: , , , , , ,

  12/09/2008 09:11:00 PM   23 Comments Links to this post
  A Java PL/SQL Port Scanner
Saturday night I started to wonder how I could get the movies of my DVR. Recently Verizon FIOS added a boatload of HD stations so I've naturally begun to record those. Problem is, the 50 hours or so I have available are quickly eaten up by the HD versions.

We have the multi-room DVR setup, which means we have one DVR but we can view those movies from any TV (with a cable box). Naturally all these boxes are on my internal network (router provided by Verizon).

I understand that with some boxes you can go out and buy additional storage, but I wanted a DIY solution. So, if I could find an open port on one of these boxes, then perhaps I could figure out a way to get the movies onto my computer.

Java to the rescue. Well, not really, but I have always wanted to learn more about that kind of lower level programming (sockets/ports/TCP/whatever). See, I don't really know what I'm talking about which is why I did it.

Since I use JDeveloper for writing SQL and PL/SQL, creating a java class (with the Main method) is easy.
import java.io.IOException;
import java.net.Socket;
import java.net.UnknownHostException;
import java.util.Date;

public class Class1
{
  private static void p ( String text )
  {
    System.out.println( text );
  }

  public static void main(String[] args) throws UnknownHostException, IOException
  {
    Date startDate = new Date();
    String machine = "192.168.1.102";
    Thread thread = new Thread();

    Socket socket;
    int j = 0;
    int x = 0;
    p( "start date/time:  " + startDate.toString() );
    for ( int i = 1000; i < 10000; i++ )
    {
      x++;
      p( "checking port:  " + machine + ":" + i );
      try
      {
        socket = new Socket( machine, i );
        p( "Port open:  " + machine + ":" + i );
        j++;
      }
      catch (UnknownHostException e) { }
      catch (IOException e) { }
      finally
      {
      }
    }
    p( "ports checked:  " + x );
    p( "ports open:  " + j );
    p( "start date/time:  " + startDate.toString() );
    p( "end date/time:    " + new Date().toString() );
  }
}
As LC would say, "That's not so bad."

Only it was slow, 1 port at a time. I did learn that trying to open up a port connection is s l o w. How can I speed this up?

I could make it threaded right? No. That would take too much time.

But hey, I know how to do it in PL/SQL (threaded that is). UTL_TCP should work nicely. I've never had to use it, so a great opportunity to familiarize myself with it (not that it's needed very often).

So I start Oracle XE and initially just try it out to see how it works:
DECLARE
  c UTL_TCP.CONNECTION;
BEGIN
  c := utl_tcp.open_connection
        ( remote_host => '192.168.1.2',
          remote_port => 1,
          tx_timeout => 1 );
  
  utl_tcp.close_connection( c );
EXCEPTION
  WHEN others THEN
    utl_tcp.close_connection( c );
END check_port;
/
Easy enough. Now let's start use DBMS_JOB to "thread" this.
DECLARE
  TYPE r_record IS TABLE OF OPEN_PORTS.IP%TYPE INDEX BY BINARY_INTEGER;
  l_table R_RECORD;
  job_count PLS_INTEGER;
  job_number PLS_INTEGER;
BEGIN
  l_table(1) := '192.168.1.102';
  
  FOR i IN 1..l_table.COUNT LOOP
    FOR j IN 1..10000 LOOP
      SELECT COUNT(*)
      INTO job_count
      FROM user_jobs;

      WHILE job_count < 20 LOOP
      
        dbms_job.submit
          ( job => job_number,
            what => 'BEGIN check_port( ''' || l_table(i) || ''',' || j || '); COMMIT; END;' );

        COMMIT;
      END LOOP;
    END LOOP;    
  END LOOP;
END;
/
So it's running, and running, and running...I opened up another session and did a count on user_jobs. 10000. 12000. 20000. Uh, I think there's something wrong here.

I finally kill the session and there are 36,446 jobs in the queue. All checking the same port.

Then I can't even issue a simple SELECT statement. Maximum number of sessions reached or some such rubbish. Just a reminder, it was Saturday night and I was just trying to see if I could do this quickly.

Since I couldn't connect, I tried bouncing the database (a loud gasp from the DBAs out there). Come on! It's a local XE instance...there's nothing important there.

Start it back up, connect as SYS and still can't issue any queries. I do notice as I hit the / sign and the Enter key a bunch of times that once in a while it will let me do a query. So I create a quick PL/SQL block to remove all the jobs from the queue.
BEGIN
  FOR i IN ( SELECT job FROM user_jobs ) LOOP
    dbms_job.remove( i.job );
  END LOOP;
END;
/
I go through the / + Enter key about 50 times until it finally runs. Success!

Or so I think. Still receiving the max sessions error. There are still 36,446 jobs in user_jobs. I can't remove them anymore as I get "job doesn't exist" when I try to dbms_job.remove it again. Hmmm...How about?
DELETE FROM dba_jobs;

36,446 rows deleted.
What?

How can that be? I didn't expect that to work at all...but it did. I have yet to look up the particulars of my actions (deleting from dba_jobs) but it did the job.

And yes, I'm still planning on writing this (correctly) in the near future...I want more movies!

Labels: , , , ,

  11/18/2008 10:12:00 PM   1 Comments Links to this post
  Oracle and The Register
I found a great headline last week via syntaxi at twitter.

Oracle shareholders choke on Ellison's package

Oracle shareholders enhance Ellison's package

It seems Gavin Clark over at The Register has a pretty good sense of humor.

Labels: ,

  10/18/2008 12:06:00 AM   2 Comments Links to this post
  Go Rays!
From chester_james


Rayhawks all the Rage!

Labels: ,

  10/02/2008 09:17:00 AM   0 Comments Links to this post
  Precious Memory?
My six year old son is brilliant. He's got my sense of humor with his mother's verbal wit.

This weekend, he says to us, "Mom, remember when I peed on you when I was a baby?" He starts laughing..."aaaah, precious memories."

What? "Precious" memories? Where the hell did he get that from?

Labels: ,

  9/14/2008 11:40:00 PM   0 Comments Links to this post
  Calculate Total Possible Score for Wii Bowling, Power Throws
Needless to say I have too much time on my hands. I've become obsessed with two Wii games, Bowling and Tanks. I play over and over. I had a perfect game in bowling the other day which wasn't quite as exciting as I thought it would be.

I've also been playing the "Training" game of Power Throws (bowling). You are given 10 shots with another row added each frame. You start out with 10, then 15, then 21, etc. If you get all the pins, you get a bonus (total pins * 2). So far, my highest score has been 651.

I've never bothered to figure out what the total possible score is though.

SQL to the rescue.

Using analytics and the CONNECT BY LEVEL option (10g and higher), it's fairly easy:

SELECT
  pin_row,
  pins,
  running_total,
  rt_with_bonus
FROM
  (
    SELECT
      rownum pin_row,
      rownum pins,
      SUM( rownum ) OVER ( ORDER BY rownum ) running_total,
      ( SUM( rownum ) OVER ( ORDER BY rownum ) * 2 ) rt_with_bonus
    FROM dual
      CONNECT BY LEVEL <= 13
  )
WHERE pins >= 4
I had to set the filter on the outer query because you start with 4 rows (10 pins). And the results:

CJUSTICE@XE>BREAK ON REPORT
CJUSTICE@XE>COMPUTE SUM OF RUNNING_TOTAL ON REPOR
CJUSTICE@XE>COMPUTE SUM OF RT_WITH_BONUS ON REPOR
CJUSTICE@XE>/

   PIN_ROW       PINS RUNNING_TOTAL RT_WITH_BONUS
---------- ---------- ------------- -------------
         4          4            10            20
         5          5            15            30
         6          6            21            42
         7          7            28            56
         8          8            36            72
         9          9            45            90
        10         10            55           110
        11         11            66           132
        12         12            78           156
        13         13            91           182
                      ------------- -------------
sum                             445           890
Voila! 890 is the best possible score if I strike every frame. I did manage to pass my high score by 9 pins tonight as well. It's a great day!

So how come no one wants to hire a guy that can do this kind of fun stuff with SQL? ;)

Labels: , ,

  8/20/2008 10:04:00 PM   3 Comments Links to this post
  iPhone AppStore oops?
My wife has an iPhone (I have the $10 phone I got with my 2 year plan). I love the thing. I steal it whenever I can.

One of my favorite things to do though is to ask her a random question, you know, one that would take 3 seconds to google and find the answer, just to test her. Usually, she just says "I don't know"

Um...you have an iPhone...and you can look it up right? (lots of sarcasm)

Oh yeah.

I probably do it because I have the $10 phone and I'm jealous.

Anyway, she showed me a screen shot yesterday of an app in the AppStore that was selling for $999.99. Funny. What do you get? A pretty picture. What else? Nothing.

So, via Jake then Digg, I found the following picture:

Labels: ,

  8/07/2008 10:17:00 PM   1 Comments Links to this post
  I Found a New Job!
http://www.foxnews.com/story/0,2933,395181,00.html

My wife thinks I'm perfect for it!

Labels: ,

  7/31/2008 11:27:00 AM   4 Comments Links to this post
  Don't Go Into the Office
Whenever my boss asks me to come with him...I will politely tell him I'm not going.

I was laid off yesterday.

I want to see how many jobs I can have in one years time apparently. I didn't undermine myself this time though.

So if you know of anything in the Tampa Bay area, please let me know.

Labels: ,

  7/29/2008 02:34:00 PM   11 Comments Links to this post
  Men Are From Venus?
So my wife calls me today to share a funny story.

Apparently my son (Little Chet) asked about metorites. He had seen them on Jimmy Neutron - Boy Genius. She starts explaining what they are and the conversation led to talk about the space shuttle and the space station.

LC: What do they do up there?

WIFE: They do all kinds of stuff, experiments, look at planets, etc.

LC: The look at planets? Like Planet Penis?

WIFE: What?!

LC: Planet Penis, the one close to Earth.

WIFE: You mean Venus?

LC: Yeah, that's the one.

He doesn't get "Uranus" yet...

Labels: ,

  7/10/2008 09:50:00 PM   2 Comments Links to this post
  Count The K's
I have this really annoying co-worker who happens to be the DBA. Everytime he walks by my desk he pounces on my keyboard. I've learned to Windows Key + L to lock the computer when I hear him approaching, but occasionally I forget.

As I'm standing near my unlocked computer he starts typing furiously in my sql*plus session (shouldn't he know better?). I should know better...

Walking away he asks, "How many K's are in there?"

I ignored him, but then wondered myself...what's the best solution to this problem?

So, here it goes:
DECLARE
  l_count_k NUMBER := 0;
  l_string VARCHAR2(300);
  l_string_length INTEGER;
BEGIN
  l_string := 'W34 6OKWE;KTL;SERT;LKSERTSLKRLTKRTKL;KERTL;ERKT;LKERLTKERKT;EKT;LEKRT;LEK;
LTKERTKERKT;LEKRT;LEKRT;LER;LTKETK;ERKT;LEKRT;LEKT;LEKR;LTKETKERTKL;ERKTL;ERKTL;ET;LEKT;LEKRT;LERTL;
EKRL;TKELTERTKL;ETL;EKTL;EKTL;ETL;ERTL;EL;TERTRTL;ERTL;ERTL;ERTL;EKRT;LEKTL;EKT;LERTL;EKL;KER;LTKE;L
TKELTKER';

  l_string_length := LENGTH( l_string );

  FOR i IN 1..l_string_length LOOP
    IF SUBSTR( l_string, i, 1 ) IN ( 'K', 'k' ) THEN
      l_count_k := l_count_k + 1;
    END IF;
  END LOOP;
  
  dbms_output.put_line( 'Kk Count:  ' || l_count_k );
END;
/
Easy enough, 45.

Then I started thinkinhg...can I do this in pure SQL? Of course!

SELECT 
  SUM( CASE
         WHEN SUBSTR( UPPER( mystring ), rownum, 1 ) = 'K' THEN
           1
       END ) k
FROM
  dual,
  (
    SELECT 'W34 6OKWE;KTL;SERT;LKSERTSLKRLTKRTKL;KERTL;ERKT;LKERLTKERKT;EKT;LEKRT;LEK;
    LTKERTKERKT;LEKRT;LEKRT;LER;LTKETK;ERKT;LEKRT;LEKT;LEKR;LTKETKERTKL;ERKTL;ERKTL;ET;LEKT;LEKRT;LERTL;
    EKRL;TKELTERTKL;ETL;EKTL;EKTL;ETL;ERTL;EL;TERTRTL;ERTL;ERTL;ERTL;EKRT;LEKTL;EKT;LERTL;EKL;KER;LTKE;L
    TKELTKER' mystring
    FROM dual
  ) c
CONNECT BY LEVEL <= LENGTH( mystring )

COUNT_OF_K
----------
        45

1 row selected.
I'm sure many of you can do better than that. So let's see 'em. Prodlife, this isn't a complicators test either. ;)

Labels: , ,

  7/08/2008 09:00:00 PM   9 Comments Links to this post
  Funny License Plate 

H20UUP2
First to figure out wins absolutely nothing but praise!

Labels: ,

  7/02/2008 08:33:00 PM   3 Comments Links to this post
  Failed Deployment...
236 Days
20 Hours
48 Minutes
30 Seconds

I hate making mistakes but I've made another one. My streak ends almost 237 days from my previous one.

Something so silly too.

In our source system, data was been double loaded somehow. So we decided on a surgical delete. A total of 7 DELETE statements needed to be run; 4 on the source system and 3 on the target system.

The source system went off without a hitch. I babysat the re-load of the source tables and was ready to have our load jobs run in our target system.

What's this? It ran in half the time?! How's that possible?

I pulled up our logs to find that zero rows were loaded into one of our tables. There should have been 45 Million plus.

I started to run down the possible causes:
1. Did the job we have in the scheduler that TRUNCATEs our persistent staging tables run? Nope.
2. Did I fail to instruct the DBAs correctly in the critical CR? Nope. Instructions look good.
3. Next to the logs, it ran fine on Saturday morning but not Sunday morning. What happened yesterday?
4. Ah yes, my CR. Open up the script...nothing out of the ordinary...and then I saw it.

On our target system, we use work tables to pre-generate a keys. It makes things a heck of a lot faster and removes the need for PL/SQL lookups in SQL (no, we don't have incremental builds yet).

So the work table needs to be DELETEd from first based on the keys from the first:

DELETE FROM some_key_table a
WHERE EXISTS ( SELECT NULL
               FROM the_main_table
               WHERE business IN ( 'TTT', 'TTR' )
                 AND dateof = TO_DATE( '24-MAR-08', 'DD-MON-YY' )
                 AND my_key = a.my_key );

OK, no funny business there.

Then I DELETE from the main table:

DELETE FROM the_main_table a
WHERE EXISTS ( SELECT NULL
               FROM the_main_table
               WHERE business IN ( 'TTT', 'TTR' )
                 AND dateof = TO_DATE( '24-MAR-08', 'DD-MON-YY' ) );

As I look at it I wonder WTH I was thinking using an EXISTS clause on the main table. That's the source.

But do you see what I missed?

See it yet?

OK, I left out the "AND my_key = a.my_key" from the inner query. Obviously a stupid approach, but it would have worked. The best way to do it is to just get rid of the EXISTS clause:

DELETE FROM the_main_table a
WHERE business IN ( 'TTT', 'TTR' )
  AND dateof = TO_DATE( '24-MAR-08', 'DD-MON-YY' ) );

Live and learn, live and learn...

Labels: , , ,

  3/30/2008 10:21:00 PM   0 Comments Links to this post
  Bowling for IT
Another non-Oracle related post. Just fun at work.

Last Friday we had another one of our IT all-hands meetings. My goal at each one is to make either the rumor list (of the top ten variety) or to be somehow be involved (hopefully good) in other ways.

Two months ago I was promoted to SVP of IT, because I was able to talk my CIO into it. Last month, I was promoted to CEO because I happen to resemble our new CEO. This month, the "light" piece was a "Where are they now?" complete with old/new pictures of IT employees. (Needless to say I am an only child...I crave attention!)

To say I've gained weight since starting a career/marriage/family would be an understatement. I went from a lean and mean 170 to about 250 now. The first pic I had just completed a sprint triathlon in Clermont, Florida. The second was sometime after the birth of my first child.


I take about every opportunity I can to send out the first one to new friends. "I didn't always look like a slob. That got to be my "before" picture and my "after" picture was my mugshot from my ID badge (yikes).

There I was 10 feet tall and looking great! I no longer had to send the picture out to anyone (and risk possible graffiti, though I guess posting it here doesn't help matters).

The important part was that I made all-hands again. I think that's 12 in a row.

And finally to the title of the post. After our all-hands meeting we went bowling. Food and bowling were free. I did notice however that there are quite a few, um..., drinkers among IT. I would certainly say that I fall into that category. At one point, it took so long to get a drink (only one bartender for 100 some odd people) I tried to ban the sale of mixed drinks so it would speed things up. However, my ploy didn't work.

I just bought two beers and waited for everyone else to follow-up with pitchers!

Gotta have fun at work right?

Labels: ,

  3/04/2008 09:08:00 PM   0 Comments Links to this post
  Application Developers vs Database Developers
It started innocently enough with this article. I sent it out to about 20 colleagues.

The best line from the article:
"Jerry: "Yeah, databases cause lots of headaches. They crash all the time, corrupt data, etc. Using text files is better."

One of my more recently arrived colleagues (I'll call him Mr. M) replied to everyone with this statement:

"Kind of funny actually, databases are less and less important at the large investment banks, where they basically load everything up into a data grid across a several hundred node cluster. Writing to the db is way too slow."

This started a day long exchange of emails. What follows is the entire thread (up until my last post tonight).

Me:
"I would just argue that they don’t necessarily know how to write to databases. I would however love to see benchmarking done on both methods. Would be an interesting test..."

Mr. M:
"Well, my understanding is they just can’t scale out the db enough. Even something like Oracle RAC won’t work. And outside of the military, these are probably the top 1% of programmers in the world building this stuff."

Me:
"A benchmark would be the only way I would believe it.

If you said the top 1% of database developers tried it and failed, I would be more likely to agree.

My experience is that application developers != database developers. Different type of thinking involved."

Mr. M:
"'A benchmark would be the only way I would believe it.'

Do you need a benchmark before you would believe in-memory retrieval is faster than disk retrieval? Essentially, this is what we’re talking about.

'If you said the top 1% of database developers tried it and failed, I would be more likely to agree. My experience is that application developers != database developers. Different type of thinking involved.'

Why? It’s an issue to do with application performance not simply database performance. Database concerns are a subset of application concerns, essentially a specialization, requiring less encompassing knowledge. ;)

From the article you linked to (http://www.watersonline.com/public/showPage.html?page=432587)

"Better data management is the answer, says Lewis Foti, manager of high-performance computing and grid at The Royal Bank of Scotland (RBS) global banking and markets. "For very large compute arrays, the key issue is data starvation and saturation. This problem requires data grids with high bandwidth and scalable, parallel access,
...
Banks are learning that data management in a distributed grid environment is very different from online transaction processing. "With so many data sources, distribution channels, demands for aggregation and analytics, surges in data volumes and complex dynamics between the flows, we need to manage 'data in motion' and give up the notion that data is somehow stored. It's dynamic, not static," says Michael Di Stefano, vice president and architect for financial services at GemStone Systems
...
There is even some debate over how small a unit of work can be put on today's grids. Di Stefano at GemStone, for example, says, "One client has gone from 200 trades per second in a program trading application to more than 6,000 trades per second. This shows what the technology can do."

Yep, the writing is on the wall. Oracle knows it too.

http://www.google.com/search?hl=en&q=oracle+buys+tangosol&btnG=Google+Search"

Me:
"Good points. If it is in-memory it would be faster. I have not had the pleasure to work on such a system.

I do disagree with the database concerns being a subset of application concerns. The data drives the app. We’re probably getting religious at this point (or am I)."

Mr. M:
"‘The data drives the app.”

Exactly, but who’s to say where the data comes from or in what format? My application data may reside completely in xml files, or maybe I get it from some third party web services a la the en vogue “mashup.” Heck, I may not even need to worry about a database anymore…. http://www.amazon.com/gp/browse.html?node=16427261 The database is only one particular concern of the overall application. And it’s the application that matters. Data is useless if it just sits on a disk somewhere. It’s the ways in which the application lets the users view and manipulate the data that adds value to the business.

Yep, definitely a different type of thinking between application developers and database developers."

Me:
"Definitely religious now.

Applications come and go, data stays the same. Think Green Screens, EJBs, Ruby…what’s next?"

Mr. M:
"'Applications come and go'

Exactly. Businesses are not static, nor are the markets they compete in. Changing applications are a function of changing business processes and changing markets.

'data stays the same.'

Nonsense. Otherwise UPDATE would not be an SQL reserved word. If you mean database technology stays the same, well, I’m more inclined to agree with that.

'Think Green Screens, EJBs, Ruby...what’s next?'

Whatever comes along to let the business more effectively respond to current market realities. Application platforms have evolved much faster than database platforms have. They’ve had to, their sphere of operation is much broader than that of databases, this is only natural, they deal with much broader concerns than do databases. Databases in the internet era function in essentially the same role they did in the era of dumb terminals. Clearly application platforms have evolved orders of magnitude more. Hence the statement, database concerns are a subset of application concerns.

Here’s a simple test….if I take some business application and I’m forced to throw away one or the other, either the database or the appl- wait a second, it doesn’t even make sense to finish it, does it? The business can live without the database. I could do all kinds of things with the data, I could stick it anywhere. The business can’t live without the application though. Another way to look at is, what do the business users look at, test, approve, and use? The database? Of course not, they look at the application. They could care less whether the data sits on disk in an RDBMS, xml, or flat files."

Me:
"We obviously violently disagree.

Without the database (and I use database and data interchangebly), the business could no longer function. The app is meaningless. How would you contact your customer? You couldn’t find it.

'Exactly. Businesses are not static, nor are the markets they compete in. Changing applications are a function of changing business processes and changing markets.'

Poorly designed applications…that is all."

A Feisty Colleague:
"Using data and database interchangeably is incorrect. A database is a mechanism for data storage. XML data sets and flat files are mechanisms for data storage, too. So is a file cabinet, because, the data doesn’t have to be electronic, it could be … gasp! … on paper, and the application to use that data would be hands for holding the paper and a pencil to update and add data to the page."

Me:
"No it isn’t. I take into account xml files, flat files, web services (but not paper, unless it’s scanned) and all that. It would be consumed by the database and then accessed by the application via SQL.

(that’s for Mr. M and the feisty one)"

At which point someone forwarded the home page for Oracle's TimesTen In-Memory Database.

Me:
"A database on/in the mid-tier...Perfect!"

Mr. M:
"Implicit acknowledgment that disk IO operations that come with traditional database access simply can’t match the performance of in-memory data access (a point which you previously were unconvinced of but now seem perfectly accepting of the idea once you see it’s got Oracle’s imprimatur on it).

Of course, why any application developer would want to program against an SQL interface if they weren’t forced to is beyond me. It is orthogonal to the programming model of most application platform languages.

Surely Oracle recognize this fact too or they wouldn’t be buying Tangosol and other data grid technologies. Of course, most of those products are far more technically advanced than TimesTen or anything Oracle has in that space.

Incidentally, it’s illustrative to note that Coherence and other products like it were for the most part designed and built by application programmers. The development of all these products is pretty much driven by the needs of the large investment banks on Wall Street. These trading applications simply had too many concurrent transactions to use an RDBMS (a problem quite a number of public domains now share, most famously google.com, nope, no RDBMS there, yet miraculously there is still data). The database just simply would not scale to such a degree. So the application developers, by necessity, came up with an alternate solution that did work, a fully transactional cache of data replicated across a cluster with node numbers in the thousands, and no relational model whatsoever to speak of. A perfect example of how database concerns are only one, sometimes small, concern amongst many that application developers must be aware of and ready to solve."

Me:
"Like you said initially, the top 1%.

Many of us will never touch a system like this.

I will certainly concede that it is faster (still would love to see benchmarking though), but that still leaves 99% of the applications out there that do not require that kind of performance."

Me (again):
"And don’t forget, I use data and database interchangeably. Applications are nothing without the data right?

As to the object/relational impedance mismatch...well, more people that don’t know how to work in sets. Looping is what they understand. I understand the application side more than you seem to give me credit for.

I’m not saying applications aren’t important, they are. Data (databases) and applications go hand in hand. If the application went away though, they could still access their data via SELECT statements (yes, via an application client tool), however painful that may be. Applications make retrieving data that much easier for our users.

If anyone wants to unsubscribe from this mailing list, just let us know. This is fun for me (I’m guessing Mr. M too)."

Needless to say it was a fun day. It didn't get [too] personal. More than anything I'm happy to have an equally passionate colleague.

Besides, he claims he was just fracking around with me. ;)

Labels: , , , ,

  2/20/2008 10:29:00 PM   9 Comments Links to this post


Guest Authors

How To

Popular

Previous Posts

Code Projects

Archives
August 2007 / September 2007 / October 2007 / November 2007 / December 2007 / January 2008 / February 2008 / March 2008 / April 2008 / May 2008 / June 2008 / July 2008 / August 2008 / September 2008 / October 2008 / November 2008 / December 2008 / January 2009 / February 2009 / March 2009 / April 2009 / May 2009 / June 2009 / July 2009 / August 2009 / September 2009 / October 2009 / November 2009 / December 2009 / January 2010 / February 2010 / March 2010 /


Aggregated by OraNA