Showing posts with label database. Show all posts
Showing posts with label database. Show all posts

Wednesday, June 12, 2013

Required Reading

It's not often that I run across articles that really resonate with me. Last night was one of those rare occasions. What follows is a sampling of what I consider to be required reading for any IT professional with a slant towards database development.

Bad CaRMa


That led me to Bad CaRMa by Tim Gorman. This was an entry in Oracle Insights: Tales of the Oak Table, which I have not read, yet.

A snippet:

...The basic premise was that just about all of the features of the relational database were eschewed, and instead it was used like a filing system for great big plastic bags of data. Why bother with other containers for the data—just jam it into a generic black plastic garbage bag. If all of those bags full of different types of data all look the same and are heaped into the same pile, don't worry! We'll be able to differentiate the data after we pull it off the big pile and look inside.

Amazingly, Randy and his crew thought this was incredibly clever. Database engineer after database engineer were struck dumb by the realization of what Vision was doing, but the builders of the one-table database were blissfully aware that they were ushering in a new dawn in database design...

This is from 2006 (the book was published in 2004). Not sure how I missed that story, but I did.

Big Ball of Mud

I've read this one, and sent it out, many times over the years. I can't remember when I first encountered it, but I read this once every couple of months. I send it out to colleagues about as often. You can find the article here.

A BIG BALL OF MUD is haphazardly structured, sprawling, sloppy, duct-tape and bailing wire, spaghetti code jungle. We’ve all seen them. These systems show unmistakable signs of unregulated growth, and repeated, expedient repair. Information is shared promiscuously among distant elements of the system, often to the point where nearly all the important information becomes global or duplicated. The overall structure of the system may never have been well defined. If it was, it may have eroded beyond recognition. Programmers with a shred of architectural sensibility shun these quagmires. Only those who are unconcerned about architecture, and, perhaps, are comfortable with the inertia of the day-to-day chore of patching the holes in these failing dikes, are content to work on such systems.

Read it. Remember it.

How To Ask Questions The Smart Way

Ever been in a forum? Has anyone ever given you the "RTFM" answer? Here's how you can avoid it. How To Ask Questions The Smart Way. I read this originally about 9 or 10 years ago. I've sent it out countless times.

The first thing to understand is that hackers actually like hard problems and good, thought-provoking questions about them. If we didn't, we wouldn't be here. If you give us an interesting question to chew on we'll be grateful to you; good questions are a stimulus and a gift. Good questions help us develop our understanding, and often reveal problems we might not have noticed or thought about otherwise. Among hackers, “Good question!” is a strong and sincere compliment.

Despite this, hackers have a reputation for meeting simple questions with what looks like hostility or arrogance. It sometimes looks like we're reflexively rude to newbies and the ignorant. But this isn't really true.

What we are, unapologetically, is hostile to people who seem to be unwilling to think or to do their own homework before asking questions. People like that are time sinks — they take without giving back, and they waste time we could have spent on another question more interesting and another person more worthy of an answer. We call people like this “losers” (and for historical reasons we sometimes spell it “lusers”).

Business Logic - PL/SQL Vs Java - Reg

The article can be found here.

I don't believe this is the one that I would read just about every day during my first few years working with Oracle, but it's representative (I'll link up the original when I find it). I cut my teeth in the Oracle world by reading AskTom every single day for years. Some of my work at the time included working with java server pages (jsp) - at least until I found APEX. I monkeyed around with BC4J for awhile as well, but I believe these types of threads on AskTom kept me from going off the cliff. In fact, I got to a point where I would go to an interview and then debate the interviewer about this same topic. Fun times.

if it touches data -- plsql.

If it is computing a fourier transformation -- java.

If it is processing data -- plsql.

If it is generating a graph -- java.

If it is doing a transaction of any size, shape or form against data -- plsql.

Thinking Clearly About Performance

Cary Millsap. Most of the people seem to know Cary from Optimizing Oracle Performance, I didn't. I first "met" Cary virtually and he was gracious enough to help me understand my questions around Logging, Debugging, Instrumentation and Profiling. Anyway, what I've learned over that time, is that Cary doesn't think of himself as a DBA, he's a Developer. That was shocking for me to hear...I wonder how many others know that. So I've read this paper about 20 times over the last couple of years (mostly because I'm a little slow). I organize events around this topic (instrumentation, writing better software, etc) and this fits in perfectly. My goal is to one day co-present with Cary, while playing catch, on this topic (I don't think he knows that, so don't tell him). Link to his paper can be found here. Enjoy!

The Complicator's Gloves

One of my favorite articles from The Daily WTF of all time. Find the article here. The gist of the story is this: an internal forum where people were discussing how to warm a given individuals hands on his bike ride to work. The engineers then proceeded to come up with all kinds of solutions...they spent all day doing this. Finally, someone posts, "wear gloves." End of discussion. Love it. I wrote about it years ago in Keeping it Simple. For a few years I considered buying up thecomplicatorsgloves.com and try to gather related stories, but I got lazy. You should read this often, or better, send it out to colleagues on a regular basis to remind them of their craziness.

I'll continue to add to this list as time goes on. If you have any suggestions, leave a comment and I'll add them to the list.

Thursday, May 31, 2012

Building an OBIEE Test Lab

About 1.5 years ago I bought myself a nice little desktop. The only issues I've had with this computer over that time are the OS upgrades (Ubuntu). It has been an outstanding machine. I currently have 3 "computers" running, 2 Windows VMs and the host machine. I've had upwards of 5 running at once. That's with only 24 GB of RAM. I plan on upgrading to 48 in the near future (max of 96).

One of the motivators for buying the machine was the recent (at the time) release of OBIEE 11g, which required significantly more power. I've installed OBIEE 11g a number of times in virtual machines, but usually just put everything on a single machine (Oracle Enterprise Linux).

Now, I'd like to build out a machine for each component. Why? I have no earthly idea. Fun perhaps?

So here's my starting point:



There are no lines or anything yet, I'm just trying to get it down on paper before I start.

There are 4 distinct sets of software there: That means you can ignore the "Oracle LDAP" and replace it with Oracle Identity Management. I'll figure out later what all the components will be.

Also, you could split that up into 5 software components. You can install WebLogic by itself and then install (software only) OBIEE on top of that. Doubt I'll go that route for this first go, but we'll see. None of this includes source systems either. Somewhere around here I have a fresh install of EBS. Then I would have to install Informatica and the DAC. I'm sure I could get this up to 10 machines. I must watch out for scope creep. Bah, who am I kidding?

As I update my Visio doc I'll update the blog as well and share the progress with you.

I'll start by creating a snapshot of Oracle Enterprise Linux (5.7 I believe is the latest compatible release with OBIEE) and then use that as the base for everything else (database, web tier, etc). If I'm missing something or you think I should add something, leave a comment.

Tuesday, October 18, 2011

KScope + DevOps

Last year I had the pleasure of getting the Sunday Symposium together for KScope 11, this year, I have completed my takeover of the Database track by becoming the track lead.

I thought this was the best job ever, then I was attacked Nancy Kerrigan style by my handlers.

All that said, I think I've gathered a pretty good group of people to help review and select the abstracts for next year's conference (San Antonio, TX).

There will be 4 sub-tracks this year:
- Design/Data Modeling
- Maintenance (Performance, Tuning, Upgrades)
- MySQL
- (Dev)Operations

The one I am most excited about is the (Dev)Operations sub-track, aka, DevOps.

What is DevOps?

I'm glad you asked..

"DevOps" is an emerging set of principles, methods and practices for communication, collaboration and integration between software development (application/software engineering) and IT operations (systems administration/infrastructure) professionals.[1] It has developed in response to the emerging understanding of the interdependence and importance of both the development and operations disciplines in meeting an organization's goal of rapidly producing software products and services.

I am not necessarily a fan of the movement, but I am a fan of the principles behind it.

Every developer has a story about working with an evil DBA. LIkewise, every DBA has a story about some application that went to production where they were left completely out of the process.

But it is more than just a simple, "Can't we all just get along?" plea, this is about creating better software and streamlining processes.

My personal experience has been one of woeful cooperation, at any level. Our thought, our hope, is that this well help give other Oracle professionals better ideas on how to start down this road.

If you are interested in this topic, sign up. If you want to present on this (or any other) topic, register here.

Monday, January 24, 2011

ORA-00130: invalid listener address...

The actual error was:
SQL>STARTUP;

ORA-00119: invalid specification for system parameter LOCAL_LISTENER
ORA-00130: invalid listener address '(ADDRESS=(PROTOCOL=TCP)(HOST=oracle)(PORT=1521))
I had finally managed to create my 11gR2 instance when I got this. So what happened?

1. Started OEL 5.5 64 bit VM
2. Logged in as user oracle
3. Opened terminal and ran . oraenv
4. sqlplus / as sysdba
5. Issued the startup command.

ORA-00130 from the docs:
Cause: The listener address specification is not valid.
Action: Make sure that all fields in the listener address (protocol, port, host, key, ...) are correct.

So it has something to do with how I configured everything. What else did I change on the system?

After installing the database, I ran netca to set up the listener. I manually edited the file to change localhost.localdomain to just localhost. I also added an entry in the tnsnames.ora file to point to the listener.

From a system perspective, I decided I didn't like "new-host" showing up on my router configuration page. So I changed the hostname to oracle. I used the network GUI tool, not the CLI (still hazy on how exactly that is done).

I'm back to my sql prompt.
SQL>STARTUP;

ORA-00119: invalid specification for system parameter LOCAL_LISTENER
ORA-00130: invalid listener address '(ADDRESS=(PROTOCOL=TCP)(HOST=oracle)(PORT=1521))
Come on!

I then asked The Google Machine for help. That led me to this thread on OTN. About 6 or 7 replies in, Hans Forbrich offers up some very helpful troubleshooting tips.

1. Check that your environment variables are set. Check.
2. Make sure the listener is started. Check.

There was a caveat for #2 though. If your listener blew up on start, check these other things. That's when I saw cat /etc/hosts and his output for that. I quickly checked mine and saw this:
[oracle@oracle bin]$ cat /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
Ah...where's oracle? I tried pinging oracle, no response. I open up the file and add it in:
[oracle@oracle bin]$ cat /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost oracle
::1 localhost6.localdomain6 localhost6
I ping oracle again, response!

Start and restart the listener. tnsping TESTING OK!
SQL> startup;
ORACLE instance started.

Total System Global Area 1068937216 bytes
Fixed Size 2220200 bytes
Variable Size 616566616 bytes
Database Buffers 444596224 bytes
Redo Buffers 5554176 bytes
Database mounted.
Database opened.
SQL>
Awesome.

In my defense (or will this be my excuse?), I don't do this everyday, or ever week, or every month. It is a once a quarter, at best, exercise. I forget things. Next time, I won't forget. If I do, I'll have this helpful page to remind me what an ID10T I am.

Friday, January 21, 2011

11gR2 Installation Mess

I recently destroyed (on purpose this time) all my Oracle virtual machines. It was time for me to re-install to see if I had actually learned anything. The short story, I haven't learned anything.

I've been using Ubuntu at home for almost 2 years now. I'm much more comfortable working from the CLI, but I'm thankful there's a pretty GUI.

I've even managed to install the Oracle database on Linux. What's that you say? I did document it here. In my eyes, that was nothing short of a miracle.

Anyway, to my subject of my post.

For 2 weeks now I have been trying to rebuild my virtual machines. Upgraded to Oracle Enterprise Linux 5, Update 5 (Carcinogen? Carthage?). Easy, takes 6 minutes.

Download the 64 bit software from OTN. File 1 and File 2.

Unzip the files.

Install (software only).



If somehow I made it past the installation of the binaries, I'd get this running the DBCA.



Googling seemed to provide a consensus, make sure the checksum after downloaded matches what is on the download page.

Easy enough.

Here's what is on the page:
linux.x64_11gR2_database_1of2.zip (1,239,269,270 bytes) (cksum - 3152418844)
linux.x64_11gR2_database_2of2.zip (1,111,416,131 bytes) (cksum - 3669256139)
From the OEL VM:
[oracle@localhost 11gR2]$ cksum linux.x64_11gR2_database_1of2.zip 
856173531 1239269270 linux.x64_11gR2_database_1of2.zip
[oracle@localhost 11gR2]$ cksum linux.x64_11gR2_database_2of2.zip
3669256139 1111416131 linux.x64_11gR2_database_2of2.zip
OK, the first file doesn't match. WTF? I run it from my machine.
oraclenerd@oraclenerd:/media/software/oracle$ cksum linux.x64_11gR2_database_1of2.zip 
3152418844 1239269270 linux.x64_11gR2_database_1of2.zip
oraclenerd@oraclenerd:/media/software/oracle$ cksum linux.x64_11gR2_database_2of2.zip
3669256139 1111416131 linux.x64_11gR2_database_2of2.zip
Matches.

Copy it over to the other machine.

It doesn't match.

Move it over to the other machine.

It doesn't match.

Remove everything, re-download. Run the checksum. No match. Just to throw another little twist in, the checksum changes.

I'm pretty sure I've lost my mind at this point.

Anyone encountered something like this before?

Update
Monday January 24, 2011 00:43:00 EST

On Saturday morning I finally go everything running. Not exactly sure what changed, but here's what I did.

With an idea from Mr. Piwowar to use wget to download the files (didn't know there was a CLI to do that...), I used that method. Previously I had issues downloading from eDelivery using Chrome (I'm not the only one, Mr. Seiler has as well). I had tried Firefox and that wouldn't get it either (the checksums wouldn't match).

I tried wget from my host machine using something like this:
wget <url_to_file> --user=$OTN_USERNAME --password=$OTN_PASSWORD
After downloading, I verified the checksum.

I then went into my guest system and used the same method. The checksum on the first file matched, but not on the second. It actually changed 2 times while calculating it...which is weird. Anyway, I gave the copy across shared folders method a try again. Ran the checksum and it matched. I now had 2 files that were validated against the checksum.

I unzipped and installed the software with no issues. I then ran dbca and...it worked.

I wish I could explain what went wrong and where it went wrong, but I cannot. At least I have a working 11gR2 database now. Time to break it some other way.

Thursday, September 2, 2010

TNS-03505: Failed to resolve name

I've been trying to build an image of OBIEE 11g recently, with not much success.

The process is as follows:
1. Fire up big-ass (to me anyway) Windows 2003/8 Server on Amazon.
2. Go through all the security hassle of Internet Explorer to download the software (I've since put it on it's own volume, and then attach that to my instance, so I don't have to do this 300 times)
3. Install 11gR2 database, software only.
4. DBCA to build database.
5. Net Manager to build the listener.
6. Create the weblogic metadata for OBIEE.
7. Install OBIEE.
8. Scream and yell when the configuration doesn't work.

Rinse and repeat 12 times.

I've gotten quite good at it.

Then I ran into this lovely error...
TNS-03505: Failed to resolve name
Cause: The service name you provided could not be found in TNSNAMES.ORA, an Oracle Names server, or a native naming service.
Action: Verify that you entered the service name correctly. You may need to ensure that the name was entered correctly into the network configuration.

I went through Net Manager and Net Configuration Manager about 1000 times. You know, instead of actually reading the error.

I thought I needed a Loopback Adapter. I didn't.

I started to go through the files by hand, sqlnet.ora, listener.ora, tnsnames.ora (which I'm much more confident with surprisingly)...it had to be something small.
# tnsnames.ora Network Configuration File:
c:\oracle\product\11.2.0\dbhome_1\network\admin\tnsnames.ora
# Generated by Oracle configuration tools.

ORACLR_CONNECTION_DATA =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = 127.0.0.1)(PORT = 1521))
)
(CONNECT_DATA =
(SERVICE_NAME = TESTING)
(SID = TESTING)
)
)

LISTENER=(Address=(PROTOCOL=TCP)(HOST=127.0.0.1)(PORT=1521))
I name all my sandbox databases "testing." Always have, probably always will. From the command line I issue:
SQLPLUS SYS/TESTING@TESTING AS SYSDBA
(Yes, I know I don't need to specify the user/password@service_name).

Spot it yet?

Could it be that my alias says: ORACLR_CONNECTION_DATA?

Yup, foiled again. ID10T in the house.

Friday, April 2, 2010

Database Testing - FIXED_DATE

Years ago I learned this trick from this AskTom post. I was doing a lot of database unit testing with SQLUnit and I had the need to set the clock to a certain date. Of course I could change the system clock, but I found this more useful, especially on my own personal sandbox.

If you do this on a shared database, it will affect everyone, so be careful.
CJUSTICE@TESTING>SELECT TO_CHAR( SYSDATE, 'MM/DD/YYYY HH24:MI:SS' ) d 
FROM DUAL;

D
-------------------
04/02/2010 15:50:35

1 row selected.

CJUSTICE@TESTING>ALTER SYSTEM SET FIXED_DATE = '02-MAR-10';

System altered.

CJUSTICE@TESTING>SELECT TO_CHAR( SYSDATE, 'MM/DD/YYYY HH24:MI:SS' ) d
FROM DUAL;

D
-------------------
03/02/2010 00:00:00

1 row selected.

CJUSTICE@TESTING>ALTER SYSTEM SET FIXED_DATE = 'NONE';

System altered.

Elapsed: 00:00:00.09
CJUSTICE@TESTING>SELECT TO_CHAR( SYSDATE, 'MM/DD/YYYY HH24:MI:SS' ) d
FROM DUAL;

D
-------------------
04/02/2010 16:03:01

1 row selected.
Happy testing.

Monday, March 29, 2010

RDBMS + NoSQL Articles

There seem to be a whole lot of these running around lately. So I'm going to post the ones I know of here and update it as I find new ones. If you know of any, let me know and I'll post them here.

Somewhat related are the ones posted here:

Application Developers vs Database Developers
Application Developers vs. Database Developers: Part II
The "Database is a Bucket" Mentality
-->Everything is a Bit Bucket by Michael O'Neill
---->The Case for the Bit Bucket by Michael Cohen

Here are some of the ones rolling in:

Getting Real about NoSQL and the SQL-Isn't-Scalable Lie by Dennis Forbes
Getting Real about NoSQL and the SQL Performance Lie by Dennis Forbes
I Can't Wait for NoSQL to Die by Ted Dziuba
-->Not Everyone Using noSQL is a Rails-Lovin’ Ass-Clown by m3mnoch (h/t Mr. Cohen)
-->Why NoSQL Will Not Die by Stephan Schmidt (h/t Mr. Cohen)
My Thoughts on NoSQL by Eric Florenzano
Social Media Kills the Database by Bradford Stephens
NoSQL vs. RDBMS: Let the flames begin! by Joe Stump
-->Responding to Joe Stump on the NoSQL Debate by Dennis Forbes

NoSQL vs. RDBMS: Apples and Oranges? by Mike Kavis
Mike seems to be advocating the use of NoSQL type products for the larger datasets, i.e. data warehouses.
Oracle inspires an open source NoSQL tea party by Dana Blankenhorn
Is it Oracle's fault?
Mapping The NoSQL Space by Gwen (Chen) Shapira

Tuesday, March 16, 2010

The Case for the Bit Bucket

By Michael Cohen

Mr. Cohen is most famous here for the discussions we've had before about Application Developers vs. Database Developers. Part I is here, Part II is here. Mr. Cohen is a friend of mine. I have a great deal of respect for him. We obviously disagree in some areas, but I've learned to appreciate his push-back and learn from it.

He had left a comment the yesterday on my most recent post, The "Database is a Bit Bucket" Mentality (Michael O'Neill posted his followup, Everything is a Bit Bucket, as well). I thought the comment would get short shrift, so I asked him to write up a full post. I deleted that comment and here is his post.


Modern RDBMS's are quite powerful today. Pretty much every one of them has full support for SQL, including vendor extensions, all of the features we've come to expect from a relational database, a full fledged programming language built in, and quite often support for extras like full text search or native handling of XML. Most now also now ship with highly feature specific add-ons - PostgreSQL has a geospatial package that makes it the defacto standard in that domain, MySql has hot replication in a master-slave paradigm, Oracle has....well, Oracle has all kinds of things, a full object system and Java inside, a message broker, an HTTP server, a complete UI toolkit, among other things.

So the question arises as to how much of this capability one should use. I think it's becoming apparent that the answer to this is, "not much." Why shouldn't you take advantage of as much of the database's feature set as possible? The answer is performance and scalability. But wait, aren't stored procedures faster than ad hoc queries? Yes (theoretically). Won't it be more performant to execute business logic as close as possible to the data it operates on? Why should we introduce yet another component into the architecture when the database is perfectly capable of handling a particular task?

For one thing, the programming languages and environments offered by relational databases are now relatively long in the tooth, and have been eclipsed by modern OO languages. Developers are much more productive building applications with these new languages, and find it painful and tedious to work within the relational model, with SQL. You can see proof of this now with the overwhelming popularity of ORM frameworks in most of the popular OO languages out there. Java has Hibernate/EJB/JPA and many others. Ruby has ActiveRecord, DataMapper, and Sequel. Python has SqlAlchemy and Djanjo's ORM. And it's not because these developers lack the skills to work with the database directly. Quite the contrary actually, it takes intimate knowledge of the database to work effectively with an ORM. What's more, the ORM is often able to make runtime optimizations that would be difficult or prohibitively time consuming to hand code. Finally, clustered caches offer massive performance and scalability improvements, handling writes back to the database transparently behind the scenes, but for the most part they preclude implementing complex business logic in the database.

The overall trend is clear, across languages and platforms. It's the movement of data out of the database and into the application layer. Less and less reliance on the database, perhaps only for archival purposes. John Davies has a good comment on this. He's operating in a unique environment with extremely rigorous performance requirements, but we're now starting to see similar constraints imposed by the web. There's a whole class of software that has come about due to the inability to scale the relational database beyond a certain point. Facebook developed Cassandra, now used by Twitter, Reddit, and Digg, among others. LinkedIn built Voldemort. My employer doesn't deal with the massive scale of these companies, but we do large scale data processing with Hadoop. HBase, another non-relational persistent data store, is a natural fit, and just about the only option really. We use MySql less and less.

Of course, not everybody is building applications with such high scalability requirements. But even for applications with less intensive scalability requirements I would argue the same tendency to minimize the workload on the database should apply. Cameron Purdy has a good quote, "If you don't pick your bottlenecks, they'll pick you." Design your application to bottleneck, he says. What he means is, your application is going to bottleneck on something, so you need to explicitly decide what it will bottleneck on. Unfortunately, most applications bottleneck on the database, as this is the hardest layer to scale. It's pretty easy to scale the front end, we just throw more instances of Apache out there. It's a little bit harder, but not much, to scale the app server. But it's pretty hard to scale the database tier, particularly for write intensive applications. For well funded organizations, Oracle RAC is the standard. MySql's master-slave setup and hot replication saw it win out over PostgreSQL despite the fact that Postgres is a much better database in just about every other respect. The NoSql projects listed above grew out of the inability even to scale out MySql.

The trend is clear. We're collecting and processing more data than ever before, and this will only increase as we go forward. Unfortunately, the relational database (at least in it's current form) isn't well suited to the scale of data processing an already significant and growing number of organizations deal with on a daily basis. We're now seeing new solutions come forth to address the shortcomings of the traditional RDBMS, and the same forces that have necessitated such developments are at work even in smaller organizations. At all levels, developers would do well to require as little functionality as possible from the database, essentially, to treat it as a bit bucket.

Wednesday, March 10, 2010

Everything is a Bit Bucket

By: Michael O'Neill
@oraclenude
oraclenude.crisatunity.com

In response to Chet's frustration over yet another encounter with a database agnostic, I wanted to contribute my first article to the oraclenerd franchise. My thoughts seemed too long for the comment stream.

I ascribe the kernel of thought behind "the database is a bit bucket" primarily to each and every database vendor that ever existed. Every database vendor, in an effort to persuade users of competitive products to adopt their product, has participated willingly in espousing some core aspect of how "same as the other guy" their product is in addition to whatever differentiation pitch they have.

Now, the generally weak-minded and lazy developer (yes, I think the majority of developers are in fact weak-minded and lazy) latches on to the vendor's selective "sameness" claims for professional and personal reasons. (full disclosure: I am both an Oracle DBA and .NET developer)

Professionally, because they are financially invested in writing third-party code not database code. To them, the less they spend learning and understanding the particulars of things like databases, operating systems, networks, human beings, etc. the better. Personally, because there is a dominate thread in the culture of developers to dismiss the database as interesting or meaningful. It is a form of heresy to show affection towards any platform in any specificity.

This is why Java's Big Lie of "write once, run any where" swoons so many. Java's Big Lie is analogous to "the database is a bit bucket" by declaring that even the language of software code should be as absolutely interchangeable as possible - even at the expense of being cost-effective or useful. There is an unquestioned faith that decoupling everything from everything is a good thing. This faith gives us code that is as far from the simplest thing that could work from the first moment writing the code is undertaken. It is a faith I reject. That's why I'm an ORACLENERD.

P.S. I know oraclenude and oraclenerd is confusing. It's supposed to be.

Tuesday, March 9, 2010

The "Database is a Bucket" Mentality

Front and center again...I just woke up from a nap, I'm grumpy, so I must write. Besides, I haven't had a good rant in quite some time.

Friend of mine asked me last week for some advice, specifically asking if there was a tool to convert Oracle SQL Syntax to the ANSI SQL syntax. (A quick search turned up this (it was the first result), if you're interested).

I had to ask why.

Client is switching to an open source database, i.e. "free." Oracle licensing is way too pricey.

I'm sure Oracle costs a lot of money, it's pretty darn good software. Quite possibly the best in the world especially in the database realm. I've written about the incredibly feature rich goodness that is the Oracle database here here...actually, just trust me. It's in my name.

Why is there even a comparison?

Could it be that everyone out there believes that the sole purpose of a database is to store data? That it can't do anything else? The storage and retrieval of data...that's all it does of course.

It's like saying the Democrats and Republicans are the same...at face value, perhaps, but the devil is in the details.

This, this "Bit Bucket" mentality is what is so incredibly frustrating.

I am no position to argue the differences between the various flavors of database, I lack the experience. But if I were using SQL Server, I would leverage the shit out of it's capabilities. If I were using MySQL, I would leverage the shit out of it's capabilities. If I were using Firebird, I would leverage the shit out of it's capabilities. Same goes for every single flavor out there. Get my point here?

The database is NOT a bit bucket!

Do I need to use more 4-letter words?

I know that Oracle is feature rich and that 99% percent of your code can live in the database...think APEX and PL/SQL. You could probably put ALL of your code inside the database if you wanted to put the javascript in BLOBs as well.

Please, please please quit telling me they are the same...they are not.

Follow up rant by Mr. O'Neill can be found on this following post Everything is a Bit Bucket

Tuesday, February 23, 2010

Connect to HP Neoview using Heterogenous Services

Being the lazy sort that I am and not wanting to pay for a SQL client, I decided to use my local Oracle instance to access a HP Neoview database. Please don't ask me any questions about it, because like much of the world, I don't know either.

Hat(s) off to Tak Tang. I've used his guide a number of times throughout the years. If he blogs, I can't find it, so if you know about it, please link him up. The first time I used it was back in aught (sp?) 5 to connect to a DB2 instance.

Setup
Database (source): Oracle 10gR2
OS: Windows Vista Ultimate running as a VirtualBox Guest on Ubuntu
Database (target): HP Neoview 2.4 (or something)
OS: Doesn't matter

Materials
HP Neoview driver which can be found here. (Thanks Christian)

Steps
Download the HP Neoview driver and install it. Since it is windows, just click away accepting all the defaults.

Next, configure an ODBC Datasource. I won't go into the gory details, but it's pretty easy. When you are done, you should see something like this:

ODBC Datasource

Now comes the fun part.

In your <ORACLE_HOME>\hs\admin directory you should see the following:

HS Folder

Open up the inithsodbc.ora file and set the parameters as follows:
HS_FDS_CONNECT_INFO = NEOVIEW
HS_FDS_TRACE_LEVEL = 1
Save the file as initNEOVIEW.ora in the same directory.

Now traverse to <ORACLE_HOME>\network\admin and open up your listener.ora file. Add a SID_DESC so that your file now looks like this:
LISTENER=
(DESCRIPTION=
(ADDRESS_LIST=
(ADDRESS=(PROTOCOL=tcp)(HOST=localhost)(PORT=1521))
(ADDRESS=(PROTOCOL=ipc)(KEY=extproc))))
SID_LIST_LISTENER=
(SID_LIST=
(SID_DESC=
(GLOBAL_DBNAME=testing)
(ORACLE_HOME=c:\oracle)
(SID_NAME=testing))
(SID_DESC=
(SID_NAME=plsextproc)
(ORACLE_HOME=c:\oracle)
(PROGRAM=extproc))
(SID_DESC=
(SID_NAME=NEOVIEW)
(ORACLE_HOME=c:\oracle)
(PROGRAM=hsodbc)))
Reload or stop and start your listener.

Now open up your tnsnames.ora file and add an entry for NEOVIEW. It should look like this:
NEOVIEW =
(DESCRIPTION=
(ADDRESS_LIST=
(ADDRESS=
(PROTOCOL=TCP)
(HOST=localhost)
(PORT=1521)
)
)
(CONNECT_DATA=(SID=NEOVIEW))
(HS=OK)
)
The last step is to create a database link.
CREATE DATABASE LINK neoview
CONNECT TO "username"
IDENTIFIED BY "password"
USING 'NEOVIEW';
You should be all set now. You can test it out by issuing a simple
SELECT * FROM dual
If that works, you're done.

If you have problems, check out Tak's Troubleshooting section.

Sunday, February 21, 2010

Database Application Security - A Visualization

Security, in regards to the database, is pretty broad. There are a multitude of ways to secure your database. From physical or logical access to SQL Injection.

This is a (not so) pretty picture of how I visualize security from an application point of view.

Secure OLTP Application

The base of any application are the tables. Access to those tables should be limited to database views and PL/SQL packages (API) that are owned by the application (schema).

SELECT privileges on those views should be given to only users who need it (users also includes other applications). Views present the data in the format that is required by the application. VPD (column or row) can be used to further restrict access to sensitive data.

EXECUTE privileges on the APIs (PL/SQL) should only be given to those applications that need it. Under no circumstances, should direct INSERT/UPDATE/DELETE privileges be given to another user.

Much of my thoughts on this matter are directly traceable to reading AskTom for so many years. I can't point to any one specific article, I think it's the accumulation/internalization of reading for so long.

It makes sense though.

Who better to decide the functions (INSERT/UPDATE/DELETE) that can be performed against a given set of tables than the person(s) who created them? If said person has left, make someone else responsible for them. I've fought hard over the years to implement this "pattern" and have been fairly unsuccessful. It is difficult to go against years of doing it one way (full access to the underlying tables) to forcing someone or something to use the API.

The idea to visualize it came to me recently and I need to put it down.

What do you think? Am I full of it? Is this reasonable? Anything I left out?

Friday, November 13, 2009

11gR2: Installfest

I installed Oracle Enterprise Linux 5.4 a few weeks ago in a VirtualBox virtual machine...I just didn't have the inclination to install the database.

Way back in June, I installed VirtualBox, OEL (5.3), 11gR1 and ApEx. To do that, I followed this guide on OTN by John Smiley (thanks John!). This time, I was going to try to do it from the docs. I can't say it was easy to read, so I was back at John's article before to long.

First up was preparing OEL for the database install. This included setting up kernel parameters and loading some extra libraries. There were slight differences between OEL 5.3 and 5.4, but I had the docs handy...and the installer complained loudly if something was amiss.

Next up, mount a shared folder in the virtual machine so that I could keep it to a relatively small size, 15 GB to be exact. Plus, I wanted to use the shared folders functionality. So I created a directory in the /mnt directory called software (as root of course), then mounted the shared folder.
root@oracledb#mkdir software
root@oracledb#mount -t vboxsf software /mnt/software
"software" was the name I gave the shared folder through the VirtualBox interface, that mapped to /opt/software on my host machine (ubuntu 9.04).

Then I downloaded the 11gR2 software, 2 files, linux_11gR2_database_1of2.zip and linux_11gR2_database_2of2.zip. I then used Archive Manager to extract the files. Strangely, it created 2 folders, "database" and "database(2)." That can't be right.

Back to OEL and I cd'd to the directory, /mnt/software/11gR2/database and ran the installer.



I did not provide my email address and I unchecked the box that would send me updates from Oracle Support.



Selected Yes to continue.



I'm just accepting the defaults here.



Not really sure what the difference is between a Desktop Class and Server Class install, so I again accepted the default (Desktop).



Now I have to fill in 4 fields:
* OSDBA Group (drop down)
* Global database name
* Administrative password
* Confirm password



* OSDBA Group = oracle (for me)
* Global database name = testing
* Administrative password = testing, whoops, strong password rules, not testing
* Confirm password = same as above



I chose "oracle" as the oraInventory Group Name.



Some prerequisite checks, to make sure I have the right libraries and the kernel parameters are OK.



This is when I realized the values were different between 11gR1 and 11gR2 and went back to the docs to get the correct ones. I realized afterwards though, that if you select the particular failure, it will tell you what the values should be. Nice.

After I went back and changed the values, I only had a failure on the memory check, but I checked the "Ignore All" checkbox and proceeded forward.



(I missed a screen shot the second go-around, so ignore the OS Kernel Parameters and Package errors.



Confirmation screen.



And were off!



I like to take a lot of pictures.



Creating the database...



Screen behind the screen.



Almost done.



Post-installation tasks.



Running /u01/app/oraInventory/orainstRoot.sh



Running /u01/app/oracle/product/11.2.0/db_home_1/root.sh



Done.



And done.



I can login from the virtual machine...



And I can login from the host machine.



Enterprise Manager from the host machine.



And my new account from the host machine.

Easy right?

Special thanks to Tim Hall as well, he's got a great primer for installing 11gR2, Oracle Database 11g Release 2 (11.2) Installation On Enterprise Linux 5.0.

Almost forgot, my problem with Archive Manager creating "database" and "database(2)"? Solved by twitter naturally:





So I unzipped the 2 files from a terminal window and it (apparently) put everything in the same folder. Win.

Monday, November 9, 2009

Oracle Press Release on Sun Merger

Original can be found here:
Redwood Shores, Calif. - November 9, 2009

Oracle's acquisition of Sun is essential for competition in the high end server market, for revitalizing Sparc and Solaris and for strengthening the Java development platform. The transaction does not threaten to reduce competition in the slightest, including in the database market. The Commission's Statement of Objections reveals a profound misunderstanding of both database competition and open source dynamics. It is well understood by those knowledgeable about open source software that because MySQL is open source, it cannot be controlled by anyone. That is the whole point of open source.

The database market is intensely competitive with at least eight strong players, including IBM, Microsoft, Sybase and three distinct open source vendors. Oracle and MySQL are very different database products. There is no basis in European law for objecting to a merger of two among eight firms selling differentiated products. Mergers like this occur regularly and have not been prohibited by United States or European regulators in decades.

The U.S. Department of Justice carefully reviewed the proposed acquisition during the normal Hart-Scott-Rodino review and considered it again when the European Commission initiated a second phase review. On both occasions the Justice Department came to the conclusion that there is nothing anticompetitive about the deal, including specifically Oracle’s acquisition of the MySQL database product. The U.S. Department of Justice approved the acquisition without conditions and terminated the waiting period under the Hart-Scott-Rodino Act on August 20, 2009.

Sun's customers universally support this merger and do not benefit from the continued uncertainty and delay. Oracle plans to vigorously oppose the Commission’s Statement of Objections as the evidence against the Commission’s position is overwhelming. Given the lack of any credible theory or evidence of competitive harm, we are confident we will ultimately obtain unconditional clearance of the transaction.
Apparently the European Commission, the European Union's executive arm, issued a statement of objections on Monday. Sun issued a statement as well:
On November 9, 2009, the European Commission issued a statement of objections relating to the acquisition of Sun by Oracle Corporation. The Statement of Objections sets out the Commission's preliminary assessment regarding, and is limited to, the combination of Sun's open source MySQL database product with Oracle's enterprise database products and its potential negative effects on competition in the market for database products. The issuing of a Statement of Objections allows addressees to present arguments in response to the Commission's preliminary assessment of the competitive effects of a notified transaction. A Statement of Objections is a preparatory document that does not prejudge the European Commission's final decision. Any final decision by the European Commission is subject to appeal to the European Court of First Instance.
I'm sorry, but the EU just doesn't get it. First, it's open source. Do they really lack the technical acumen to determine what this means?

Second, there is plenty of competition out there. DB2, SQL Server, Sybase, Intercache, BTree...not to mention Postgres and other open source alternatives. This is just embarassing.

Update
Looks like the Department of Justice got into the act as well:
WASHINGTON - Deputy Assistant Attorney General Molly Boast of the Department of Justice's Antitrust Division issued the following statement today after the European Commission (EC) issued a statement of objections regarding Oracle Corporation's proposed acquisition of Sun Microsystems Inc.:

"After conducting a careful investigation of the proposed transaction between Oracle and Sun, the Department's Antitrust Division concluded that the merger is unlikely to be anticompetitive. This conclusion was based on the particular facts of the transaction and the Division's prior investigations in the relevant industries. The investigation included gathering statements from a variety of industry participants and a review of the parties' internal business documents. At this point in its process, it appears that the EC holds a different view. We remain hopeful that the parties and the EC will reach a speedy resolution that benefits consumers in the Commission's jurisdiction.

"Several factors led the Division to conclude that the proposed transaction is unlikely to be anticompetitive. There are many open-source and proprietary database competitors. The Division concluded, based on the specific facts at issue in the transaction, that consumer harm is unlikely because customers would continue to have choices from a variety of well established and widely accepted database products. The Department also concluded that there is a large community of developers and users of Sun's open source database with significant expertise in maintaining and improving the software, and who could support a derivative version of it.

"The Department and the European Commission have a strong and positive relationship on competition policy matters. The two competition authorities have enjoyed close and cooperative relations. The Antitrust Division will continue to work constructively with the EC and competition authorities in other jurisdictions to preserve sound antitrust enforcement policies that benefit consumers around the world."
For the life of me I can't find the "official" statement of objection by the European Commission.

Thursday, November 5, 2009

Method R + C.J. Date = Fun!

I just got an email blast from Mr. Cary Millsap.

It reads as follows:
This brief note is to update you on some of the things we've been doing at Method R Corporation.

C. J. Date seminar - There's a new course on our schedule: we've invited Chris Date to teach "How to Write Accurate SQL and Know It: A Relational Approach to SQL" January 26-28, 2010 in the Dallas/Fort Worth area. Our make-or-break date for the course is December 4, 2009, so if you're interested please enroll soon. I'm personally excited about the opportunity to sit in a classroom for three days with one of the inventors of our whole industry. http://method-r.com/education/107-cj-date-course.

MR Tools - A few weeks ago, we released a new set of software tools that let you do some pretty unbelievable things with Oracle trace data. Take a look at our Guided Tour pages behind the individual tool links at http://method-r.com/software/mrtools.

Method R Profiler - Recently, we released Method R Profiler 5.2.0. The Profiler now runs on Mac and Linux, in addition to Windows. And new with 5.2 is a graphical user interface that's a genuine pleasure to use. I was always a command line guy when I used the Profiler, but with the new 5.2 GUI, I'm sold. http://method-r.com/software/profiler.
The tools are currently a bit out of my league, but the opportunity to listen to C.J. Date would be cool (cool as in geek chic or nerd chic of course). I have no idea what my schedule will be like at this point, but I would definitely like to attend. Hopefully I'll have some things tied up so I can plan for it.

Tuesday, October 27, 2009

Free Oracle Developer/dba

There is a serious lack of work in the Tampa market and desperate times call for desperate measures.

Now, I've always wanted to do this, but was never in a position to do so financially...I'm still not, but something is way better than nothing.

I'm going to offer my services for free.

Not forever mind you, somewhere between 2 and 4 weeks. It is negotiable.

If it works out, i.e. you like what I can bring to your organization, I prove that I pick new systems up quickly and I fit in well with your team; and I like working in your organization, then you pay me for that time worked and I sign up full-time either as a permanent employee or some sort of contract.

If it doesn't work out, we part ways and no one is worse for wear. You get free work from me and I get to experience a new environment (i.e. meet new people, expand my network, etc).

You can view my resume here (permanent link up top coming soon).

Some basic highlights:
PL/SQL: Expert (I don't like that term)
DBA: Junior to Mid-level (or DBA in lowercase, "dba")
APEX: Worked with up to version 3.2, admittedly rusty, but passionate about the product.
Design: Love modeling data. Model just about everything in my head (in normal life). Use Visio extensively for visual representations. Experience with SQL Developer Data Modeler, ERwin, etc.
Documentation: Give me a wiki or something similar and I'll document just about everything that I do.

That's it. Contact info is on my resume or up top on the "email" icon.

It's now posted on craigslist as well.

Monday, October 26, 2009

Followup Question for Mr. Ellison

In my first year in IT, I was fortunate enough to be sponsored by my employer to go to Oracle OpenWorld. This was way back in 2002. Hard to remember that far back isn't it?

Anyway, I attended Larry's keynote, and if I remember correctly, he wasn't there. He was in Australia (or somewhere) in the middle of a some sort of boat race. I do remember being dared, during the question and answer period, to ask him for a job. Amazingly I didn't do it.

One thing that stuck with me though...

He was talking about the internal operations of Oracle. He stated that there were something like 500 databases in use throughout the company. With those 500 databases there was the requisite staff to support those databases. His goal was to get all of Oracle into a single database.

So my question to Larry, or anyone else that might remember this particular speech; Did this ever happen?

I was reminded of this last week at the SOUG meeting. Charlie Garry gave a "where we have been" type presentation. The title of the first slide was Oracle Database: 2000-2009, which I thought was a misprint, until he got started. The gist of the talk was that way back in 1999 (before I was IT born), they set out a plan for the database for the next 10 years. I can't remember specifics because that's the time I started to come down with The Plague. In essence, this phase of Oracle database development has been completed with the release of 11gR2. Pretty interesting presentation (stupid Plague).

* I still have The Plague, so pardon the haphazard ramblings

Specifically what reminded me of Mr. Ellison's statement back in 2002 was Mr. Garry telling us that all of his files were in a database somewhere.

Seven years later, I'm asking the followup, Did Oracle ever get down to a single database?