Learning Pentaho Data Integrator

aka Kettle, aka PDI.

I've recently taken on a data integration gig and I'll be using Pentaho Data Integrator (PDI). I've read about Pentaho for years but never got around to actually using it. I'm excited about the opportunity.

What is PDI?

...delivers powerful Extraction, Transformation and Loading (ETL) capabilities using an innovative, metadata-driven approach. With an intuitive, graphical, drag and drop design environment, and a proven, scalable, standards-based architecture, Pentaho Data Integration is increasingly the choice for organizations over traditional, proprietary ETL or data integration tools.

I'll be using the enterprise edition (EE), which is supported, similar to how RedHat works...I think.

This post if mainly for me, naturally. I'm going to list out the references I've found so far and add to it over time. Similar to what I do (err, did) for Learning OBIEE.

Actually, I'll just start with the helpful email I received after being added to the account.

• Support - https://pentaho.zendesk.com/home - ask questions of the engineers.
• Evaluation Resources - http://www.pentahoevalcenter.com - great technical resource for those new to the tools – recorded tutorials as well - NOTE the drop downs.
• Documentation - http://infocenter.pentaho.com/help/index.jsp
• Download - and run the full suite (or just Pentaho Data Integration) right on your own computer. Perfect for eval, test/dev, training. Getting started docs included: http://www.pentaho.com/download/
• JIRA Tracking - http://jira.pentaho.com - you can request and track features and fixes – triaged twice week by our engineering team. As a reminder, let Support enter any JIRA tickets you want to open.

I love that you can download and play with the software yourself. Of course the Community Edition (CE) is open source, so that makes sense. I'm not sure if you can get the EE version for free though.

There's a community page as well with links to a lot of great resources. So far, my favorite has to be the IRC channel hosted at freenode. Note, there are two hashtags as in ##pentaho. I've been lurking there for a few weeks and finally got up the nerve (what? me shy?) last week. HansVA and mburgess_pdi helped get me moving again on a particular problem. Good stuff.

I'm sure I'll add more as time goes on. That's it for now.

Update after original posting...

• (Kettle, aka PDI) Wiki

The Fun of Oracle OpenWorld

Another one in the books.

What did I accomplish while there?

1. I helped organize a Sunday Symposium for ODTUG. I convinced 9 very busy, talented and gracious people to speak. I couldn't have imagined that just a few short years ago.
2. Oracle ACE Dinner
3. Interview with Bob Rhubart
4. I swam in the Bay with other crazy people.
5. Caught a few sessions at Oak Table World
• Karl Arao
• Cary Millsap
• Tanel Poder
6. RAC Attack
7. ...

I had this long list in my head, but I can't remember everything I did.

I got to play catch with Mr. Millsap. That's two conferences in a row, I plan on keeping that streak alive.

What I remember most are the people. Shocking, I know.

- Lunch with Floyd Teter and Jake
- Seeing my old (first IT) boss for the first time in years.
- Being yelled at by people who said I was mean, that I never gave them props for their hard work. Ahem.
- Walking around San Francisco with Jeff Smith and testing the water for Monday's swim.
- The view. Man, everywhere had a gorgeous view (I know, that's not a person).
- The ODTUG crew. Mike, Crystal, Monty, Lauren, Bambi, Jerry, John and Barbara. It's a fun group of people who make work fun, and it shows.
- Catching up with Bex.
- Hanging out with Lillian and Vikki.
- The Blogger's meet-up sponsored by Pythian (among many events). Fuad Arshad has the best pictures too. I'll end up stealing a few of those I'm sure.
- Dinner at the Pier Market with an incredible group of people. Dirty jokes ensued (jam vs. jelly) and then Connor McDonald brought down the house. I couldn't breathe for about 5 minutes I was laughing so hard.
- The walk home from the Pier Market listening to Cary and Greg Rahn talk.
- Foley's. Joe, Tom and Bex.
- Dinner with Jake, David and Terrence. A repeat of 2 years ago. It felt like it had only been a day. That's pretty cool.
- Bjoern Rost. Wow, that guy is full of life. So much fun to be around.
- Tim and Kellyn. Good people.
- And my remora, Alex. I mean that in a good way too. His first time at OOW and he wasn't shy, that's not always easy to do.
I'm sure I could do this all night.

I learned a lot of cool things in San Francisco which I can use immediately. For me it's the opportunity to meet a lot of fun, intelligent, kind people in a very short amount of time.

Oracle OpenWorld - Swim in the Bay 2012

Last Monday at 7:30 AM PST, 17 brave souls joined me in the First Annual Oracle OpenWorld Swim in the Bay, a refreshing swim in San Francisco waters.

To be precise, it was at the Aquatic Part near the Dolphin Swim & Boat Club. OK, here's the map to make it easier.



And here's a pretty panoramic:

From Oracle Open World 2012

At my very first OOW, I did this swim 3 or 4 times. I was in much better shape back then. This time, I just thought it would be a fun, easy diversion with a chance to bond with fellow travelers. On Saturday, Jeff Smith and I wandered over to the "beach" and got our feet wet. Here's my interview with him:



Here's part of the crew before disrobing



Note John Hurley's cow hat, that's John in the lime green tank top over to the left. Others pictured here from left to right: Kent Graziano, John Hurley, Bjoern Rost, Gustavo Rene Antunez, I believe the yellow green jacket is Henning Voss, Stanley the ACE Director sitting on Debra Lilley's lap, another Bjorn (help please?) Björn Ole Voß, Connor McDonald and Mogens Norgaard. Now Connor had an unfair advantage, he's currently training for an English Channel Crossing, so I'll photoshop all future pics of him to make him look less like a superhero.

Now, here's the big group shot, disrobed this time. If you have an issue with bright lights, cover your eyes now.



From left to right: Me, Gustavo, Alex Crane, Jeremiah Wilton, Connor McDonald (who we made stand in the back so we wouldn't look so bad), Debra Lilley and Stanley the ACE Director, Bjoern Rost, Bjorn (help again), Sheeri Cabral, Henning Voss, John Hurley, I can't remember his name, help Björn Ole Voß and finally Mogens again. Not pictured: Kent Graziano and Tom Wurzbach (the guy who gave you me, i.e. my first boss).

Debra getting her feet wet...sparing Stanley the cold. In the background, you can see Sheeri Cabral and Alex Crane (I think).



And finally Kent Graziano all wet.



Good times.

Did you notice I said First Annual up at the top? Well, I did. We're doing this again next year and we have sponsors! Nothing terribly fancy, perhaps a ribbon for your badge saying, "I swam in the Bay." Or ORACLENERD swim caps. Or coffee, lots of coffee after the swim to warm up. The goal is 50 people. If you're planning on going to OOW next year, join us!

Origins of ORACLENERD, katezilla and Stuff

On Sunday I attended the Oracle ACE dinner. This is an annual gathering at OOW of the ACE and ACE Directors who are in attendance (lots). The venue was the San Francisco Yacht Club. Very nice. They made me take off my hat, which I usually do inside, so it was fancy too.

Bob Rhubart of OTN Architect Community fame, asked if I would share the story of the origins of ORACLENERD, how it came about. Since I have a problem saying No, I agreed.

I wasn't very concise about the origins, I rambled for a few minutes finally getting on point. Then I started to talk about katezilla...

Long, tiring day, plus an adult beverage on board and I was just spilling my guts. Near the end, I think I cried, on TV. I asked Bob if he could see tears, he said No.

The following morning a friend of mine in Europe told me he saw the video. I asked him if he saw tears. He said, "No, I was at work, trying not to show emotions, but it was very powerful."

To which I replied, "I was talking about me. :)"

Anyway, katezilla comes on about 6:00 minutes, skip to there if you don't want to hear the whole thing.

Oracle OpenWorld 2012 Schedule

Still working on my general sessions throughout the week, but it's pretty packed already.

Sunday's events are part of ODTUG @ OpenWorld.

The Swim event.

If you have any suggestions for me, let me know.

Custom ORACLENERD T-Shirts

Jeff Smith has said for some time now that I need to market the t-shirts better than I do. Selling them has never been a priority. Yes, it's pretty cool seeing the t-shirts out and about. It's cool that people can express their inner oraclenerd like I seem to do on a daily basis. The real reason I put effort into t-shirts and the like, is katezilla.

Recently we got hit with a couple of things: 1, a $190 iPad app for Kate and 2, her ABA therapy co-pays finally came due. For #1, I ran a GoFundMe campaign and encouraged you to buy t-shirts. The GoFundMe campaign raised $420 in less than a day, more than covering the cost of Kate's iPad app. I used the remaining funds to make a payment on #2, her ABA co-pays. I also sold about 10 shirts just after that post and close to 20 in total since.

I am constantly humbled at your generosity.

What does all of this have to do with the title? Well, people have asked for shirts either through Twitter, IM or email. Shirts with specific sayings or different styles. Two weekends ago on a Friday night, Don Seiler suggested a hoodie. Living in Florida I had never thought about it, but he lives in Wisconsin where it gets a little cold. We went back and forth on twitter, I would go on Spreadshirt and spin something up, screenshot it, and send it across the wire. On Monday I had a final product and I named it after Mr. Seiler.

Mr. Seiler (picture) ordered one and so did Mr. Smith. Awesome.

The Don Seiler Hoodie	The German Austrian

The other shirt there is the German Austrian edition. Martin Berger was responsible for that one (I have a Cyrillic and Russian version in the hopper for Greg Rahn).

Late last week I talked to Lisa Dobson. She got one last year for OpenWorld and wanted a new one, with a twist.

The Lisa Dobson	The Lisa Dobson Pink

Upside down logo (she's British). So I named it after her. Note, the pink version is only available in the US, that brand isn't carried on the European Spreadshirt site.

So, if you have an idea (Mr. Seiler has designed two now, the long sleeve baseball jersey and the hoodie), send it to me and I'll spin something up for you.

How could I forget the APEX version (Joel Kallman)? Or the OBIEE version (Adrian Ward)?

Because I'm forgetful. I'm sure there are others.

Just so it's clear, there are two shops: North America and Europe. They're not exactly the same as it's two separate systems, but I'm trying to keep them in sync.

Send me your suggestions...all proceeds go to a great cause katezilla.

ODTUG @ Oracle OpenWorld

I really need to learn how to say no.

I, somehow, got roped (pun intended) into helping to put together a Sunday Symposium for ODTUG at Oracle OpenWorld this year. If you've been following along for the last couple of years, you'd know that I have this thing for DevOps (devops to me). For KScope 2011, I tried to put together a Sunday Symposium on this very topic. Lewis slapped me down though, "Tools Justice, tools." So we amended the Sunday Symposium to focus on the tools while doing those devopsy kind of things. Much better.

This year I tried to do the same thing. (Side note: The reason I am so "obsessed" with this topic is because I think things like testing, source control, and testing are done very poorly in the database world, the Oracle database world specifically. I know there are places that do it right, my first organization did it right, and then made me a bitter old cynic at each successive organization I came across. I also realize that doing some of these things on/in the database are much more difficult than wrapping up a binary or a web application. Data. It usually comes down to the data. Blah blah blah.). Kris Rice of Oracle SQL Developer fame ran KScope 12's Sunday Symposium...I gave him the topic, and he made it happen. Bonus points for Kris as he won the ODTUG Volunteer Contributor of the Year Award. You can find the slides from that Symposium here.

So here I am again, pushing the methodology or process involved around Oracle Database development. I think it's important. Very important. Fortunately I have a mentor who has spent way more time thinking about this than I. That would be Dominic Delmolino. Cary Millsap had sent me a presentation that Dominic had done at UKOUG in 2010. It was fantastic and about 23 hours worth of material. That was the original inspiration behind the 2011 KScope Sunday Symposium and I have tried to kept it going ever since.

So, back to the ODTUG Sunday Symposium at Oracle OpenWorld this year.

When: Sunday September 30th, 2012, 8:00 AM - 4:30 PM
Where: Moscone West - 2009
Who: Anyone can attend (provided you have an OOW pass of course)
Title: From Dev to Ops and Back Again: Insight into the Development Lifecycle from the ODTUG Experts
Presenters:

• Gwen Shapira - Pythian, Oracle ACE, Oak Table
• Dominic Delmolino - Agilex, Oak Table
• Timothy J. Gorman - Evergreen Database Technologies, Oracle ACE Director, Oak Table
• Robyn Sands - Oracle, Oak Table
• Stewart Bryson - RittmanMead, Oracle ACE
• Kellyn Pot'vin - Enkitec, Oracle ACE
• Dan Norris - Oracle, Oracle ACE Director Alumnus, Oak Table
• Cary Millsap - Method R, Oracle ACE Director, Oak Table

Pretty impressive line-up isn't it?

I'll be posting more details in the days and weeks to come. If you're heading to Oracle OpenWorld this year, check out one or more of these sessions, you won't be disappointed.

Pearl Jam + Oracle OpenWorld

So, it was announced today that Pearl Jam would be playing at Oracle OpenWorld's appreciation event today. You can read the news here.



For about 6 months now I've been thinking how cool it would be to get someone from the band to wear an oraclenerd shirt. Originally I had approached staff at Oracle about somehow getting in touch with the band, but that wasn't to be. "Think, McFly, think." Umm, Twitter anyone? How about a campaign on twitter? Yeah, that'll do it.

hey @pearljam - do you think I can talk one of you into wearing one of these t-shirts at #oow12? lh3.googleusercontent.com/-Swnem86AcYc/T…

— oraclenerd (@oraclenerd) August 14, 2012

Did I mention that all of this helps pay for katezilla? Yeah, it does. Let Eddie know!

Help me out here. Let's see if we can make this happen.

katezilla + AAC

About 8 months ago one of the staff at Sydney's School suggested we let katezilla try out one of the Augmentative and Alternative Communication (AAC) iPad apps. My parents let us use their iPad (v1) and bought the app that was suggested, My First AAC.

She's mostly used this at school, only really getting into it at home over the last month or three.

Fortunately, she has outgrown it. I say fortunately because it's great for her, but "bad" for us. I like to joke and say that Kate will have to pay for her own college and wedding as she's spent all of our money (I've been saying this since she was two). Of course, it's not true. I would spend all the money in the world on Kate if I had it.

So she's outgrown the current app. Her teacher has suggested that we upgrade to Proloquo2Go. Here it is on iTunes.

$189.99

Now, our insurance has been great, a lot better than we initially expected. Most of her therapy is covered...but we've learned recently that we've missed her co-pays...going back to January. Ugh. Would have been way easier to deal with on a weekly or monthly basis, but staring at it with 6 months, not so fun.

More bright side, her tuition has gone down this month and for the upcoming year. Yay for that!

So how can I help?

Great question. I figured I'd do a t-shirt drive to help pay for Kate's AAC app. Oracle OpenWorld is coming up and if you're going, you need a shirt. Even if you're not going, you want one of these...I know you do. Go here if you're in the US.

I've heard from a few of you that shipping across the pond is simply cost-prohibitive. It should be easier for you now, I created a European "shop" which you can find here. No more excuses. Go buy one.

Easier:
There are a few others (and I obviously need to study my geography a bit better).

Go buy one. Please.

(Apologies to those in other countries, India for example. Or continents, Latin America. Working on that. Hit me up if you're going to OOW, I'll bring some shirts with me so you don't have to pay for shipping.)



Update 2012/08/06 01:22 EST

Some people don't want or already have a t-shirt. If you'd like to simply donate, you can. Here. Actually, everything has a widget these days:

Update 2012/08/06 12:20 EST
Wow! Made goal in less than 12 hours. How awesome is that?



Finally, some katezilla goodness. Just in case you needed a reminder of why I do this stuff. She's interspersed throughout the video. The girl with the long black hair with a flower in it is one of Kate's friends.

Kscope 2012 Recap

The Venue


The event took place this year in hotter-than-Hades San Antonio, Texas. Fortunately for all of us, the hotel and conference were in a single (Texas-sized) building; the JW Marriott San Anonio.

Just 20 minutes from downtown San Antonio and the airport, the JW Marriott Resort is situated in Texas Hill Country in Cibolo Canyons, where crystal clear streams and towering oak and cedar trees meet the majesty of the hills. The elements of authentic roots, hacienda style, beautiful views and healing waters serve as the inspiration in bringing this magnificent resort to life. The Hill Country resort features 265,000-square-feet of meeting and event space, the 36 hole TPC San Antonio and the sophisticated Lantana Spa.

Let's not forget the Lazy River, where many of my compatriots...lost themselves.

The River Bluff Water Experience offers six acres of heated pools, fountains, waterfalls, rivers, and an abundance of poolside lounges to soak in the sun.

Pretty awesome.

One other thing about the venue, I don't recall a single complaint or tweet about wi-fi. I've not been to a conference where someone doesn't complain about that. Well done Marriott, well done.

The People
If you known me for any length of time, you probably know this is my absolutely favorite part. To me, people are bright shiny objects. This sums it up nicely as well, a friend said this:

OH: "I don't think the world has enough people for me to feel like I have enough friends - all are welcome"

— oraclenerd (@oraclenerd) March 16, 2012

Tough to argue with that.

Day -1 I met Jeff Smith and Dan McGhan at the airport. Well, we didn't plan on meeting at the airport, but we happened to be there at the same time. Anyway, Jeff gave us a ride to the Marriott.

In the lobby we met Patrick Barel and Galo Balda, who we had been planning (via Facebook) on going to The Riverwalk and The Alamo that day. Unfortunately for Patrick, he's of the Dutch persuasion and I had recently seen Austin Powers in Goldmember. Now, I really have no context for the "hatred" of the Dutch, but it's funny, so I went with it. I'm sure others would be proud.



Where do you get a chance to publicly torture Don Seiler? Kscope. I'm sure this battle will escalate.



You can see all the pictures I took at Kscope here.

The Party
Armadillo racing? Check.
Sitting on a Texas Longhorn. Check.
Barbeque. Check.
Bull Riding (mechanical). Check.
Rodeo. Check!

Last year it was on the Queen Mary with Busta Move artist Young MC. This year, a full blown rodeo at the Knibbe Ranch. I'll spare you the picture of my war-wound from being thrown by the bull. In my defense, it was night. I may or may not have imbibed an adult beverage or two.



A friend and co-worker was even spotted chasing after a small bovine for a chance to win a pass to Kscope 2013. You should see the video. Fun. Speaking of videos, there are videos of Lisa Dobson riding the bull, Jeff Smith, Tim Gorman and Kellyn Pot'Vin out there. If you're nice, maybe I'll share.

My favorite moment though is definitely this one:



That's Danny Bryant awesomesauce.

The Staff


The YCC staff, minus Kathleen McCasland. From left to right: Aimee, Brianne, Larissa, Lori, Bo (err, Lauren), Melissa, Heather and Crystal (the new Executive Director).

It's difficult to say enough about this team. They're excellent at what they do and have a lot of fun doing it.

Let's not forget the ODTUG Board.

• Monty Latiolais (pronounced like Frito-lay) - President
• Barbara Morris - Vice President
• Jerry Ireland - Treasurer
• Bambi Price - Secretary
• John King - Director
• Cameron Lackpour - Director
• Mike Riley - Director (and former President)
• Tim Tow - Director
• Martin D'Souza - Director
• Edward Roske - Not Sure (site's been updated, and I'm lazy right now. I just know that he's everywhere and part time comedian.)

The Board members are all volunteers. They give up a significant amount of their own time to help put this event together. They are one of the bravest groups too...they got on stage for the General Session in singlets, you know those things wrestlers wear? I know way too much about some of them. Consider yourself lucky I am not including a picture.

I think that's part of what makes them great though, their ability to put themselves out there like that.

So maybe now you're starting to think, "Man, that would be a great conference to geek out at." You're in luck because early bird registration is open and next year's event is in New Orleans.

OOW 2012 Twitter List

I've been urged to provide a Twitter list for OOW 2012 attendees. You can see last year's list here.





Check out the Storify story of how it unfolded on Twitter (love Storify).

KScope 2012 Advanced Registration Ends June 9th!

It's almost here. Kscope 2012. Just 18 more days.

What's that? You're not going?

Need to justify the trip? Check out the bottom of the registration page here. There are some helpful hints for Developers, Managers and BI/EPM Professionals.

On June 9th, Advance Registration ends. If you hurry, you can save $300. Go now.

If you do sign up in the next three days AND show proof of such, I'll bring you an ORACLENERD T-Shirt (white on black, the classic). Limit 5. Donations always accepted (joke, I won't make you donate...unless I've had a beer, then I'll maybe hassle you for a few minutes, but nothing too harsh, I promise).

KScope + DevOps

Last year I had the pleasure of getting the Sunday Symposium together for KScope 11, this year, I have completed my takeover of the Database track by becoming the track lead.

I thought this was the best job ever, then I was attacked Nancy Kerrigan style by my handlers.

All that said, I think I've gathered a pretty good group of people to help review and select the abstracts for next year's conference (San Antonio, TX).

There will be 4 sub-tracks this year:
- Design/Data Modeling
- Maintenance (Performance, Tuning, Upgrades)
- MySQL
- (Dev)Operations

The one I am most excited about is the (Dev)Operations sub-track, aka, DevOps.

What is DevOps?

I'm glad you asked..

"DevOps" is an emerging set of principles, methods and practices for communication, collaboration and integration between software development (application/software engineering) and IT operations (systems administration/infrastructure) professionals.[1] It has developed in response to the emerging understanding of the interdependence and importance of both the development and operations disciplines in meeting an organization's goal of rapidly producing software products and services.

I am not necessarily a fan of the movement, but I am a fan of the principles behind it.

Every developer has a story about working with an evil DBA. LIkewise, every DBA has a story about some application that went to production where they were left completely out of the process.

But it is more than just a simple, "Can't we all just get along?" plea, this is about creating better software and streamlining processes.

My personal experience has been one of woeful cooperation, at any level. Our thought, our hope, is that this well help give other Oracle professionals better ideas on how to start down this road.

If you are interested in this topic, sign up. If you want to present on this (or any other) topic, register here.