That method produced a stream (that's what it's called in PDI) with each and every file and each and every record in those files. If I were just loading that into a table, it would have worked. However, I was assigning a identifier to each file using a database sequence. I needed a sequence for each file, but I wasn't getting it.
With some help and pointers from the ##pentaho IRC channel, I found this post (more on that one in the future), Run Kettle Job for each Row. I downloaded the sample provided to see how it worked.
![](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimg_opO6SQJ8fsY7rG0gIe6yjawNaxn9h6Zr3ZzTBTsP3HwI6094ofZYCDKUQCdJUe5fwvQhQ_24koQMQgxkzDT9Z1-ROQ37_h6NpplmC-_7n8rFNfJXwcqg9KiFJAj9mPNMIkYJciPeTE/s800/Screenshot%2520from%25202012-12-20%252021%253A04%253A27.jpg)
The calc dates transformation just generates a lot of rows. Not much to see there. The magic, at least for me, was in the run for each row job entry.
![](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgLlfucfyaDlswKEJKeuc7BY3gO82-3_KIF3Wxn5SH23hG5KfkP1KBSyh6h_SW03kWiVL8zz_IG29hQSPHI_hl9ORW6iyFDpueuWn261M_jX2oU6BhOYbAEGzK4G40tEDn81tuIJdMDqPB9/s800/Screenshot%2520from%25202012-12-20%252021%253A06%253A20.jpg)
Specifically, the Write to log step. (I have this need to see things, since I don't understand everything about the tool yet, Write to log provides me that ability.)
![](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgT1J4O4l8yTl0v6WKrMsTH8BDu7lKXvEaxNxx5HH9Nhc4JSU8WMdrnIf_Y99LSnlj6BoDDHWvWQI290u-LNfp47Vbishlv9EqYQ4uE-qcw1mf6XP7kqj6E_xz7ZFFxZuny3_J4U6c6nLZQ/s800/Screenshot%2520from%25202012-12-20%252021%253A08%253A50.jpg)
See date, better, ${date}? That's how you reference parameter and variables.
I ran the job and watched the date scroll by. Nice. Then I tried to plug it into my job.
Zippo. Instead of seeing, "this is my filename: /data/pentaho/blah/test.csv" in the log output, I just saw "this is my filename:" Ugh. I went back to the sample and plugged in my stuff. It worked. Yay. Went back to mine, it didn't. Gah! I tried changing the names, then I'd just see "this is my filename: ${new_parameter_name}" so it wasn't resolving to the value.
Finally...after comparing XML for the sample file and mine and finding no real differences, I just about gave up.
One last gasp though, I went to the IRC channel and asked if there was some way to see the job or transformation settings. No one was home. I tried right-clicking to bring up the context menu and there was Job Settings
![](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGhE_BwYW69xyewGtOFpKXeZsVCBZOr7-_lPI-A8STiq8LeTsA613Ef0B5qsAF6Mb_ibJdMnlv4yzl6Bn8UyoSHXKNRJCIO3f65yhgZLDyeAPV9BJA54YQmtU6GCYuAxueGl3h7w7gFZpb/s800/Screenshot%2520from%25202012-12-20%252021%253A17%253A05.jpg)
Job Settings brought up this one:
![](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiD2iwFDPFwSfmlJ_vTb_hQJXWuiJ2WisUad4peTylujBQ12aDfeaziudmrllmXeMsHKKRYbASSqUf7EcykA375FyK867fBnS2zrcvnJmKjizRAQ9uUXj25bTt4mUQpttwWKdJdUUtKbo-z/s800/Screenshot%2520from%25202012-12-20%252021%253A18%253A56.jpg)
date is defined there. I checked mine. Nothing defined. Added filename to mine, ran it, Success!
No comments:
Post a Comment