Is it defined by volume? Size?
Structured, unstructured (content management systems)?
Naturally, the first thing I did was ask the Twitter Machine. It's like sitting in a giant room and yelling out questions to a wide variety of people, in other words, fun.
From the one year older John Piwowar:
Definitely agree with that sentiment. It is just ones and zeros. Lots of it, apparently.
I think my question goes deeper though. I have a pretty good understanding of structured data, i.e. that used by the majority of business applications. You know, you define a data model, it changes over time, no big deal. But...and it's a giant but, what about all that unstructured stuff?
Next up, future beer drinking buddy and obvious fellow smart-ass.
A couple of non-believers stroll in...
I sometimes wonder, do we, as in database people, who work in the data information business, somehow miss the boat on things like Big Data? I can't imagine that's possible. We know the importance of data...is there a disconnect? Is it just me? That ain't a swipe at the above people either.
Gary chimes in with a more...philosophical answer? I've read Gary's stuff for a few years and once in awhile, he just goes way over my head...
Ted comes in and follows up to me:
There were definitely more, but I think these were the highlights for me.
Then it was off to the Google Machine. I love The Google.
Wikipedia to the rescue:
Big Data is a term applied to data sets whose size is beyond the ability of commonly used software tools to capture, manage, and process the data within a tolerable elapsed time. Big data sizes are a constantly moving target currently ranging from a few dozen terabytes to many petabytes of data in a single data set.
Examples include web logs, RFID, sensor networks, social networks, Internet text and documents, Internet search indexing, call detail records, genomics, astronomy, biological research, military surveillance, medical records, photography archives, video archives, and large scale eCommerce.
Definitely coming together now...and we're back to John's point, it's just ones and zeros.
I'll let you use The Google Machine yourself as well. Lots of good articles.
For a really quick (4 minutes) primer, here is some dude from O'Reilly
Thanks everyone for their responses on twitter. Definitely helped to clear up some of my confusion.