Problem with “Big Data” is the more you have, the more you have to sort to get anything useful, and the more misleading paths are possible. Curation is necessary by humans using logic, critical thinking and especially instincts gained through years of experience.
We are told to follow the data and the truth will be revealed, but data tells many tales and it depends on the data and how you interpret it. It makes me wonder if anything is definitive if you can present two similar sets of data and draw wildly different conclusions, depending on your emphasis. That’s because data is a tool in the hands of humans and we can interpret it as we choose. And to be clear, this isn’t because we choose to be deliberately deceptive either, although that’s probably true sometimes. It’s because being human, we can bring unintended biases to the data.
It’s a huge conundrum in the age of big data. How do you find definitive answers when you can look at different data points on the same topic and come to different interpretations?
DATA SCIENTISTS MATTER
Pam Baker who is author of the book Data Divination: Big Data…
View original post 1,315 more words