There are times when it is better to remain quiet and be thought a fool, than open your mouth and remove all doubt. (Attributed to Abraham Lincoln, but I can’t find the source)
This week I am going to sort of take this to heart. There are two great posts, one at Lynn’s site, and one at Carl’s site, that discuss data visualization very well. Instead of highlighting my foolishness on this site, I will remain quiet and provide my observations as comments on their sites.
I will comment on Professor Cohen’s article about trying to make sense of digital research with data mining techniques. We are at the forefront of data mining in the digital age. In the past historians were limited by time and distance into what they could review and try to correlate. Looking at the court records of 19th century Britain is possible – correlating the data between jurisdictions and developing time analysis is daunting. Taking this court data and correlating it with social data, such as parish records or economic data such as tax receipts is impossible, except for isolated cases. If all this data was digitized and correctly tagged historians could write queries that asked for correlations between data sets for whole sections or all of Britain. Trends of the whole country could be reviewed. Even if the data wasn’t tagged, if an API existed like the Google API for filtering queries within parameters, or H-bot could be used, the data could yield new correlations which historians have not yet theorized or investigated. We are lucky to be at the forefront of this trend, but only if we take advantage of what is there and get involved in setting the direction for future historians.
Getting to the problem of Abundance or Scarcity it seems as if historians will have some great tools on data born digital. For instance the British Court System now digitizes all records and even has a link that allows researchers to sign up for, and use these digital records. Not only are some records born digital – they are being prepared for researchres of the future.
OOPS! I guess I removed all doubt about my being a fool. 
What is so interesting to me about this post, first of all, is what it teaches me about the learning process in a class such as ours, where everyone has different levels of knowledge in different areas. I read your first line and laughed – because your posts are anything BUT foolish. I always learn a lot from them and when I don’t understand them, I feel foolish myself. So I guess what I’m realizing is that we need to share what we do know and be willing to admit what we don’t and take advantage of others’ knowledge. And yours has been very useful to me.
Also, as you said about data-mining, the very idea of the power of this online tool should make any historian thrilled with the possibilities. I expect you and Carl to start working on this immediately.
Comment by lprice3 — November 14, 2009 @ 6:19 pm |
The tool is easy. Getting the data digitized, categorized and maintaining it is the hard part. The algorithms for data mining are pretty straightforward. I’m waiting on Carl to finish his project that will allow everyone to easily digitize their collections.
It’s Carl’s fault we aren’t done yet.
Comment by theoldscholar — November 14, 2009 @ 6:33 pm |