First Things First
When assessing sources we want to know whether what we are consulting is primary or secondary in nature (first or second hand). When reading a newspaper this translates roughly into “Was the reporter there or not?” although other factors, such as creative editing can also come into play.
We should be asking:
- Who is writing?
- Were they there?
- Was there more than one source?
- What judgements have been made in presentation?
- Will anything have been omitted – intentionally or otherwise?
- Are we looking at personal experience or second-hand reporting?
- What record does this source have for being accurate?
- If it’s second hand, how does the original source stand up to this same scrutiny?
However, how often do we realistically get the chance to ask these sorts of questions of a newspaper? Could we actually finish any articles if we did?
In fact, the reality is that we tend to base our judgements of sources and information on “gut feeling” about how reliable be know or believe them to be. These beliefs naturally by their nature have the potential to be wildly inaccurate.
Take for example newspapers: most people have 1 or 2 newspapers that they read regularly, and if asked will give negative opinions of a number of other papers accuracy or editorial policy. But, ask the readers of those papers & you will likely hear the same sorts of opinions back at the original groups choice of read. It should go without saying that both groups cannot be 100% correct, or 100% wrong, the reality falls somewhere between.
On the internet the situation worsens. We often do not have the benefit of a reputation to base our judgements and in these cases we often fall back on quite obscure value judgements. For example, I regularly reference BBC News articles because a) I’m British and I pay for it, and b) they tend to be fairly accurate and balanced. On the other hand the Klu Klux Klan may be found to be marginally partisan and as such less trustworthy. They also, conveniently, suck at web design.
On the basis of my personal prejudice (and design) I would estimate an accuracy respectively of 90% / 1%. However, personal prejudice is not a good basis for an opinion (just look at the KKK). If you’re questioning the 1%, think about this: “Even monkeys can write Shakespeare“.
Lies, damn lies….
As accurate as my personal opinions obviously are, it would be probably be beneficial for us to be a little more scientific in real life. Lets talk numbers.
A good basis for any assessment of a source would be our list of key questions given earlier. That is, “Who is writing…” etc. However, in order to make them easier to apply to a particular source let’s reword them a little:
- Was the writer present in person?
- No: Who was this writers source? Is there more than one?
- Are there any inaccuracies or omissions?
- Yes: What are the other sources?
- What record does this source have for accuracy?
With this we have a simple 3 step process of assessing our confidence in a particular source. In fact, by assigning a weight to each answer (and subsections) it is possible to reduce our rating numerically – a single, impartial, confidence factor.
If you’re thinking…
Hang on, if we didn’t have time to double-check references while we were reading, we won’t have time to double-check references & calculate our factors.
…then you’re right.