Google Scholar Illiteracy at Skeptical Science

Rebuttal to "Meet the Denominator"

In a desperate attempt to diminish the value of the list of peer-reviewed papers supporting skeptic's arguments, Rob Honeycutt from Skeptical Science not only lies but puts on a surprising display of his Google Scholar Illiteracy. It is clear that not only does he not understand how to properly use Google Scholar, he has no idea of the relevance of any of the results he gets.

1. Rob begins with a strawman argument that the list is only presenting one side of the argument. This has never been denied as one of the reasons the list was created was to show the skeptical side of the argument supported by the peer-reviewed literature. This purpose is explicitly stated on the list,

Purpose: To provide a resource for peer-reviewed papers that support skepticism of AGW or the negative environmental or socio-economic effects of AGW and to prove that these papers exist contrary to widely held beliefs.

2. Rob mentions previous criticisms of the list that have been completely refuted,

Rebuttal to "450 more lies from the climate change Deniers"

3. Rob produces erroneous results using a search query without quotes,

He does an advanced Google Scholar search query for the search words,

climate change

In his blog post he incorrectly lists them as a search phrase using quotes, "climate change". There is no mention that no quotes were used. As you will see this dramatically affects the results.

He then adds two advanced filters,

* Biology, Life Sciences, and Environmental Science
* Physics, Astronomy, and Planetary Science

The results,

climate change (no quotes + filters) - 956,000

When quotes are used you get a dramatically smaller result,

"climate change" (filters) - 635,000

It is clear Rob is only interested in results for the search phrase "climate change" yet by not using quotes he included erroneous results that simply included both words in any context, including having nothing to do with "climate change". Such as,

Managing the Process of Engineering Change Orders: The Case of the Climate Control System in Automobile Development
(Journal of Product Innovation Management, Volume 16, Issue 2, pp. 160–172, March 1999)
- Christian Terwiesch, Christoph H. Loch

Why is Rob counting results about climate control systems in automobiles?

4. Rob arbitrarily pulls out 10% of the results to remove citations failing to understand that these can be automatically filtered using Google Scholar,

What are the results marked [citation] and why can't I click on them? (Google Scholar Help)

"To exclude them [citations] from your search results, select at least summaries from the dropdown menu labeled include citations."

The results without citations,

climate change (no quotes + no citations + filters) - 901,000

"climate change" (no citations + filters) - 600,000

This just further demonstrates his inability to properly use Google Scholar.

5. Rob fails to understand that Google Scholar does not have a peer-reviewed only filter and thus he is counting erroneous results,

What do you include in Google Scholar? (Google Scholar Help)

"Google Scholar includes journal and conference papers, theses and dissertations, academic books, pre-prints, abstracts, technical reports and other scholarly literature from all broad areas of research. ...Shorter articles, such as book reviews, news sections, editorials, announcements and letters, may or may not be included."

These non-peer-reviewed sources are easily proven to exist in Google Scholar search results; such as 21,000 from the Guardian, 86,000 from Newsweek and 144,000 from the New York Times.

Thus the only way for Rob to verify his numerical result total from his Google Scholar search is by checking that every single search result is a peer-reviewed paper from a peer-reviewed journal. He clearly did not do this.

Note: It is important to remember that every single result needs to be checked that it is a peer-reviewed paper and not simply in a peer-reviewed journal. As Google Scholar makes no distinction between non-peer-reviewed content that appears in peer-reviewed journals from peer-reviewed content.

6. It is impossible for Google Scholar to be used to verify more than 1000 results for any search query because it is hard limited to 1000 verifiable results,

Can I see more than 1,000 search results? (Google Scholar Help)

"Sorry, we can only show up to 1,000 results for any particular search query. Try a different query to get more results."

Thus it is impossible to verify Rob's claims as searched, making his conclusions meaningless.

7. Rob gets caught lying about how many papers he "perused",

"I did a pretty thorough perusal of the 200 pages of articles and it looks like they are all actual papers and not just references to any blogs or websites." - Rob Honeycutt, Skeptical Science

This is a lie as it is impossible to go past page 100 (1000 results) for any search query using Google Scholar.

8. A Google Scholar search result that includes the search phrase "climate change" does not mean it explicitly endorses "Anthropogenic Global Warming" theory. Explicit endorsement would require the inclusion of that phrase. How many results Google Scholar shows using this search phrase can easily be determined,

"anthropogenic global warming" (no citations + filters) = 662

However inclusion of this phrase can also mean criticism of the theory. The context of the phrase can only be determined by reading each and every result. Implicit endorsement would require reading each and every result for alternate search queries.

Conclusion:

No meaningful conclusion can be drawn regarding the number of peer-reviewed papers supporting AGW theory using numerical result totals from Google Scholar searchs due to the inclusion of erroneous results. Thus no meaningful comparison of these results can be made to the list of 850 Peer-Reviewed Papers Supporting Skepticism of "Man-Made" Global Warming (AGW) Alarm.

Note: All numerical result totals from Google Scholar searches can change at anytime.

Update: After repeatedly pressed on how he "perused" 200 pages Rob eventually decided to change it to say, "I did a pretty thorough perusal of 200 articles of the 100 pages of results and it looks like they are all actual papers and not just references to any blogs or websites.". Unfortunately I do not believe this was an honest mistake as he did not immediately admit to the error instead posting six other comments avoiding it. This is not something anyone else would do if they made a legitimate typo. I believe he originally just made up the number so it appears he made a token effort at validation. The word "pages" and "papers" is not easily confused, let alone "pages" and "articles".

Source