Love the way you outlie

*** A quick hello from the Saturday Writing Boot Camp! ***

When I am wearing the researcher developer hat and speak to PhD students, I am extra extra extra careful not to promote one particular methodological stance over another. (Note to self – a couple of books on the topic of quant-versus-qual tension in research: Quantity and Quality in Social Research (Bryman, 1988); A Tale of Two Cultures: Qualitative and Quantitative Research in the Social Sciences (Goertz & Mahoney, 2012); and Q-Squared: Combining Qualitative and Quantitative Approaches in Poverty Analysis (Shaffer, 2013).

As a researcher, however, I am aware that I get more excited about qualitative lines of inquiry. And I am continuously reminded that methodological gaps are perhaps bigger than disciplinary gaps, especially when I see how outliers are handled in quantitative work.

Since my summer with them in 2006, I follow, with interest and even a little bit of pride, what the Oxford Internet Institute produces. A research outcome that they released a couple of months back, however, left me unconvinced, if not confused. The map titled Age of Internet Empires illustrates the most visited website in each country, providing an overview of the power dynamics of Internet moguls. What an interesting initiative. However, the following passage left me unconvinced, if not confused.

20131126_internet_empiresThe researchers say:

The supremacy of Google and Facebook over any other site on the Web is clearly apparent. We also see an interesting geographical continuity of these two “empires”. Google is the most visited website in most of Europe, North America, and Oceania. Facebook, in contrast, is the most visited website in most of the Middle East and North Africa, as well as much of the Spanish-speaking Americas.

The situation is more complex in Asia, as local competitors have been able to resist the two large American empires. Baidu is well known as the most used search engine in China, which is currently home to the world’s largest Internet population at over half a billion users. At the same time, we see a puzzling fact that Baidu is also listed as the most visited website in South Korea (ahead of the popular South Korean search engine, Naver). We speculate that the raw data that we are using here are skewed. However, we may also be seeing the Baidu empire in the process of expanding beyond its traditional home territory.

I find the last sentence to be particularly worrisome. My concern is that given the expert position of the OII in the field, such a finding gets cited a lot. It has indeed been cited on Twitter as evidence of Baidu’s influence outside China. As an Internet researcher with a focus on the Asia-Pacific region, I bet my money that Baidu is NOT the most visited website in Korea. It’s not even in the scene. Despite the large cultural common ground, the languages the two countries speak are not even remotely similar.

It was surprising, for lack of a better word, to read that the researchers speculated that the data are skewed but still went ahead to say that Baidu is expanding beyond its home territory. In Asian countries, (the incompatibility of) languages and regulatory frameworks demarcate cyberspace. Hence a lot of domestic services in Korea stay strong, often outdoing Google, Facebook, etc. It is quite tempting to state that Chinese influence goes beyond their territorial boundaries. We do see incidents of that. Look at how other smaller Southeast Asian countries (allegedly) benchmark the Chinese government’s online surveillance techniques, for example. I understand the awe. But this particular finding from the OII should have been triangulated with other sources of information, preferably qualitative investigations, or should have been discarded.

To be fair, this is not the first work where I see Korea standing out like an odd ball. A couple of years ago, I had a chance to help a colleague with making sense of some Korean search results for a journal paper. In that paper, ‘Assessing global diffusion with Web memetics: The spread and evolution of a popular joke‘, Korea was the only country showing a reverse trend, resulting in an endnote by the authors as follows: “We couldn’t find a satisfactory explanation for this result”. I understand that they had to draw a line at some point and that not every finding can be pursued to exhaustion. It’s just that it is quite hard for me to imagine, however, not to get drawn to such an outlier. If it was I who found that only one country in my sample shows a different pattern, the first thing I would have wanted is to investigate further until I have come up with an answer why the heck not – why it doesn’t fit the overall finding.

I don’t think this post is to be read as a criticism. It is more of a note to myself that researchers do operate differently. I am sure there are enough researchers out there who would tell me that one shouldn’t be drawn too much to the fine-brush details of a few outliers and should pay more attention to the broad strokes of things. No wonder one of the common tips for write-up PhD students on the nomination of their examiners is to find someone who is methodologically sympathetic, because researchers really do operate differently.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s