Decision: Blogs vs. New York Times

Posted on Friday, February 1st, 02008 by Alexander Rose - Twitter: @zander
link Categories: Long Bets   chat 0 Comments

 

Long Bets has arrived at a decision for Long Bet #2 between blogger Dave Winer and Martin Nisenholtz of the NY Times. At stake is US$2000.00 plus half the interest that has accrued over the last 5 years in the Farsight Fund, all of which will go to the charity of the winner’s choice.

In the bet Winer asserts, “In a Google search of five keywords or phrases representing the top five news stories of 02007, weblogs will rank higher than the ‘New York Times‘ Web site”. The premise of this bet is excellent, but unfortunately the arguments were quite vague on how to adjudicate the bet. Long Bets encourages bettors to construct arguments that involve the least amount of interpretation possible. Once this bet came up for adjudication we urged both parties to come to their own decision, but they asked Long Bets to be the final arbiter. We have done our best with the information and resources available to us, but this process should be a good instructor both to future bettors and ourselves…

The major questions that affect the interpretation of this bet:
Q: Which list of “2007 top stories” to use?
A: We chose the Associated Press list, as it was the only one suggested by one of the bettors (Nisenholtz), and it was in effect at the time of the bet origination. We found many others, (some listed in the notes below), that may actually be better indices of what a “top story” is, but we felt that the AP list was our best choice for this bet.

Q: What is a weblog? Does Wikipedia count? What about the NY Times blog or other commercial blogs? Does it include any non-commercial user submitted web site?
A: We decided that a weblog had to be something that would have been recognized as a blog in 02002. This includes ad supported blogs and commercial blogs like those of the NY Times. While the bettors argument in this case discusses why non-commercial content will beat out commercial content, Winer never provides a definition of a weblog. As it turns out, including major news source blogs like those of the NY Times or sources like Wikipedia do not affect the ultimate outcome in the case of this bet, but they certainly could have.

Q: What is the NY Times? Does the International Herald Tribune count (which is owned by the NY Times and its content comes from there)?
A: We determined that it had to be on the nytimes.com web site to count. If the bettor wanted subsidiaries or other associated derivative content to count, they should have specified it in their argument. This did affect the outcome of one of the searches where the IHT.com result came in at 9 and blogs came in at 10. This result would not have affected the ultimate decision however.

Some other notes: The bettors also never defined what the search semantics should be, and or what date the searches should occur on. Both of which affect the data a fair amount. We tried the searches in a number of ways and a number of times since AP released their list of stories in December to arrive at our decision. We disregarded any search results that were dated after 12/31/02007 when calculating search rank.

Here are 02007′s top stories, as voted by AP Journalists with search rankings (lower is better). We also include results of the highest non-commercial/user submitted content and highest ranked commercial content as a reference.

“VIRGINIA TECH KILLINGS” (NYT score 26, blog 10) winner Blogs
Highest user contributed result: Wikipedia 1
Highest commercial news outlet result: USA Today 2

“MORTGAGE CRISIS” (NYT score 2, blog 10) winner NYT
Highest user contributed result: Wikipedia 1
Highest commercial news outlet result: NYT 2

“IRAQ WAR” (NYT score 24, blog 5,) winner Blogs
Highest user contributed result: Wikipedia 1
Highest commercial news outlet result: CNN 3

“OIL PRICES” (NYT score 172, blog 38) winner Blogs
Highest user contributed result: Monga Bay Blog 38
Highest commercial news outlet result: Bloomberg 1

“CHINESE EXPORTS” (NYT score 57, blog 3) winner Blogs
Highest user contributed result: Blogging Stocks 3
Highest commercial news outlet result: China Today 1

  • Adding up page rank winners blogs win 4 to 1.
  • Adding up page rank winners of user submitted content vs. commercial content, user submitted content wins 3-2.
  • If you average page ranks of the NYT (avg rank 56.2) vs. blogs (avg. rank 13.2) Blogs win.
  • If you use an average rank of user submitted content (avg. rank 8.8) vs. commercial (avg. rank 1.8) Commercial news outlets win.

The Long Bets decision on this bet is in favor of Winer’s side, weblog page ranks came out ahead of the NY Times. We will be calculating interest and sending a check on to Dave Winer’s charity of choice the World Wide Web Consortium in the next month.

Notes:
Aside from the observation that Wikipedia often ranks very high and was not really considered at the time of this bet in 02002, another interesting note was how well government sites ranked in subjects like oil prices, Chinese exports, and others. The government sites are often listed in the top ten of these types of subjects showing that people are also turning to the government websites for authority.

The other interesting thing to us was how much the bettors own definitions (or lack there of in this case) affected the bet. For instance had the bet been structured around commercial vs non-commercial content, and they had chosen an average ranking system (which actually seems to answer the question being asked more clearly), commercial content would have won by a factor of more than four.

Also of note is that with a slightly different analysis Rogers Cadenhead did come up with the same winning results based on page rank over at his blog Work Bench.

For reference here are some other “Top Stories of 2007″ lists that could have been considered. Testing the first two of these lists yielded results similar to the AP list.

Pew’s Project for Excellence in Journalism’s News Coverage and Interest Indexes.

Time Magazine

About.com

Foreign Policy, top 10 stories missed in 2007

CNN (not ranked – chronological)

MSNBC graph showing top story of the day, for the year (most clicked)

Telegraph UK Top read stories of 2007, by category

Crikey’s Top Ten List

Doctors Without Borders (top *underreported* humanitarian stories):

BBC News (most popular)

Yahoo! News (most emailed)

  • Pingback: Cheryl’s Mewsings » Blog Archive » Blogs 1 - NYT 0

  • Pingback: Scripting News for 2/1/2008 « Scripting News Annex

  • Pingback: links for 2008-02-02 at The New Reader

  • Pingback: NY Times sázku s blogy prohrály : Online žurnalistika

  • B Ruby Rich

    Interesting, but deeply flawed. When I publish articles in The Guardian (London) or the Bay Guardian (San Francisco), they rarely show up on my “Google alert,” but whenever I am mentioned however passingly on a blog, then I immediately get an alert. Your contest did not measure newsworthiness at all. It measured Google’s arcane and deeply bizarre search engines that favor blogs over real news sources. Please report further!
    –B. Ruby Rich
    Professor, U. of California, Santa Cruz

  • http://www.lot49.com Thomas Claburn

    The outcome of this bet is highly dependent on the search engine used.

    Google’s PageRank algorithm puts more weight on the “wisdom of the crowd” than, say, Yahoo Search. Thus, specifying a Google Search gives Winer an edge. Try “Iraq War” with Yahoo or Ask, and you get mainstream media sites before anything else. Ask, for certain terms like “Virginia Tech Killings”, even includes a “OneBox” result that calls out mainstream press links before its sponsored and organic search results list. Live Search, interestingly, like Google, puts Wikipedia first.

  • Pingback: Blogs win over mainstream media | AccMan

  • Pingback: New York Times vs. blogs: wrong question - - mathewingram.com/media

  • http://futuryst.com Stuart Candy

    Professor Rich,

    You correctly note that Google search was a key criterion in settling bet #2. It was incorporated by the participants themselves in 02002.

    Long Bets is a forum in which bettors are free to decide on their terms, subject to rules including:
    » The minimum period of Predictions and Bets is 2 years.
    » The subject of the Prediction or Bet must be societally or scientifically important.
    » Predictors and Bettors must provide an argument explaining why the subject of their prediction is important and why they think they will be proved right.

    We at Long Bets encourage participants to bet on the clearest terms possible, to enable a self-evident outcome (or failing that, a fair decision) when the time comes. One of the things we’re learning through this experiment in Accountable Predictions is that drafting predictions and bets that endure is no mean feat.

    You suggest, as does Mr Claburn, that any search method contains its biases. However, the search engine to be used was among the few unambiguous elements for us as adjudicators, and was not of our choosing. As a measure of “newsworthiness” it is indeed idiosyncratic. As a spur to deeper debate about what constitutes a significant news source, and how to track that after years have elapsed, however, it seems to be working quite well.

    Another bet, similarly themed, could address these issues in a different way (not just by varying the search criteria but, for instance, by being more internationally oriented). There are many possible variants, and bet #2 was just one of them. In any case we would certainly like to see people take up the challenge of betting anew as to the shape that the media landscape might assume years from now, incorporating whatever lessons there are to glean from this early example.

    Stuart Candy
    Long Bets Research Fellow, The Long Now Foundation

  • Pingback: …My heart’s in Accra » links for 2008-02-05

  • Pingback: Official: blogs top newspapers in Google | News in brief

  • Pingback: In a Google search of five keywords or phrases representing the top five news stories of 2007, weblogs will rank higher than the New York Times’ Web site. | Midas Oracle .ORG

  • http://www.randomhacks.net/ Eric

    I’m actually intrigued by how the original bet was “overtaken by events”. If the bet were more specific, it would have been easier to adjudicate. But at the same time, a more specific bet might have hidden many of the interesting issues:

    – The arrival of a “third competitor”, Wikipedia, which was never included in the original bet.
    – The continued robustness of professional, commercial content in search results.
    – The variation between different search engines.

    Almost every single aspect of the original bet has been subtly affected by events. And this, in itself, is one of the most important lessons we can take away from the Long Bet project.

  • Pingback: links for 2008-02-05 : James Mitchell

  • Pingback: Blogs vs. New York Times « Notas ao café…

  • Pingback: Calling blogs “blogs” doesn’t demean the NY Times | Out of the Box

  • Pingback: My research proposal

  • http://www.longnow.org Alexander Rose

    I just tried to run this test on 02008 with the same criteria. It is basically not possible now that Google groups News and Blog posts up into categories in web searches. It does

    Top Stories of 02008 as voted by AP members:
    http://news.yahoo.com/s/ap/20081222/ap_on_re_us/ye_top10_stories

    But if you look at the News Vs Blog groupings this is what you get, with a clear winner in the News category:
    _1. U.S. ELECTION: News: 1, Blogs 11
    winner News

    _2. ECONOMIC MELTDOWN: News 0, Blogs 0
    tie

    _3. OIL PRICES: News 1, Blogs 11
    winner News

    _4. IRAQ: Blogs 11, News 0
    winner Blogs

    _5. BEIJING OLYMPICS: News 4, Blogs 0
    winner News

    _6. CHINESE EARTHQUAKE: News 5, Blogs 0
    winner News

    _7. SARAH PALIN: Blogs 10, News 0
    winner Blogs

    _8. MUMBAI TERRORISM: News: 1, Blogs 13
    winner News

    _9. HILLARY CLINTON: News: 0, Blogs 10
    winner Blogs

    _10. RUSSIA-GEORGIA WAR: News 0, Blogs 0
    tie

  • Pingback: blogs, longbets.org, and education of sociology | keyongtech

  • Brian Goldman

    This article surprised me that people would really bet that much money on such a dumb little bet. Whatever they want to d though, i guess. I wasn’t shocked when i saw that the search of gas prices can up with the most combined articles. This is one of the biggest issues facing America over the past few years. Of course, the Iraq war and the Virginia Tech killings would have large search results also. I was unaware of Chinese exports being a huge issue today, but i guess i was wrong, although it was not one of the top searches.

  • http://www.emedicalwebsitedesign.com Medical Site

    Can anyone atest to the service?

  • O Bloody Hell

    Another option for deciding what the top stories for the time frame in question would be is the algorithm used by the excellent Newsmap to rank its top stories of the moment — I would assume this algorithm could be applied over time, too, or varied to do so.

    Newsmap: http://newsmap.jp/#

  • Pingback: Blog | Arbor Web Solutions » News Wikis, Ann Arbor style

  • Pingback: News Wikis, Ann Arbor style

  • http://top-gear-streaming.blogspot.com TopGear

    I didn’t get this :\ can somebody pls explain?

  • http://twitter.com/northbetsbr Northbet SBR

    I guess
    I have selected a mind blowing and interesting blog

  • AQUILINO

    EXCELENTE ARITICULO


navigateleft Previous Article

Next Article navigateright