Requiem for a Website

Last year I had some harsh words for Jim Zwick, because it appeared that he had served a DMCA notice on one of my colleagues after she posted images of mutilated natives from the Congo that were shot by activists in Africa over a century ago in conjunction with the obviously educational purpose of teaching the novel Heart of Darkness. I thought that it was an incredibly abusive use of intellectual property law, given the human rights discourses that should be part of our culture's collective memory.

The whole story has turned out to be considerably more complicated and even sadder than I initially thought. My colleague chose to move her faculty website off the university server, but the consequences for Zwick from his series of IP battles have proved to be even more disastrous. I don't like websites with advertising, but there was no question that Zwick's boondocksnet was the best site about U.S. and European imperialism around, with incredible pedagogical riches in the form of a huge catalogue of images and primary source documents. Zwick, a veteran of trail-blazing projects at Georgetown and George Mason University, seemed to have a real sense for what would be most useful to make history come alive to media-saturated students who were sometimes too apathetic to grapple with multiple counternarratives otherwise. I first heard about this website from historian Robert Moeller, Director of the UCI California Social Science/History Teachers Project, when he directed students to it in lectures for the Humanities Core Course.

Unfortunately, boondocksnet, which once hosted over 20,000 pages, has since gone dark. This happened largely because of duplicate-content rules for search engines. So many state curricula thought Zwick's site was so great that they copied pages and pages of it and hosted the content on their own often government-sponsored servers. Because giant companies like Google want to simplify search results, they won't carry duplicate content if the same pages appear on more than one server. And to make sure that users reach the most authoritative sites, the algorithm of these search engines seems to default to institutional URLs with .edu or .gov extentions rather than independent producers of online information.

According to Zwick, his site on U.S. imperialism had a similar page-ranking to comparable materials at the Library of Congress back in 2003. But after technologies became available to duplicate the content of Internet behemoths like Amazon.com and Wikipedia, mechanisms had to be created to preserve their page ranking and protect their status, while removing thousands of pages of duplicate content that were appearing in search results at Google and other search engines.

Thus, because of the procedural logic dictating search protocols, what Claus Schmidt calls "page hijacking" was punished when its victims were sites that were already authoritative hubs. But if the web domain came from an independent extra-mural source, bigger "textual poachers," to use Henry Jenkins' term, such as the Internet Archive, could end up hurting the actual content-creator, who might be blocked from a given set of search results. Regardless of size, even the most obviously educational sites with a lot of author-generated content could get caught up in the no-duplication net. As Zwick says, "I think my site, the Smithsonian, and a bunch of others were banned from Google in early 2005 after a commercial script for use by high schools was released that uses 302 redirects to link to all kinds of academic sites. Google seems to have made manual corrections for well-known institutional sites but did not address the problem more generally so many independent sites continue to be penalized because of 302 links and copying by others." By early 2007, use of Zwick's site declined to less than 23% of what it had been in early 2003, and newer materials added to the site were not getting any use at all.

Ironically, this problem of "domain poisoning" was made worse the more popular Zwick's site became among authoritative content-providers. As he writes, "The first using 302 redirects did it as a bait-and-switch but by 2005 or so, many legitimate sites were using the same technology to count usage of links going to the legitimate sites." Says Zwick, "Google was the most influential problem with the 'duplicate content' issue but my site was also banned from Microsoft Live and Ask.com because of it. Yahoo was the only major search engine that listed the site, probably because it was also the only search engine not affected by the page hijacking problem."

For Zwick it was particularly disastrous, because -- like many independent scholars -- he also used his website to showcase his publications. So, an entire corpus of work on American political discourse and authors such as Mark Twain wasn't linked to his individual domain, although that was his intention in putting his publications there in the first place.

I can sympathize, because I know that my faculty page is the way that people contact me to commission articles or invite me to speak . . . or reach me with complaints about this blog. Having a more impervious domain name extension associated with a large educational institution protects me from the duplicate content problem. As more and more faculty members become content-creators and eschew using university hosting, for a variety of reasons including the inherently nomadic character of academic life and the desire to establish what Julia Lupton has called a "personal brand" independent of their more constraining institutional identities, these faculty members may be more likely to find themselves in Zwick's position. In other words, right now I may be amused by those using the relatively modest ranking associated with the "virtualpolitik" name to sell political consulting services in Mexico or web design products in Germany, but if my scholarship were more accessible to the general public, particularly to those in required high school courses, I might feel very differently about this issue.