Week 3 of the MozNewsLab had the tech evangelists step aside and the grizzled veteran news producers come to the fore. Shazna Nessa, Mohamed Nanabhay, and Oliver Reichenstein all gave us a look inside their respective histories in hauling journalism into the digital era, whether it liked it or not. While all of them had a lot of insight on working within the newsroom (I have to say, Mohamed in particular hit one out of the park) I’m not sure how much relevance these talks have to my project in the lab.
Re:Poste is intended to be a third-party, guerrilla, interventional system. If I have to work within a newsroom to get it running then I’ve clearly done something wrong. This is not to say that collaboration wouldn’t make it easier or more efficient to build the system or that I couldn’t learn a thing or two by hanging around the people writing the articles, as I clearly could. However, since Re:Poste is intended to exist independent of news organizations building it within the walls of any of them would be somewhere between a conflict of interest and a fundamental conceptual flaw. My interest is in a community that discusses, moderates, and improves upon news, not a community that creates it.
I don’t appear to be alone in this interest, either. The MozNewsLab participants have (almost) all put up double-tweet length descriptions of the projects they’re developing as part of the lab. I’ve put together a collection of ideas from that list that have similar goals to Re:Poste below, along with some commentary. (And also completely blown the max-500-words rule that MozNewsLab imposed on these posts in the process, though in my defense I put them in a neat little tab-thing and many of them are just quoted from other people. Oops. Hey, you wanted hackers, did you really expect conformity?) As my friend and frequent collaborator Jon Ippolito often says, “cheating is the pedagogy of the Internet”. Let’s get started.
??? (Not yet unveiled)
Make commenting on and discussing news articles (actually all web pages) a feature of web browsers. – Unify the commenting experience across the web – Promote openness. Users control their own data – Route around censorship – Encourage diverse thinking
There’s not much public on this one yet so I can’t say a whole lot, but it sounds like plugin-mode Re:Poste. I’m interested in what is meant by “Users control their own data” and “Route around censorship” though, so I’m eager to hear more.
It all began with that sticky idea: Geek comedian Tom Scott couldn’t stand “dodgy” journalism anymore. So he created some “warning labels” to put them on free papers he found on the London Tube.
It’s clear: If you can put stickers on newspapers, it should be possible online more than ever. Studying online journalism, we made this the subject of our diploma thesis. For three months, we analysed quality (control) in German and American journalism. We learned a lot about fact checking, accuracy and (failed) attempts to involve the public in media accountability.
Now we put all these learnings together and conceived a proper service: corrigo
It helps you flag and correct factual errors in online media – directly at the article.
In motivation and conception, Corrigo seems to be a perfect mirror of Re:Poste. Corrigo was created as a thesis project by Tobias Reitz and Kersten Alexander Riechers so there is a lot of information on the idea, but unfortunately I don’t know German so I can’t read the paper and am stuck with interpreting the pretty pictures. There are a few things in this interface sketch that are worth thinking about (or stealing outright):
- Switch timeline: I’m kind of intrigued by this as I’ve never thought of applying Re:Poste to an entire site at once. I’m not quite sure how it would work, at least if I actually included a granular timeline of comments across all articles. Maybe it becomes an entry point with article-level links and and an overview of each articles ratings, kind of like the Re:Poste home page was in my original version but embedded as part of the app chrome to allow article-to-article navigation and allow browsability within the Re:Poste interface.
- Categorizing errors: I think this could be important. I might not go as far as allowing end-users to correct typos (what would be their interest in doing so? how much value does it add to the comment thread?) but classifying an objection beyond just “this part is bad” would be useful. The problem is in finding a balance between an informative detail and a list of categories that takes more time and energy to manage than it provides benefit. “Factual error” and “missing source” seem like good options, but what else would there be? If those are the only two options, it seems like it would just be one more button to push in the interface.
- Verified account: My initial version of Re:Poste buried personally identifiable information about a poster on a second page that users had to click through to get to. This was intentional–Re:Poste is not intended to be about the people, just the information, and making it easy to identify a commenter just makes it more likely that the people will be discussed instead of the article. But a verified account would have value in the system because it adds credibility to have a real name behind the post. This seems like an excellent idea, but one that I would not build into Re:Poste’s primary interface. I’ll leave that as only exposing information about the poster’s calculated credibility, not their ID.
- Badges: Badges are hot right now, with good reason. I think this falls into the same category as verified accounts in that the data coming from a badge (“best commenter award, July 2011!”) might be worked into credibility calculations but it wouldn’t go on Re:Poste’s main interface.
- Sharing: Re:Poste’s original genesis was before the age when every page had a share button, so it was never put into the interface. It should be there, though it should be on an article level basis to link to the article+comments, not at the comment level. Another button for the chrome, I guess.
The problem – Online comment threads can be a haven for discussion, providing the journalist a link with their audience, or they can be troll filled arenas of a abuse, offering little to the user or author. What’s more, on large sites, it can be difficult to follow the many threaded discussions, and filter the signal from the noise.
The zeitgeist – There are many sites just now which allow comments which are perceived to be trolling to be downvoted, or moderated by staff. This is useful to an extent, but on sites with a political lean, for instance, voices from the opposition point of view can be drowned out.
The discussion – We began by thinking about the roll that trolls play on discussion threads. In one sense they exist simply to annoy the other posters on the thread. But we began considering how this division could be leveraged to encourage discussion, similar to the ancient greek philosophical concept of sophism, the teaching of rhetoric. If the contrary nature of online commenting could be harnessed, the ensuing discursive nature of the comments moves the article to a encompassing view of the news being discussed.
The idea – Comments could be rated not simply as an ‘upvote’ or ‘downvote’ but on a multivariate basis: support/retort, useful/useless etc. Users would be asked to consider not only how good the post is, but how it directs the discussion.
The opportunities – multivariate rating allows new forms of data visualization, giving an idea of the direction and flow of the discussion. It also allows the author or third party, to curate the discussion.
Trolls. Personally, I’d be happy to just stamp them out completely by hiding posts below a certain credibility level. But it’s interesting to think of how they can be used to improve the overall quality of comments. They do occupy a definite place in the online ecosystem, though not a great one–the final stage of life for most online forums is to have nothing but a core of old users trolling and counter-trolling each other based on years of shared experience.
Thinking about how trolls generally work may be useful, though. In general, there is not a lot of originality in their posts. They generally get their talking points from other sources and just throw them up against each other, like gamblers at a cockfight. What if, instead of tagging by word, comments were tagged by argument? Those arguments could then be filtered, hiding anybody who writes “Obama is a sekret Muslim.” Trolls only have so many arguments and would soon run out.
But this introduces a point of vulnerability as well. If anybody can come along and tag any post “Obama is a sekret Muslim” then it would be very easy to get legitimate arguments filtered. That means there would have to be a metamod system on the argument tags, which would feed back into credibility calculations. I’ll have to think about it, but it seems like this might be a case of too much overhead for users to think about.
The actual idea of having multiple categories of rating is useful though, and ties into the idea of categorizing errors from Corrigo. If used responsibly it can add a lot to the system. I still wonder how many users are going to use it, responsibly or not, since it requires additional analysis. On the other hand, deeper thinking is what Re:Poste is after.
The News Tree
Powered by user-submitted URLs and a data scraper which follows blog trackbacks, hashtags and other digital trails, this visualisation tool uses a tree metaphor to map not just the coverage of a news item but the development of user discussion around it. The scraper also gathers data from up and down-voting systems on various websites to rate and display the perceived value of each comment.
For example; the earliest articles and mentions build up the ‘roots’ of the tree, the most active and saturated conversation hubs flourish into ‘thickets’, the thickest branches are the established news outlets and the thinnest are freshly started blogs, each leaf represents a single comment which will fade over time unless referenced or replied to and leaves representing comments which are downrated by many other readers will appear diseased.
However, more than simply a pretty visualisation, the user can take advantage of these cues to quickly acquaint themselves with the fundamentals of the story by locating and reading the original reportage, the most helpful or controversial comments and the most active conversations, which they are then informed enough to take part in. They can also curate their own experience, filtering out (or indeed seeking out) ‘trolling’ or other undesirable conversation branches.
I had a version of this built into the previous iteration of Re:Poste where it merged articles with identical content to create true cross-site comment threads for syndicated stories. That was a much simpler system than The News Tree proposes, only aggregating content on target sites instead of reactions across different platforms. I very much like this idea as a visualization on top of Re:Poste’s comments and metrics though. It adds something I’ve regretted not putting in for quite a while, the ability to track an event independent of individual stories. If it were applied to track multiple articles instead of just reactions then it could provide an ongoing timeline of the development of the story. That’s something that would be tremendously useful for getting a high-level view of events as they occur. If it were applied to just a single article with Re:Poste comments as the leaves it would also be a cool viz tool or alternate interface for the data.
Roundtable is a platform for engaging readers, journalists and experts around salient news topics. Inspired by Oxford-style debate, the app allows newsrooms to crowd-source news analysis and engage readers by inviting them to table to participate in the debate.
I have to say, I absolutely love the idea of rigorous debate on news articles. I also have to say that I have absolutely no faith in a general user community to actually engage in a rigorous debate on news articles. If this is to be applied to a gated, verified, externally-credible community and then overlaid atop an article as an education tool then it would be great, provided one could come up with such a community. I don’t think that’s Katie’s goal with this project, but it would be mine.
Re:Poste has always been designed as an asynchronous application. But what if there was also a synchronous component? Schedule online discussions between high-level participants that could be run as a traditional debate, but with input from the larger community. Then, after the debate, it could be attached to Re:Poste comment streams on articles about the same event or subject. This would be an orthogonal discussion to the comment stream but would inform it and, if attached to multiple stories, could really add value to the comments as a more involved debate about the issues at hand. Tie it into the News Tree variation and it becomes a persistent means of informing an ongoing discussion at a very high level.
An open-source, distributed platform to enable sentence-level, community moderated annotation of news, blogs, legislation, scientific articles, PDFs, books, video, etc. without the consent of the target. Crowdsourced peer-review for the Web.
The simple expression of MVP I’m working on is thus:
- Pointing into text and addressing it (ala Awesome Highlighter)
- Attaching an annotation to that location
- Associating a formalized stance or sentiment to that annotation (to allow aggregated measurements of those stances)
- Moderating the annotations via a good meta-moderation model (think slashdot, but with a skew towards domains moderators are known to be knowledgeable about)
We will employ several features we believe are critical to a successful outcome: • Inline annotation—Specificity is key. Especially in a lengthy article, locating critique at a sentence or paragraph level is important to centering the dialog around that passage. • Indication of stance—Does the critique support or challenge the associated text, what is the specific relationship of the annotation to its target? • Powerful references—Properly enabling the use of references and citations is key, allowing their reputation to be inherited is an important aspect. • Collaboration—Users will be able to suggest improvements and modifications to others’ contributions.
Most important perhaps is the way reputation is handled. Providing the proper checks and incentives which will encourage quality contribution and discourage trolls and the uninformed is all important. Hypothes.is will employ randomized moderation and meta-moderation that favors moderators with similar or adjacent domain expertise, as inferred from their previous contributions.
We imagine an objective metric for articles that is a composite of the accumulated critique they receive (itself dependent on the reputation and domain expertise of those providing the critique) together with several automated measurements, such as the number of other credible articles that cite them (citation rank), their social rank, the degree to which they motivate the following of the author and the number of facts and details that they include.
Again, this sounds very similar to Re:Poste, and indeed some of the predecessors they cite are the same ones I looked at in developing Re:Poste years ago. Though there are a lot of goals floating around for Hypothes.is and claims that it will be different than all the precursors that failed, I haven’t seen much about how those goals will be accomplished yet so there’s not much I can say about implementation specifics at this point. A couple of interesting features:
- Powerful references: I have no referencing system in Re:Poste. This was an intentional decision because my goal was to make commenting as quick and painless as possible, and adding references tends to be neither. My thinking has changed somewhat though, based largely on Wikipedia. That is a model that absolutely requires sources…but only after the fact. I think Re:Poste could work the same way by offering the chance to put in a reference but not requiring it. Of course, that also means there has to be a  button, but I’m thinking that is relatively simple. Maybe if there was a reference organization system that was global to the thread, much like MS Word’s reference panel, and contributors could just pull out the applicable links…hmm.
- Collaboration: Another interesting idea that I’m unsure about from a UX perspective. It sounds like it would turn into wikiComment, which is nice in theory but I would be very hesitant to touch. If Hypothes.is has the right interface it might work, but I haven’t seen an interface put forward for it yet.
If you read through all of these ideas there is the potential for a very compelling synthesis application (or convergence of apps via APIs). I’ll try to put the pieces together for my final proposal next week. This is why open source development works.