We all rate user-generated content on the web all the time: videos, blog posts, pictures. These ratings usually serve to identify content that we might like. In online innovation communities such as Dell’s IdeaStorm or My Starbucks Idea those ratings serve a different purpose: identify the “best” idea to be implemented by the host organization (there is a whole other argument that it is only about marketing and not about the “ideas” but let’s not go down that road). So the question arises: how can we best design collective intelligence mechanisms for idea selection in innovation communities?
In a paper published at the last ICIS conference, a group of researchers from TUM (Germany) compared three different rating mechanisms in a field experiment (n=313) against a base-line expert rating.