{"id":356,"date":"2014-05-06T21:07:18","date_gmt":"2014-05-07T04:07:18","guid":{"rendered":"http:\/\/xyzzyawards.org\/?p=356"},"modified":"2014-05-07T08:03:01","modified_gmt":"2014-05-07T15:03:01","slug":"how-the-reviews-need-to-change","status":"publish","type":"post","link":"https:\/\/xyzzyawards.org\/?p=356","title":{"rendered":"How We Need to Fix Things"},"content":{"rendered":"<p>OK. I&#8217;ve had a run to burn off my fighting mood and a walk to get my brain working, and here&#8217;s where things are going.<\/p>\n<p>When I started the Pseudo-Official XYZZY Reviews, a couple of years ago, it was strictly a side-project. It ended up on the Awards\u00a0site, but it was off in a little corner and didn&#8217;t <em>feel<\/em> as if it was part of the main thing.\u00a0It was my first year helping out with the Awards, and I didn&#8217;t want to rock the boat too hard. The Reviews were to a large degree an attempt to encourage the kind of IF writing that\u00a0<em>I\u00a0<\/em>wanted to see more of; as far as recruitment goes, I just made public posts asking if people wanted to participate. I got a lot of responses and didn&#8217;t turn anyone away, even if their work didn&#8217;t fully meet the expectations I had for the project, because those were just\u00a0<em>my <\/em>expectations, and who am I to make demands, right? While the results were decidedly mixed, nobody cared all that much, because it was\u00a0<em>all<\/em> bonus.<\/p>\n<p>The next year, things got changed around: instead of putting out a general call, I headhunted known-quality reviewers, preferably with category-specific expertise or interests. This worked\u00a0<em>way<\/em> better, both in terms of the raw number of responses and in how consistently the reviewers were able to match the expectations I had for the event &#8211; which were still articulated as an informal list of rough principles. In a last-minute decision based on the realisation that doing it the old way was\u00a0<em>stupid<\/em>, the reviews got adapted to blog format, moved to the front page of the site and rolled out slowly, rather than getting dumped <em>en masse\u00a0<\/em>in one gigantic plain-HTML page.<\/p>\n<p>This year&#8230; was mostly treated like last year again, apart from doing a bit more to promote it. That was an error, partly because &#8216;business as usual&#8217; often tends to mean &#8216;let things slide&#8217;, and partly because the thing&#8217;s nature was changing even if I wasn&#8217;t.<\/p>\n<p>The response to the <a href=\"http:\/\/xyzzyawards.org\/?p=350\">last post <\/a>has made it clear that a bunch of people who are not me have taken on some of those expectations. Which is great news (please! go and write similar things in other venues! I want your essay on representations of power in early Cadre!), but it means that now I have to take the job of fulfilling them more seriously. So, it&#8217;s probably time to drop the Pseudo part, acknowledge that this has become an established component of the Awards, and start treating it as such. What does that entail?<\/p>\n<ul>\n<li><strong>In the short term, I&#8217;ll be bringing on another writer or two to take a second pass at Individual Puzzles.\u00a0<\/strong>Individual Puzzle is an inherently tough category to tackle. In an ideal world, 2+ reviewers per category serve as the safety-valve for when a reviewer reacts strongly against a work. In an imperfect one, where we will often be unable to ensure that level of coverage, we obviously need other mechanisms. This is a stopgap, not a long-term solution. (And it will, for reasons that I hope are clear, take a little while.)<\/li>\n<li><strong>We&#8217;ll be drafting a document laying out in detail the expectations for what the reviews should be<\/strong>, so that both the people writing and reading them are given a clear picture. We have some informal lists of goals and mission-statement-ish things right now, but they&#8217;re not really something you can <em>work<\/em> to. The final result will not satisfy everybody, but it will at least give us all a concrete point of reference.<\/li>\n<li><strong>The event will probably move to later in the year.<\/strong> Initially, the idea was to have the reviews form part of the same event, more or less, as the ceremony. That&#8217;s more <i>fun, <\/i>but it inevitably\u00a0loads all the socially-hardest work of running the Awards into the same time-period, so it&#8217;s not really compatible with giving the Reviews the degree of attention they require.<\/li>\n<li><strong>I&#8217;ll be looking into a system of sub-editors, or peer review among writers.\u00a0<\/strong>More eyes on a thing makes for better results. I don&#8217;t want my volunteers working alone in the dark, and alone I am not able to give them all the support that they should have.<\/li>\n<li><strong>A name change.<\/strong> The Pseudo bit is a caveat, basically, and if that doesn&#8217;t fly any more then it&#8217;s just deadweight. It&#8217;s been pointed out, too, that &#8216;reviews&#8217; may suggest the wrong emphasis.<\/li>\n<\/ul>\n<p>Let me be absolutely clear. I&#8217;m not blaming Lucian for this: he worked to the standards he was given, ran into some honest bad luck with his games, and ran into a deadline. Any failure here is mine, and I apologise for it.<\/p>\n<p>Finally: my volunteers produce giant piles of quality work. They&#8217;re talented people with other commitments, who have agreed to do a challenging job with rewards that barely count as <a href=\"https:\/\/twitter.com\/forexposure_txt\">for exposure<\/a>. If I&#8217;ve asked them to participate, that means I value their labour and trust their ability for the task. Please keep this in mind in your responses.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>OK. I&#8217;ve had a run to burn off my fighting mood and a walk to get my brain working, and here&#8217;s where things are going. When I started the Pseudo-Official XYZZY Reviews, a couple of years ago, it was strictly a side-project. It ended up on the Awards\u00a0site, but it was off in a little [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[5],"tags":[],"_links":{"self":[{"href":"https:\/\/xyzzyawards.org\/index.php?rest_route=\/wp\/v2\/posts\/356"}],"collection":[{"href":"https:\/\/xyzzyawards.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/xyzzyawards.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/xyzzyawards.org\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/xyzzyawards.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=356"}],"version-history":[{"count":5,"href":"https:\/\/xyzzyawards.org\/index.php?rest_route=\/wp\/v2\/posts\/356\/revisions"}],"predecessor-version":[{"id":363,"href":"https:\/\/xyzzyawards.org\/index.php?rest_route=\/wp\/v2\/posts\/356\/revisions\/363"}],"wp:attachment":[{"href":"https:\/\/xyzzyawards.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=356"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/xyzzyawards.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=356"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/xyzzyawards.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=356"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}