Menace

Saturday, 26th May 2007, No Comments »

Validating the 2007 Qantas media awards

I’ve never really been satisfied with the validation ‘criteria’ when validating the netguide awards, but initially for a whim it did the job. The 2007 results took me around 2 and a half hours to do and there’s an awful lot of interpretation people can take away from the results (The person that did it is a web standards zealot, I validated so why should I use semantic mark up, etc). So I decided to take this opportunity with the Qantas Media awards, having a smaller number of sites, to try some new tests and see how it worked out (still took about an hour and a half).

The new tests I did where,

• An automated Accessibility test
• Layout type
• Page Encoding
• Page Size

However, I’m still not happy with it.

Semantic mark up

Some semantic mark up could be tested, for example heading levels, but it would require creating an unofficial testing standard (what happens if a site is using headings correctly, but not tables on 3 out of 4 pages?) and trolling through a site to find examples to test. I’d prefer to use something open and transparent as testing benchmark, than something I thought up by myself while out walking the dog.

I could also see testing for semantic mark up as a slippery slope rife with interpretation and accusation. I never, ever wanted for these tests to be a ‘name and shame’ exercise. It’s not a junior developers fault if they’re not aware of the ‘q’ tag or a content administrator’s fault that their CMS can’t allow them to mark up a paragraph when it’s not in the document’s natural language (see: lang attribute). Without that kind of ‘internal knowledge’ semantic mark up testing could easily descend into a pissing match.

Automated Accessibility testing

The reason I’ve never really added accessibility tests into the results is because automated accessibility testing systems are in my opinion, next to useless. The tools can only realistically test a very small section of potential issues for whose with disabilities and the ones they do get right often overlap with the HTML and/or CSS results (making sites look worse than they are). I’m of the opinion that accessibility testing is really just usability testing with people with disabilities and thus needs real people for testing.

I had considered using an automated readability index system, but I found I got very different results based on testing the whole page versus just an article. I had also considered using a colour-blindness simulator, but I’ve found them to be a little unreliable when compared with real users.

So next year…

I’m really interested to hear people’s feedback, positive and negative, about the ‘criteria’ because I think the tests bring a lot of interesting conversion and difficult questions into the discussions about web design (”Web standards aren’t required for a successful site”, “Are we making sure we’re doing the best for every user”, etc).