[gui-talk] Fwd: Article: Web Accessibility Surveys Results are FrequentlyDisappointing

[Steve Pattison]

 From:    John Rae thepenguin at rogers.com
 To:      vip vip-l at softspeak.com.au

Web Accessibility Surveys Results are Frequently Disappointing

By David Sloan 

A recent exchange on Twitter has motivated me to write about the
contribution published surveys on web site accessibility make towards
understanding and addressing the problems that hold back web accessibility.
I've read, and continue to read, many, many papers presenting the results of
surveys of web sites, and I think we need surveys to look beyond just the
data and instead delve more deeply into why the results are as they are.
We've gone way beyond the
point where a paper simply reporting that a study of x web sites from y
sector revealed 'disappointing' levels of accessibility provides anything
more than a minor contribution. Surveys need to look at process not product.

In the early days of web accessibility, post WCAG 1.0 release, published
surveys of the accessibility of large numbers of web sites were relatively
rare (I'm distinguishing these from reviews of a single site conducted by or
on behalf of the development team, with the specific aim of identifying and
repairing barriers present). So whenever a new survey emerged, it usually
provided informative data on levels of conformance against WCAG 1.0, which
took time achieve any significant impact on the web design industry. The
data allowed us to see how particular sectors were faring, and which
checkpoints were most frequently not met.

The publicity surrounding a published accessibility survey that presented
data showing how poorly sites were dealing with WCAG conformance could also
be claimed to raise awareness of web accessibility in general, and more
specifically shame the organisations in question into doing something about
the barriers present on their site. The former effect probably did take
place, although I'd like to see concrete evidence that surveys actually have
a positive effect on the organisations whose sites were reviewed.

(Indeed, there have been concerns that surveys may have a negative impact on
'usable accessibility.' If the methodology used focuses excessively on a
technical measure of accessibility that becomes a highly public 'official'
ranking of each site's performance - with rewards for finishing high up a
ranking, there would be understandable pressures for site developers to
design to satisfy the surveyors and not disabled people.)

Many published surveys have had severe limitations in methodology and scope
- frequently conducted using automated tools only, using a subset of WCAG,
and often of the Home page only; and very, very rarely have researchers
extended their survey to contact each site's organisation for follow-up data
(a notable exception was a study by Ronald Milliman in 2002). Several
published surveys have appeared as academic papers in a wide variety of
journals (not just computing/HCI).
As the topic was initially relatively uncovered in academic literature, an
investigation into the accessibility of web sites in a particular sector -
be it higher education, government, e-commerce, tourism, or whatever - made
for an attractive publication topic. I should know, I wrote a couple! But at
least in the early days we had some data to help say ' hey, we all need to
do a bit better here'.

Over time, we've had some very high impact surveys, such as the UK DRC
Formal Investigation into web accessibility, published in 2004, and which
combined automated testing with manual inspections; evaluations with
disabled people and interviews with web developers. Just recently, at the
Accessing Higher Ground 2009 conference, Terrill Thompson presented data
from a longitudinal study that compared progress towards accessible content
between organisations who had received technical support and those who
hadn't.

Additionally, publications have presented accessibility survey data as part
of other valuable research activity, for example when testing new
methodologies and tools for large scale accessibility evaluation (important
in the real world for regulatory bodies as well as organisations with many
web pages), and testing how effective particular evaluation methodologies
could be in minimising false positives and false negatives (check the
proceedings of W4A  for papers by researchers such as Markel Vigo, Giorgio
Brajnik and Bambang Parmanto).

Putting aside limitations of scope and evaluation methodology, surveys have
not been so good at focusing on content providers and the content provision
circumstances. This is to some extent understandable, as it's easier to run
an automated tool across multiple sites than identify, contact and gather
data from all the content providers for each site.

Some exceptions are mentioned above - to add to these, my thesis focused on
the impact of accessibility audits on the recipient organisations, and my
research yielded a limited amount of information on how organisations
responded. A few surveys have specifically looked perceptions on and
attitudes to web accessibility  for example Lazar et al's
2004 paper on
Improving web accessibility: a study of webmaster perceptions  and Bloor
Research's 2009 survey), but we still don't know very much about the
organisational reasons as to why accessibility of a particular web site is
not as good as it could be.

As time goes by, therefore, the impact of a survey that presents data on web
accessibility diminishes, unless it adds something new to our understanding
of the problem. In 2009, it's not enough to simply claim that results are
'disappointing', and that web content authors must  do better'. WCAG has
been here for 10 years, so it's not as if we have no best practice; and
people and resources promoting and supporting web accessibility are easy to
find. A failure to acknowledge in a survey report that this has been a
problem for many years does a disservice to everyone who has been advancing
the cause of web accessibility (even if it also gives another stark reminder
that there's much work to do).

So if you conduct a survey of web sites and find accessibility barriers,
don't stop at reporting conformance levels. Find out why the barriers exist.
Is it a lack of awareness or training amonst the providers of the content
you surveyed? Is it a lack of involvement of disabled people in the design
process? Is it a lack of resources; of money; a lack of will, or lack of
prominence of accessibility in an organisation's business practice and
philosophy? Is it sub-standard authoring tools, content management systems;
quality assurance systems? Is there another reason?

And whatever you find out, please share it with us!

http://58sound.com/2009/11/15/web-accessibility-surveys-results-are-frequent
ly-disappointing

Regards Steve
Email:  srp at internode.on.net
MSN Messenger:  internetuser383 at hotmail.com
Skype:  steve1963
Twitter:  steve9782