The Auto WCAG Rules rules are authored for reliable WCAG testing. In order to facilitate this goal each rule has a variety of test cases in the categories of passed, failed and inapplicable. Incorporating a rule from the Auto WCAG Rules into your test tool, is considered an implementation. The test cases provided for the rules can be used to infer if an implementation yields the expected results.

In the spirit of Open source collaboration, the Auto WCAG Rules welcomes contributions of implementation statistics, to aid as a coverage metric of the rules. This in turn helps to improve the accuracy and completeness of both the rules and the test tools.

The below sections go into greater detail on the structure of a test case, tools for using the test cases, the anatomy of an implementation report and how to submit an implementation report, back to the community.

Auto WCAG Rules Test Cases

The rules authored in the Auto WCAG Rules, comes with several test cases, which are designed for easy consumption by test tools. The test cases are made available regularly at https://auto-wcag.github.io/auto-wcag/exported-testcases/testcases.json.

A test case for a given rule is represented as shown by the below excerpt, where:

 "a11y-testcases": [
    {
      "url": "https://auto-wcag.github.io/auto-wcag/auto-wcag-testcases/assets/SC1-1-1-image-has-name_passed_example_1.html",
      "expected": "passed",
      "ruleId": "SC1-1-1-image-has-name",
      "rulePage": "https://auto-wcag.github.io/auto-wcag/rules/SC1-1-1-image-has-name.html"
    },
    ...
 ]

Available tooling for using Auto WCAG Rules Test Cases

The community has authored an open source tool, that demonstrates consumption of the Auto WCAG Rules test cases against a test tool.

Contributions to the testrunner tool, and or other tooling of similar nature are welcome.

Refer contribution guide.

Tool Name Tool URL Language Tool Description
testrunner https://github.com/auto-wcag/testrunner JavaScript A puppeteer based implementation that allows running a configurable test tool against all the above Auto WCAG Rules test cases on a per page basis, and returns raw results.

Auto WCAG Rules Implementation Report

Incorporating a Auto WCAG Rules rule into your test tool, is considered an implementation. Submitting implementations from various test tools, helps not only showcase the coverage of test cases authored for the rule, but to build a robust understanding of the WCAG rules and the respective implementations.

Test tool authors are welcome to submit the implementation results for the above test cases to the Auto WCAG Rules community. The implementation report should be a valid JSON-LD document, where the assertions are expressed using EARL. Also, note that the report contains a list of assertion results for each of the test cases. An example assertion for a test case, is as below:

[
  {
    "@context": {
      "doap": "http://usefulinc.com/ns/doap#",
      "foaf": "http://xmlns.com/foaf/spec/#",
      "@vocab": "http://www.w3.org/ns/earl#",
      "earl": "http://www.w3.org/ns/earl#",
      "WCAG20": "http://www.w3.org/TR/WCAG20/#",
      "WCAG21": "http://www.w3.org/TR/WCAG21/#",
      "auto-wcag": "https://auto-wcag.github.io/auto-wcag/rules/",
      "dct": "http://purl.org/dc/terms#",
      "sch": "https://schema.org/",
      "WebPage": "sch:WebPage",
      "url": "dct:source",
      "assertions": {
        "@reverse": "subject"
      },
      "assertedBy": {
        "@type": "@id"
      },
      "outcome": {
        "@type": "@id"
      },
      "mode": {
        "@type": "@id"
      },
      "pointer": {
        "@type": "ptr:CSSSelectorPointer"
      },
      "test": { 
        "@type": "@id" 
      }
    },
    "@type": "WebPage",
    "url": "https://auto-wcag.github.io/auto-wcag/auto-wcag-testcases/assets/SC1-1-1-image-has-name_passed_example_1.html",
    "assertions": [
      {
        "@type": "Assertion",
        "mode": "earl:automatic",
        "assertedBy": {
          "@id": "URL by which the specific version of your tool can be identified",
          "@type": [
            "earl:Assertor",
            "earl:Software",
            "doap:Project"
          ],
          "doap:name": "Axe",
          "doap:vendor": {
            "@id": "==== YOUR ORGANIZATION URL ====",
            "@type": "foaf:Organization",
            "foaf:name": "==== YOUR ORGANIZATION NAME ===="
          }
        },
        "test": "Link to rule description",
        "result": {
          "@type": "TestResult",
          "info": "Ensures <img> elements have alternate text or a role of none or presentation",
          "outcome": "earl:passed",
          "pointer": "img"
        }
      }
    ]
  },
  ...
]

How to submit your implementation metrics

In order to submit your implementation metrics, it is recommended that the report be made available via a URL, in the above format, and an issue be opened for the same requesting that implementation metric be added to the site data.

The implementations are interpreted by the site, which are then listed in the coverage report. Each rule also has an implementation flag based on the above reports. The flags are as below:

Flag Description
New The rule has no implementation
In progress The rule has at least one implementation but no more than 2 implementations
Done The rule has 3 or more implementations

See pages:

Contribute






GitHub

Contributing is open to anyone. We welcome any new issues or pull requests for changes. Auto WCAG Rules has conference calls every 4 weeks. If you are interested in becoming an active contributor or reviewer, we ask that you join the Auto WCAG Rules community group through the W3C Website. This requires setting up a W3C account, may require approval by the organization you work for if they are a W3C member.

Learn more about contributing to Auto WCAG Rules Join the Auto WCAG Rules community group now!