Page Comparison

This document describes the formal testing process of the ACPF Project.   

The City evaluates applications against WCAG 2.1 Level AA, along with a set of established best best practices and WCAG 2.1 AAA criteria outlined in the City of Toronto Digital Accessibility Principles & Resources ("City Standard").

The testing processes described below assume an experienced accessibility tester is executing the evaluation. This document should be considered a guide, not an exhaustive list of tests. It is expected that experienced accessibility testers will recognize issues not included here. Tests will continue to be added to this document as they are developed and adopted. 

Please see our Web Accessibility Tools List for a list of tools, plugins, etc. that can be used when executing these tests.

Anchor
_3kzbztht3cx9 _3kzbztht3cx9

A. Semi-automated Testing

...

1. Review the page as a whole.
   • Any issues raised by a tool should be further examined as a candidate for documentation.
   • If the issue is confirmed as a Violation of the City Standard, it is documented.
2. Check the page's document structure. Document any of the following issues:
   • Missing <h1> level heading, incorrect heading nesting (skipped heading levels), headings used for display or presentation only, headings that do not describe the content that the section contains.
3. Check the page's colour contrast. (Use a tool that supports WCAG 2.1 AAA contrast testing, such as the WCAG Contrast checker browser extension.)
   • Text: All body text has a contrast ratio of at least 7:1, unless it is large scale text.
   • Large scale text: Contrast ratio of at least 4.5:1. Higher contrast ratios are preferred.
   • Large scale text is defined as:
     • 1.2 rem or larger (if bold) or 1.5 rem or larger (if not bold);
     • 120% or larger (if bold) or 150% or larger (if not bold) of the default size for body text.
     • Boldface is defined as a font-weight of 700 or more.
   • Link text: Contrast ratio of at least 3:1 with any surrounding body text.
   • UI components such as icons, inputs, buttons, their outlines, and their various states must have sufficient contrast.
     • When colour is used to distinguish state, this difference needs to have a contrast of at least 3:1 against adjacent colour(s).
     • Components must be sufficiently distinct from their own background. This contrast ratio must be at least 3:1 against adjacent colour(s).
4. Check the use of colour. View the page in grayscale and visually check for colour and contrast issues.
   • Do meaningful details disappear? Sections and boundaries denoted by colour alone?
5. View the page with styling disabled ("No style").
   • The visual reading order should remain logical and intuitive.
   • The reading order should be programmatically determinable. See Understanding Success Criterion 1.3.2: Meaningful Sequence for information on how to ensure reading order is programmatically determinable.
6. View the page after applying text spacing updates (e.g., using the Text Spacing Bookmarklet).
   • Any loss of content or functionality should be documented.
7. Check the page's use of HTML5 and ARIA Landmarks.
   • Landmarks should be present.
   • Landmarks should be used according to ARIA specifications and HTML5 specifications (e.g. nested appropriately, used only once for banner, contentinfo, and main roles).
   • Content contained within the landmark matches the correct use case for that landmark (e.g., contentinfo landmark is used for footer content).
   • All content is within a landmark. There are a small enough number of landmarks that users have a concise overview of the content.
   • For landmarks that are used more than once on a page, the landmarks are labelled where appropriate.
   • Labels, where used, are unique and sufficiently descriptive.
   • HTML5 landmarks, where available, are preferred.
8. Check the page uses valid code.
   • Any significant errors reported by code validation tools should be further examined as a candidate for documentation.
   • If the issue is confirmed as a Violation of the City Standard, it is documented.
9. For pages with dynamic code, check the page uses valid code as the DOM is updated and changed.
   • Use the browser's developer tools to test the generated markup. Do not test by URL.
   • Any significant errors reported by code validation tools should be further examined as a candidate for documentation.
   • If the issue is confirmed as a Violation of the City Standard, it is documented.
10. If the page supports Responsive Design and/or Adaptive Design, repeat all of the above steps at each major breakpoint.
    • Any new errors or differences from the above results should be further examined as candidates for documentation.
    • If the issue is confirmed as a Violation of the City Standard, it is documented.

...

While the steps outlined in the above Section A, Semi-automated Testing, can identify several major code-related issues, code inspection is still required, in particular:

1. Inspection of tool-identified issues.
   • Where an automated or semi-automated tool identifies an issue, experienced accessibility testers are expected to review all aspects of the issue raised, including the underlying code, to identify:
     • Whether the issue is a Violation of the City Standard and
     • Possible approaches to remediate the issue.
2. Check for use of inline styles.
   • Use the browser's Developer Tools to inspect the DOM.
   • Search the HTML for inline styles (e.g., search for "style=").
     • Examine the search results as candidates for documentation.
     • If an item is confirmed as a Violation of the City Standard, it is documented.
3. The page has a <title> element (or equivalent).
   • Every page has a descriptive, informative, unique title.
   • The title describes the content, topic, or purpose of the page.
   • The page's title is unique and distinguishable from other pages.
   • If there are frames on the page, all frames and iframes have titles.
   • Single page applications (SPAs) must explicitly update the page's title when changes in the view mimic a new page.
     • SPAs explicitly announce to the screen reader the new page title.
4. If the page supports Responsive Design and/or Adaptive Design, repeat all of the above steps at each major breakpoint.
   • Any new errors or differences from the above results should be further examined as candidates for documentation.
   • If the issue is confirmed as a Violation of the City Standard, it is documented.

...

1. Explore the page content for interactive elements.
   • Identify all interactive elements that respond to the mouse (e.g., hover, click, etc.).
   • These interactive elements must have corresponding responses when keyboard testing (see C.2 Keyboard Testing for more information) .
2. Test pointer cancellation for all interactive elements that can be activated with mouse.
   • With mouse cursor positioned over target, click and hold primary mouse button.
     • Ensure element is not activated on the down event of the mouse click.
   • With mouse cursor positioned over target, click and hold primary mouse button. While holding the mouse button, move mouse cursor away from target, then release the mouse button.
     • Ensure element is not activated.
   • If element is a drag-and-drop component:
     • Clicking primary mouse button selects an item.
     • If mouse cursor is not moved, releasing mouse button should deselect the item.
   • Exception: If the functionality requires an action to happen on the down event (e.g., a drum machine).

...

1. Check bypass block links (or "skip links") are present.
   • These links can be visible by default or visible on focus.
     • Absence of skip links should be documented.
   • If skip links are present, one of them needs to be the first element in the focus order of the page.
     • If the first focusable element is not a skip link, this should be documented.
2. Check for logical focus order.
   • The sequence of the focus order of all focusable elements on the page follows the intuitive reading order of the page (typically top-down, left-to-right).
3. All elements which visually appear to be interactive receive keyboard focus.
   • Focus should move through all actionable objects, elements, and controls (e.g., buttons, radio buttons, checkboxes, dropdowns, menus, etc.).
   • If elements thought to be interactive do not get keyboard focus, determine whether they can be interacted with using a mouse or other input.
4. All elements with mouse interactions (e.g., hover effects, etc.) can receive keyboard focus.
   • Focus should move through all actionable objects, elements, and controls (e.g., buttons, radio buttons, checkboxes, dropdowns, menus, etc.).
5. Check for visible focus.
   • All focusable elements have a visible focus state.
   • Visible focus must have sufficient contrast with adjacent UI elements and their own background (contrast ratio is at least 3:1).
6. Focus is never lost on an invisible element.
   • Visible focus does not land on an otherwise visibly empty portion of the screen.
   • Focus does not disappear between elements.
7. Keyboard traps.
   • The page is free of keyboard traps.
   • Where traps exist, there is a clear instruction (also announced by screen readers) on how to exit the trap using keyboard alone.
8. Check that focusing on any element does not trigger a change of context.
   • Changes should only occur when the user interacts with an element.
9. If focusing on an element reveals content (e.g., tooltip, popup, dropdown).
   • Determine if the revealed content is Dismissable, Hoverable, and Persistent.
   • See C.2.2 Interaction Test for more information.
10. If the page supports Responsive Design and/or Adaptive Design, repeat all of the above steps at each major breakpoint.
    • Watch for new elements that are introduced in each view.
    • Any new errors or differences from the above results should be further examined as candidates for documentation.
    • If the issue is confirmed as a Violation of the City Standard, it is documented.
11. Repeat the above steps using a screen reader.
    • Document any new errors or differences.

...

1. Check bypass block links (or "skip links

   Anchor
   _GoBack _GoBack

   ") work correctly.
   • Enter key activates the link.
   • Activating the link moves you to the expected target of the link.
   • Keyboard focus is moved as well as visible focus. Pressing the tab key should move to the next focusable element after the target destination.
2. Check links work correctly.
   • Enter key activates the link.
   • Activating the link moves you to the expected target of the link.
   • Link takes you to another page or a destination within the current page.
   • The link text is appropriately descriptive and accurate. The purpose/target of the link is easily understood.
3. Check buttons work correctly.
   • Both Enter key and Spacebar key activate the button.
   • The button text is appropriately, accurately, and uniquely descriptive. The resulting action is expected.
4. UI elements that change context upon input:
   • Change as expected based on UI element type (e.g., checkbox is selected or deselected).
   • Change as expected based on information provided to the user in advance.
5. Buttons that toggle content (e.g., popup, accordion, other content disclosure - click to open the content):
   • The new content receives focus or is the element immediately following the toggle in the DOM.
   • The new content can be interacted with.
   • Closing the new content returns focus to button being tested.
6. Autocomplete Search inputs. Try to use when empty, with a character or two for an initial search, and with phrases and words.
   • If there is a dropdown result list, you can interact with it (e.g., arrow up and down).
   • Focusing on any element does not trigger a change of context (e.g., pressing tab while on a suggested result triggers a search, arrow down to a result triggers a search, etc.).
7. Content revealed on hover or focus (tooltips, popups, dropdown menus - hover or focus to open the content).
   • Note: these tests are NOT intended for modals or browser-generated "tooltips" (i.e., tooltips from the title attribute or the HTML5 required attribute).
     1. Revealed content can be dismissed with the Escape key.
     2. Content persists until dismissed (i.e., remains on screen and does not disappear after a set time).
     3. User can tab to focusable elements within the revealed content without the content disappearing.
     4. If using a mouse, hovering over the same element will trigger the same revealed content.
8. If the page supports Responsive Design and/or Adaptive Design, repeat all of the above steps at each major breakpoint.
   • Watch for new elements that are introduced in each view.
   • Any new errors or differences from the above results should be further examined as candidates for documentation.
   • If the issue is confirmed as a Violation of the City Standard, it is documented.
9. Repeat the above steps using a screen reader.
   • Document any new errors or differences.

...

Note: As a rule, keyboard controls and ARIA roles, states, and properties for components should match the patterns outlined in the WAI-ARIA Authoring Practices 1.1 Specification.

Anchor
_lp8todg53z97 _lp8todg53z97

F.1 General

...

1. All form controls (including any submit button) can be interacted with according to WAI-ARIA Authoring Practices specification(s).
2. If the form imposes any kind of time limits, ensure it meets WCAG SC 2.2.1 Timing Adjustable.
3. If the form involves legal or financial transactions, modifying user data, or submitting test responses, ensure it meets WCAG SC 3.3.4 Error Prevention (legal, financial, data).

Anchor
_m6mnrvwrhpsk _m6mnrvwrhpsk

G.2.2 Forms Code Inspection Testing

...

1. Review all of the keyboard testing items listed in C.2 Keyboard Testing above.
2. Visible labels.
   • All input fields have a visible label. Placeholder text is not the sole way to identify the purpose of the field.
   • Label is adequately descriptive. The user understands what is being requested.
   • Using a mouse to click on the label sets focus to the associated input.
   • The label is programmatically associated with the form element.
3. Groups of controls
   • Related form elements are grouped in a fieldset, section, or similar programmatically semantic way.
   • Grouped controls have a heading, legend, and/or aria-label.
   • Any heading, legend, or aria-label describing the group is adequately descriptive. The user understands the relationship among the grouped controls.
4. Required fields
   • Required fields are indicated using more than colour alone.
   • Required fields are programmatically indicated as required (e.g., aria-required="true").
   • Symbols indicating a required field are explained in body text.
5. Error handling
   • Try to submit the form with as many errors as possible and evaluate the error messaging strategy:
     1. Users should be aware there are errors in the form submission.
     2. Error text is clearly visible near the input.
     3. Error text is appropriately descriptive. The user knows what the issue is and how to fix it.
6. Complete the form correctly using the keyboard, then submit the form.
   • A success message should appear. Users should be aware that the form has been submitted successfully.

...

1. Repeat the steps listed in G.2.3 Forms Keyboard Testing using a screen reader.
   • Navigate the form content using the "forms/focus mode" of the screen reader.
   • Using the TAB key will navigate between form controls and buttons.
2. For each form element that you navigate to in "focus mode":
   • The correct label for the element is announced.
   • Help text visually placed near the form element is announced as part of the form element (e.g., using aria-describedby).
   • If the form element is visually indicated as required, it is announced as "required".
   • If a form element is announced as "required", it is also visually indicated as "required".
3. Groups of controls.
   • For form controls grouped together visually with a heading or legend (and/or aria-label), the heading/legend is announced before the form controls' label.
   • The announced heading, legend, and/or aria-label is sufficiently descriptive such that a non-visual user will understand the relationship(s) shared by the group.
4. Error handling.
   • Try to submit the form with as many errors as possible.
     1. Users are made aware that there are errors in the form submission.
   • Navigate through the form elements in "focus mode".
     1. For every field with an error, the error text is announced by the screen reader.
     2. The error announcement is sufficiently descriptive such that a non-visual user will understand the error and how to fix it.
5. Complete the form correctly, then submit the form.
   • A success message should be announced. Users should be aware that the form has been submitted successfully.

...

1. View the page with styling disabled ("No style").
   • With CSS disabled, content of HTML tables still makes sense.
2. If the page supports Responsive Design and/or Adaptive Design, review the table at each major breakpoint.
   • The table should resize gracefully.
   • If the table requires horizontal scrolling, ensure that this is clear to the user and remains keyboard operable.
   • Watch for new elements that are introduced in each view.

...

1. Navigate the table using the "tables mode" of the screen reader.
   • Ensure the screen reader's keyboard table navigation shortcuts behave as expected.
2. If the page supports Responsive Design and/or Adaptive Design, review the table at each major breakpoint.
   • Confirm that the table remains screen reader accessible at all major breakpoints.

...