-
Notifications
You must be signed in to change notification settings - Fork 1.6k
fix: Screen reader not announcing Adaptive Card content in stacked layout #5784
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
uzirthapa
wants to merge
14
commits into
microsoft:main
Choose a base branch
from
uzirthapa:fix/adaptive-card-screen-reader-a11y
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
c594808
fix: Screen reader not announcing Adaptive Card content in stacked la…
uzirthapa 51cc2a3
fix: Use inset outline-offset so focus ring is visible inside overflo…
uzirthapa d6b191e
Revert "fix: Use inset outline-offset so focus ring is visible inside…
uzirthapa 8409eef
test: Add e2e tests for Adaptive Card screen reader a11y fix
uzirthapa ee877bf
fix: Address lint errors in a11y changes
uzirthapa 36a2a3a
fix: Address lint errors in a11y changes
uzirthapa 9a1279e
ci: Retrigger CI to verify flaky useTextBox test
uzirthapa aaee156
fix: Use tabIndex={-1} to avoid breaking focus traps in Adaptive Cards
uzirthapa b8b9f02
ci: Retrigger CI for flaky useTextBox test
uzirthapa 07bfe74
fix: Only use role="form" when card has original aria-label from speak
uzirthapa 2a1926d
ci: Retrigger CI
uzirthapa 8866e95
fix: Remove tabIndex and focus CSS from AttachmentRow to avoid test f…
uzirthapa 7185a19
fix: Update comment to accurately describe textContent usage
uzirthapa dd9cbfd
fix: Remove aria-label truncation to announce full card content
uzirthapa File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
78 changes: 78 additions & 0 deletions
78
__tests__/html2/accessibility/adaptiveCard/attachmentRow.focusable.html
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| <!doctype html> | ||
| <html lang="en-US"> | ||
| <head> | ||
| <link href="/assets/index.css" rel="stylesheet" type="text/css" /> | ||
| <script crossorigin="anonymous" src="/test-harness.js"></script> | ||
| <script crossorigin="anonymous" src="/test-page-object.js"></script> | ||
| <script crossorigin="anonymous" src="/__dist__/webchat-es5.js"></script> | ||
| </head> | ||
| <body> | ||
| <main id="webchat"></main> | ||
| <script> | ||
| run(async function () { | ||
| const { directLine, store } = testHelpers.createDirectLineEmulator(); | ||
|
|
||
| WebChat.renderWebChat({ directLine, store }, document.getElementById('webchat')); | ||
|
|
||
| await pageConditions.uiConnected(); | ||
|
|
||
| await directLine.emulateIncomingActivity({ | ||
| attachments: [ | ||
| { | ||
| content: { | ||
| type: 'AdaptiveCard', | ||
| $schema: 'http://adaptivecards.io/schemas/adaptive-card.json', | ||
| version: '1.5', | ||
| body: [ | ||
| { | ||
| type: 'TextBlock', | ||
| text: 'First card content' | ||
| } | ||
| ] | ||
| }, | ||
| contentType: 'application/vnd.microsoft.card.adaptive' | ||
| } | ||
| ] | ||
| }); | ||
|
|
||
| await directLine.emulateIncomingActivity({ | ||
| attachments: [ | ||
| { | ||
| content: { | ||
| type: 'AdaptiveCard', | ||
| $schema: 'http://adaptivecards.io/schemas/adaptive-card.json', | ||
| version: '1.5', | ||
| body: [ | ||
| { | ||
| type: 'TextBlock', | ||
| text: 'Second card content' | ||
| } | ||
| ] | ||
| }, | ||
| contentType: 'application/vnd.microsoft.card.adaptive' | ||
| } | ||
| ] | ||
| }); | ||
|
|
||
| await pageConditions.numActivitiesShown(2); | ||
|
|
||
| const attachmentRows = document.querySelectorAll('[aria-roledescription="attachment"]'); | ||
|
|
||
| // Both attachment rows should exist with proper a11y attributes. | ||
| expect(attachmentRows).toHaveProperty('length', 2); | ||
| expect(attachmentRows[0].getAttribute('role')).toBe('group'); | ||
| expect(attachmentRows[1].getAttribute('role')).toBe('group'); | ||
|
|
||
| // The Adaptive Cards inside should have aria-labels derived from text content. | ||
| const firstCard = attachmentRows[0].querySelector('.ac-adaptiveCard'); | ||
| const secondCard = attachmentRows[1].querySelector('.ac-adaptiveCard'); | ||
|
|
||
| expect(firstCard.getAttribute('aria-label')).toContain('First card content'); | ||
| expect(firstCard.getAttribute('role')).toBe('figure'); | ||
|
|
||
| expect(secondCard.getAttribute('aria-label')).toContain('Second card content'); | ||
| expect(secondCard.getAttribute('role')).toBe('figure'); | ||
| }); | ||
| </script> | ||
| </body> | ||
| </html> |
95 changes: 95 additions & 0 deletions
95
__tests__/html2/accessibility/adaptiveCard/hack.roleMod.ariaLabelFromTextContent.html
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| <!doctype html> | ||
| <html lang="en-US"> | ||
| <head> | ||
| <link href="/assets/index.css" rel="stylesheet" type="text/css" /> | ||
| <script crossorigin="anonymous" src="/test-harness.js"></script> | ||
| <script crossorigin="anonymous" src="/test-page-object.js"></script> | ||
| <script crossorigin="anonymous" src="/__dist__/webchat-es5.js"></script> | ||
| </head> | ||
| <body> | ||
| <main id="webchat"></main> | ||
| <script> | ||
| run(async function () { | ||
| const { directLine, store } = testHelpers.createDirectLineEmulator(); | ||
|
|
||
| WebChat.renderWebChat({ directLine, store }, document.getElementById('webchat')); | ||
|
|
||
| await pageConditions.uiConnected(); | ||
|
|
||
| await directLine.emulateIncomingActivity({ | ||
| attachments: [ | ||
| { | ||
| content: { | ||
| type: 'AdaptiveCard', | ||
| $schema: 'http://adaptivecards.io/schemas/adaptive-card.json', | ||
| version: '1.5', | ||
| body: [ | ||
| { | ||
| type: 'TextBlock', | ||
| text: 'Flight Status Update' | ||
| }, | ||
| { | ||
| type: 'TextBlock', | ||
| text: 'Flight AA1234 from Seattle to New York' | ||
| } | ||
| ] | ||
| }, | ||
| contentType: 'application/vnd.microsoft.card.adaptive' | ||
| }, | ||
| { | ||
| content: { | ||
| type: 'AdaptiveCard', | ||
| $schema: 'http://adaptivecards.io/schemas/adaptive-card.json', | ||
| version: '1.5', | ||
| speak: 'Custom speak text for screen readers', | ||
| body: [ | ||
| { | ||
| type: 'TextBlock', | ||
| text: 'This text should not be the aria-label' | ||
| } | ||
| ] | ||
| }, | ||
| contentType: 'application/vnd.microsoft.card.adaptive' | ||
| }, | ||
| { | ||
| content: { | ||
| type: 'AdaptiveCard', | ||
| $schema: 'http://adaptivecards.io/schemas/adaptive-card.json', | ||
| version: '1.5', | ||
| body: [ | ||
| { | ||
| type: 'Input.Text', | ||
| id: 'name', | ||
| label: 'Your Name', | ||
| placeholder: 'Enter your name' | ||
| } | ||
| ] | ||
| }, | ||
| contentType: 'application/vnd.microsoft.card.adaptive' | ||
| } | ||
| ] | ||
| }); | ||
|
|
||
| await pageConditions.numActivitiesShown(1); | ||
|
|
||
| const [cardNoSpeak, cardWithSpeak, cardFormNoSpeak] = Array.from( | ||
| document.querySelectorAll('.ac-adaptiveCard') | ||
| ); | ||
|
|
||
| // Card without speak: aria-label should be derived from visible text content. | ||
| expect(cardNoSpeak.getAttribute('aria-label')).toContain('Flight Status Update'); | ||
| expect(cardNoSpeak.getAttribute('aria-label')).toContain('Flight AA1234'); | ||
| expect(cardNoSpeak.getAttribute('role')).toBe('figure'); | ||
|
|
||
| // Card with speak: aria-label should use the speak property value. | ||
| expect(cardWithSpeak.getAttribute('aria-label')).toBe('Custom speak text for screen readers'); | ||
| expect(cardWithSpeak.getAttribute('role')).toBe('figure'); | ||
|
|
||
| // Card with form inputs and no speak: aria-label should be derived from text content, | ||
| // but role should be "figure" (not "form") to avoid duplicate form landmarks on the page. | ||
| expect(cardFormNoSpeak.getAttribute('aria-label')).toBeTruthy(); | ||
| expect(cardFormNoSpeak.getAttribute('role')).toBe('figure'); | ||
| }); | ||
| </script> | ||
| </body> | ||
| </html> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
textContentisn't the appropriate way to get the accessible content out of DOM.This is the right approach, https://www.w3.org/TR/accname-1.2/#computation-steps.
Some time ago, I was trying to do the same thing. 🤣
Right now, you can either:
axe-corepackageaxe-coreis MPL 2.0 while we are MITAlso, you need to see if
<button>is done properly, e.g. we probably want it to say, "Click me button", than just "Click me". That means we need to localize "button" and every ARIA roles. Then we might need to say "checkbox checked" (aria-checked) and kinda... then we might need to localize every ARIA attributes. Notably,aria-posinset(position in set) andaria-setsizeneed to be localized in pair ("checkbox, checked, 1 of 3"), etc.Then... after this is implemented, you may find "checkbox, checked, 1 of 3" is Windows way of narrating stuff, and "1 of 3 checkboxes, checked" is macOS way (I forget which is which, but you got the idea). And people may say, "the narration is not uniform" or they may never discover this, kinda-kinda.
I am definitely not discouraging you working on this, I know too much of this. You will soon find out you are writing a non-small part of a screen reader. 🤣
p.s. I also looked at the old screen reader on Chromebook which is JS-based, but Google discontinued it long time ago and it's defunct.
The other way, just use screen reader to read the card... but if the card content is too complex, then screen reader bugged out (says, stop narrating in the middleware of the card, etc.) Then people complained it's a Web Chat issue than AC or screen reader issue...
That's why we stay status quo on rendering side. And instead, ask AC author to add
speakproperty than Web Chat compute it for them. Adaptive Cards is more like a figure/image and alt-text for figure/image should be mandated.p.s. I don't think
speakis a good name and that's why people don't recognize the need of it, it should bealtor something, but I don't control AC schema.