[Accessibility-ia2] next changes to IAccessible2

Thu Jun 30 11:27:17 PDT 2011

I use IAccessibleEditableText. But not in relation to the clipboard. 
insertText, deleteText, and replaceText are the only member functions I 
use.

Arnstein

On 28.06.2011 20:54, Pete Brunet wrote:
> AT devs, do you use IAccessibleEditableText?
>
> Is there a kind of AT that can't use the GUI for clipboard operations?
> The same question often comes up regarding IAccessibleAction, i.e. why
> have actions if the same thing can be done via the GUI.  My
> understanding is that this is for those who can't use the mouse and
> keyboard.  Do ATs for those users not have the ability to use GUI menus?
>
> Pete
>
> On 6/28/2011 2:29 AM, Malte Timmermann wrote:
>> Actually, it's also not clear to me why the clipboard stuff exists at all.
>>
>> Maybe it was introduced with JAA/ATK, because there is no "standard
>> platform mechanism" on all supported platforms?
>>
>> I don't know - how would an AT tell any application on GNOME or Mac to
>> copy/paste, w/o faking some key events?
>>
>> The clipboard API is incomplete anyway. For example, I don't have an
>> equivalent API for copy/paste on a drawing document or a bitmap editor.
>> Only for objects supporting accessible text.
>>
>> Malte.
>>
>> Carolyn MacLeod wrote, On 27.06.2011 21:06:
>>> An excellent point, Peter.
>>>
>>> In fact, I just read Malte's discussion on the same API, and something
>>> he said - "If [the user] really is interested in selecting a certain
>>> format, he can use the dialog in the application." - reminded me of our
>>> initial reaction when we were asked to implement
>>> IAccessibleEditableText::copyText, cutText, and pasteText in our
>>> StyledText control.... namely, "Why??".
>>>
>>> Why do AT need a separate way of doing something that every editable
>>> text control already provides by a multitude of standard platform
>>> mechanisms?
>>>
>>> I still don't have the answer. However, since we implemented
>>> IAccessibleEditableText in order to provide insert/replace/delete and
>>> allow setting of attributes, we then implemented cut/copy/paste anyway,
>>> but only because they were in the same interface.
>>>
>>> Our second question was "What are the offsets for? Why not just use the
>>> selection?". The answer to that was that since IAccessibleText::copy
>>> needed offsets because there might not be a selection, someone decided
>>> that IAccessibleEditableText cut/copy/paste needed offsets, too, for
>>> consistency. This was an unfortunate decision, because it gives these
>>> methods a very strange side effect: the server is forced to change the
>>> selection to match the offsets before doing the clipboard operation. We
>>> grudgingly implemented these methods, and hoped that no AT would ever
>>> send us any offsets other than those for the already existing selection.
>>>   :)
>>>
>>> (Epilogue: I cannot find a definition of IAccessibleText::copy anywhere,
>>> so the original reason that there are offsets for the clipboard methods
>>> has been completely lost in the mists of time).
>>>
>>> Now, since multi-selection makes the strange semantics even stranger, I
>>> am happy to revert back to our initial impression that
>>> IAccessibleEditibleText[2] doesn't need clipboard operations at all.   :)
>>>
>>> Carolyn
>>>
>>>
>>> From: 	Peter Korn<peter.korn at oracle.com>
>>> To: 	Carolyn MacLeod/Ottawa/IBM at IBMCA
>>> Cc: 	Alexander Surkov<surkov.alexander at gmail.com>,
>>> accessibility-ia2-bounces at lists.linuxfoundation.org, IA2 List
>>> <Accessibility-ia2 at lists.freestandards.org>
>>> Date: 	27/06/2011 02:22 PM
>>> Subject: 	Re: [Accessibility-ia2] next changes to IAccessible2
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>>
>>>
>>> Carolyn, gang,
>>>
>>> Before we get too deep into (re-) designing this API... Do we have a
>>> clear set of use cases?  What AT do we expect to call this/these API
>>> call(s), supporting which use cases?
>>>
>>>
>>> Regards,
>>>
>>> Peter
>>>
>>> On 6/27/2011 11:13 AM, Carolyn MacLeod wrote:
>>>
>>> Hi, all.
>>>
>>> Regarding *IAccessibleEditableText2::pasteText(startOffset, endOffset,
>>> mimeType)*:
>>>
>>> 1) If we are going to use "mime type" to specify the clipboard format,
>>> then I think we need an explicit mapping from mime type to
>>> platform-specific clipboard format so that AT and server are both
>>> speaking the same language. For example, something like:
>>> MIME type	Windows	GTK/ATK
>>> text/plain	CF_TEXT (predefined)	gdk_atom_intern("COMPOUND_TEXT")
>>> gdk_atom_intern("UTF8_STRING")
>>> gdk_atom_intern("STRING")
>>> text/rtf	CF_RTF = RegisterClipboardFormat("Rich Text Format")
>>> gdk_atom_intern("text/rtf")
>>> gdk_atom_intern("TEXT/RTF")
>>> gdk_atom_intern("application/rtf")
>>> text/html	CF_HTML = RegisterClipboardFormat("HTML Format")
>>> gdk_atom_intern("text/html")
>>> gdk_atom_intern("TEXT/HTML")
>>>
>>>
>>>
>>>
>>> 2) Do we allow only text/* MIME types? Or do we allow all mime types?
>>> For example, is *pasteText(10, 20, "image/jpeg")* allowed? Probably
>>> should be, given that most word processors can paste an image (among
>>> other things) at a specific location, but the name pasteText is wrong
>>> for that context (unless it means "paste the *into *the text"...).
>>>
>>> For the curious, here is a typical list of text/* mime types, taken from
>>> Java AWT data transfer's "best text flavor" method:_
>>> __http://download.oracle.com/javase/6/docs/api/java/awt/datatransfer/DataFlavor.html#selectBestTextFlavor(java.awt.datatransfer.DataFlavor_
>>> <http://download.oracle.com/javase/6/docs/api/java/awt/datatransfer/DataFlavor.html#selectBestTextFlavor%28java.awt.datatransfer.DataFlavor>[])
>>> •"text/sgml"
>>> •"text/xml"
>>> •"text/html"
>>> •"text/rtf"
>>> •"text/enriched"
>>> •"text/richtext"
>>> •"text/uri-list"
>>> •"text/tab-separated-values"
>>> •"text/t140"
>>> •"text/rfc822-headers"
>>> •"text/parityfec"
>>> •"text/directory"
>>> •"text/css"
>>> •"text/calendar"
>>> •"application/x-java-serialized-object"
>>> •"text/plain"
>>> •"text/<other>"
>>>
>>> 3) We need to more fully specify: "If mime type is missed then
>>> application choose more appropriate one depending on insertion context."
>>> i.e. If mime_type is empty string or NULL? Also, if mime_type is
>>> unsupported by the server, I assume we would return E_INVALIDARG and
>>> paste nothing?
>>>
>>> 4) Paste gets a bit complicated in the context of multiple selections,
>>> so the following sentence may need revision:
>>> "If both start and end offsets are equal to IA2_TEXT_OFFSET_SELECTION
>>> (value -3) then they point to start and end offset of active selection
>>> (i.e. start or end of selection has a caret)."
>>>
>>> If we truly mean that pasteText only pastes onto the selection that has
>>> the caret, then perhaps we can spec it as:*
>>> pasteText(IA2_TEXT_OFFSET_CARET, IA2_TEXT_OFFSET_SELECTION, mime_type)*
>>> rather than:*
>>> pasteText(IA2_TEXT_OFFSET_SELECTION, IA2_TEXT_OFFSET_SELECTION, mime_type)*
>>> This would make it clearer that it is the selection with the caret that
>>> will be operated on.
>>>
>>> However, it probably makes more sense to spec that the paste should
>>> happen on all selections. Consider the following semantics of clipboard
>>> operations on multiple selections in MS Word (which is not the defining
>>> app for multiple selection clipboard operations, however their semantics
>>> mostly make sense... and when I tried FF, I got odd behavior for
>>> everything except Copy).
>>>
>>> If we have this line of text:
>>> One fish two fish red fish blue fish
>>> Now select One two red blue by double-clicking while holding down Ctrl
>>> (I don't know how to do this with only the keyboard), then Copy, and the
>>> clipboard contains:
>>> One
>>> two
>>> red
>>> blue
>>> (I am not sure why line breaks are inserted between words... that is a
>>> bit odd. Multi-select copy in FF concatenates the words without line
>>> breaks).
>>>
>>> Start again with the original text, select One two red blue as before,
>>> and Cut, and we are left with:
>>> fish fish fish fish
>>> The clipboard contains the same data as for Copy.
>>>
>>> Start again with the original text, select One two red blue, and Paste
>>> "the ", and we get:
>>> the fish the fish the fish the fish
>>>
>>> Finally, start again with the original text, select One two red blue,
>>> and type "green", and we have:
>>> One fish two fish red fish green fish
>>>
>>> In other words, clipboard commands operate on all selections, but insert
>>> (typing) only operates on the selection that contains the caret.
>>>
>>> What I am getting at by all of this is that maybe pasteText should not
>>> have any offset parameters at all, because startOffset and endOffset are
>>> inadequate to define multiple selections).
>>> Perhaps pasteText should simply be specified to always operate on the
>>> selection, whatever that may be.
>>>
>>> And if that is the case, then it only makes sense to remove the
>>> parameters from copyText and cutText as well, for the same reason.
>>>
>>> Carolyn
>>>
>>>
>>> From:	Alexander Surkov _<surkov.alexander at gmail.com>_
>>> <mailto:surkov.alexander at gmail.com>
>>> To:	Brian Cragun _<cragun at us.ibm.com>_<mailto:cragun at us.ibm.com>
>>> Cc:	_accessibility-ia2-bounces at lists.linuxfoundation.org_
>>> <mailto:accessibility-ia2-bounces at lists.linuxfoundation.org>, IA2 List
>>> _<Accessibility-ia2 at lists.freestandards.org>_
>>> <mailto:Accessibility-ia2 at lists.freestandards.org>
>>> Date:	08/06/2011 06:45 AM
>>> Subject:	Re: [Accessibility-ia2] next changes to IAccessible2
>>> Sent by:	_accessibility-ia2-bounces at lists.linuxfoundation.org_
>>> <mailto:accessibility-ia2-bounces at lists.linuxfoundation.org>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>>
>>>
>>> Hi, Brian.
>>>
>>> I added this suggestion to wiki -_
>>> __https://wiki.mozilla.org/Accessibility/IA2_1.3#IAccessibleEditableText2_interface_.
>>>
>>> Thank you.
>>> Alex.
>>>
>>>
>>> On Tue, Jun 7, 2011 at 10:51 PM, Brian Cragun _<cragun at us.ibm.com>_
>>> <mailto:cragun at us.ibm.com>wrote:
>>>> I propose we add a way for IA2 Paste Text to provide both a Start and
>>> an End
>>>> Offset parameter.  Also to provide an Attributes parameter.
>>>> Copy-with-parameters should be implemented as an additional method in
>>>> IAccessibleText.
>>>>
>>>> See previous exchanges on this topic to the list:
>>>>
>>> _https://lists.linux-foundation.org/pipermail/accessibility-ia2/2010-September/001219.html_
>>> _https://lists.linux-foundation.org/pipermail/accessibility-ia2/2010-October/001223.html_
>>>> Previously this was not changed because it required a new interface.
>>>   Now we
>>>> are making new interfaces.  Good time to add.
>>>>
>>>> Regards,
>>>>
>>>> Brian
>>>>
>>>> Brian Cragun
>>>> IBM AbilityLab Consultant
>>>> Human Ability&  Accessibility Center
>>>> _www.ibm.com/able_&  w3.ibm.com/able
>>>> W:(720)-663-2801    H:(507)288-2437
>>>>
>>>>
>>>>
>>>>
>>>> From:        Pete Brunet _<pete at a11ysoft.com>_<mailto:pete at a11ysoft.com>
>>>> To:        IA2 List _<Accessibility-ia2 at lists.freestandards.org>_
>>> <mailto:Accessibility-ia2 at lists.freestandards.org>
>>>> Date:        06/06/2011 11:12 PM
>>>> Subject:        Re: [Accessibility-ia2] next changes to IAccessible2
>>>> Sent by:        _accessibility-ia2-bounces at lists.linuxfoundation.org_
>>> <mailto:accessibility-ia2-bounces at lists.linuxfoundation.org>
>>>> ________________________________
>>>>
>>>>
>>>> Hi all, Please take a look at this and provide your feedback:
>>>>
>>>> _https://wiki.mozilla.org/Accessibility/IA2_1.3_
>>>>
>>>> Thanks, Pete
>>>> --
>>>> Pete Brunet
>>>>
>>>> a11ysoft - Accessibility Architecture and Development
>>>> (512) 238-6967 (work), (512) 689-4155 (cell)
>>>> Skype: pete.brunet
>>>> IM: ptbrunet (AOL, Google), _ptbrunet at live.com_
>>> <mailto:ptbrunet at live.com>(MSN)
>>>> _http://www.a11ysoft.com/about/_
>>>> Ionosphere: WS4G
>>>>
>>>> On 3/11/2011 11:10 PM, Alexander Surkov wrote:
>>>> Hi, Jamie. I missed Mick suggestion on the list. It's sounds
>>> reasonable and
>>>> I agree we should try it before getting new API for this since the
>>> issue is
>>>> mostly about events.
>>>>
>>>> Thank you.
>>>> Alex.
>>>>
>>>>
>>>> On Sat, Mar 12, 2011 at 11:43 AM, James Teh _<jamie at nvaccess.org>_
>>> <mailto:jamie at nvaccess.org>wrote:
>>>> Hi.
>>>>
>>>> Nice work; good to get the discussion going. :)
>>>>
>>>> I still don't see a need for this registry API. Why not just use
>>>> IsWinEventHookInstalled(), as Mick suggested on the IA2 list?
>>>>
>>>> Thanks.
>>>>
>>>> Jamie
>>>>
>>>>
>>>> On 12/03/2011 3:48 AM, Alexander Surkov wrote:
>>>> Hi.
>>>>
>>>> I gathered ideas into one doc -
>>>> _https://wiki.mozilla.org/Accessibility/IA2_1.3_. Please give feedback
>>>> here and feel free to edit the wiki.
>>>>
>>>> Thank you.
>>>> Alex.
>>>>
>>>> --
>>>> James Teh
>>>> Vice President, Developer
>>>> NV Access Inc, ABN 61773362390
>>>> Email: _jamie at nvaccess.org_<mailto:jamie at nvaccess.org>
>>>> Web site: _http://www.nvaccess.org/_
>>>>
>>>> _______________________________________________
>>>> Accessibility-ia2 mailing list
>>>> _Accessibility-ia2 at lists.linuxfoundation.org_
>>> <mailto:Accessibility-ia2 at lists.linuxfoundation.org>
>>>> _https://lists.linux-foundation.org/mailman/listinfo/accessibility-ia2_
>>>>
>>>>
>>>> _______________________________________________
>>>> Accessibility-ia2 mailing list
>>>> _Accessibility-ia2 at lists.linuxfoundation.org_
>>> <mailto:Accessibility-ia2 at lists.linuxfoundation.org>
>>>> _https://lists.linux-foundation.org/mailman/listinfo/accessibility-ia2_
>>>>
>>>>
>>> _______________________________________________
>>> Accessibility-ia2 mailing list_
>>> __Accessibility-ia2 at lists.linuxfoundation.org_
>>> <mailto:Accessibility-ia2 at lists.linuxfoundation.org>_
>>> __https://lists.linux-foundation.org/mailman/listinfo/accessibility-ia2_
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Accessibility-ia2 mailing list
>>> _Accessibility-ia2 at lists.linuxfoundation.org_
>>> <mailto:Accessibility-ia2 at lists.linuxfoundation.org>
>>> _https://lists.linux-foundation.org/mailman/listinfo/accessibility-ia2_
>>>
>>>
>>> -- _
>>> _Oracle<http://www.oracle.com/>
>>> Peter Korn | Accessibility Principal
>>> Phone: _+1 650 5069522_<tel:+1%20650%205069522>
>>> 500 Oracle Parkway | Redwood City, CA 94065_
>>> _Green Oracle<http://www.oracle.com/commitment>Oracle is committed to
>>> developing practices and products that help protect the environment
>>>
>>>
>>>
>>> _______________________________________________
>>> Accessibility-ia2 mailing list
>>> Accessibility-ia2 at lists.linuxfoundation.org
>>> https://lists.linux-foundation.org/mailman/listinfo/accessibility-ia2
>> _______________________________________________
>> Accessibility-ia2 mailing list
>> Accessibility-ia2 at lists.linuxfoundation.org
>> https://lists.linux-foundation.org/mailman/listinfo/accessibility-ia2
>>
> _______________________________________________
> Accessibility-ia2 mailing list
> Accessibility-ia2 at lists.linuxfoundation.org
> https://lists.linux-foundation.org/mailman/listinfo/accessibility-ia2