| 24 Nov 2020 |
 @gitter_zilch42:matrix.org | And i may have to learn how toTrain The Model On My Voice.It's not picking up everything from my australian accent. It's quick though! | 03:26:30 |
 @gitter_lexiconcode:matrix.org | It tries really hard to match an utterance to a command | 03:28:06 |
| @gitter_zilch42:matrix.org | I've got 2 weird things going on with the sleep rule. When the mic has been silent for a while, it doesnt wake up again. The status window does recognise commands and print output, but those commands aren't actually actioned (nothing happens). I have to restart caster to get it to do anything again. Secondly, I cant figure out how to manually put the mic to sleep. I've tried 'halt listening' but that isn't being recognized. | 05:07:24 |
| @gitter_zilch42:matrix.org | When it isn't doing anything, 'start listening' doesn't wake it up | 05:08:39 |
| @gitter_lexiconcode:matrix.org | Say 'caster on' | 05:53:49 |
| @gitter_lexiconcode:matrix.org | Caster sleep | 05:54:05 |
| @gitter_lexiconcode:matrix.org | Those commands now work for DNS/DPI as well. | 05:55:31 |
| @gitter_lexiconcode:matrix.org | You can chang the time out in the caster settings | 05:59:27 |
| 25 Nov 2020 |
|  @gitter_aschneiderman:matrix.org joined the room. | 01:43:06 |
| @gitter_aschneiderman:matrix.org | I want to create Caster commands that make it easier to work with Jira boards and product backlogs — eg, move a user story from To Do to In Progress, add a component to a user story. Jira used to have keyboard shortcuts for navigating a backlog & board column, but no more. Given that these items aren’t easily clickable by voice, any thoughts on strategy? I’m up for doing some nontrivial coding — eg using Selenium (w Python) — but figured I’d ask if there are easier approaches before diving in. | 01:43:06 |
| @gitter_lexiconcode:matrix.org | Greetings @aschneiderman Well there are a number of different techniques. You could use image recognition with Sikulix or you could use Selenium. Sikulix Is a bit fragile and doesn't integrate very well to grammars but it is functional. Selenium Is definitely doable and I've seen some grammars use it however requires a plug-in and some set up but it is also very doable. Possibly harder than the other 2 but I think a lot more generalizable is using accessibility APIs like UI automation from Windows to grab anything from most programs. pywinauto it is probably the best Python is the one best implementation. Accessibility Insights For Windows Allows you to inspect live applications to assist in leveraging UI automation. There are a few things missing for text navigation pywinauto but hopefully that can be resolved in future. | 02:45:33 |
| @gitter_lexiconcode:matrix.org | Here's an example from pywinauto save_from_firefox.py Automating the UI. I give you a point of reference you won't really need to emulate keys as you can do that to dragonfly. | 02:52:56 |
| @gitter_lexiconcode:matrix.org | | 02:53:00 |
| @gitter_lexiconcode:matrix.org | (edited) [](https://files.gitter.im/55dba9af0fc9f982beae7a63/TIUG/image.png) => Accessibility Insights For Windows to show the address bar
[](https://files.gitter.im/55dba9af0fc9f982beae7a63/TIUG/image.png) | 02:53:40 |
| @gitter_lexiconcode:matrix.org | Here's an example from pywinauto save_from_firefox.py Automating the UI. A point of reference as you won't really need to emulate keys as you can do that to dragonfly. | 03:15:16 |
 @gitter_tteig:matrix.org | tw | 03:19:35 |
| @gitter_aschneiderman:matrix.org | Thanks @LexiconCode! I’m using a Windows 10 box, so I can only use Jira via the web, so pywinauto and Accessibility Insights for Windows would give me a lot less semantic info than a tool like Selenium that can read elements of the DOM, right? And Sikulix would be helpful if I was clidking on static images, but it wouldnt work well with dynamic lists pf user stories. Or am I missing what you’re saying? | 06:03:45 |
| @gitter_aschneiderman:matrix.org | (edited) ... was clidking on static images, but it wouldnt work well with dynamic lists pf user ... => ... was clicking on static images, but it wouldn't work well with dynamic lists of user ... | 06:05:07 |
| @gitter_aschneiderman:matrix.org | Has anyone used Selenium with Caster or seen grammars that do? I did o | 06:31:11 |
| @gitter_aschneiderman:matrix.org | (edited) ... did o => ... did a bit of Googling and didn’t find exs, so if you know of one, can you post a link? I’m fluent in Python but haven’t written anything either with Caster or Selenium before, so it’d be great to reduce the time needed to figure out connecting the two. Thanks! | 06:33:28 |
| @gitter_lexiconcode:matrix.org | I'm pretty sure you can get access to anything on the Dom as it's used by screen readers in other projects. However that's more of an educated guess than actual experience. For your second request here you go https://github.com/wolfmanstout/dragonfly-commands/blob/master/_webdriver_utils.py | 07:14:28 |
| @gitter_lexiconcode:matrix.org | (edited) ... request here ... => ... request implemented in dragonfly here ... | 07:16:39 |
| @gitter_lexiconcode:matrix.org | That should give you enough to implement the driver and so forth. You might have to poke around a little bit more to find the grammar that utilizes the web driver. | 07:18:01 |
| @gitter_lexiconcode:matrix.org | Let me know how it goes as I'd love to see something like this pulled upstream into the project. | 07:22:03 |
| @gitter_aschneiderman:matrix.org | Thanks so much, @LexiconCode! | 11:34:09 |
| 26 Nov 2020 |
| @gitter_aschneiderman:matrix.org | I’m starting to experiment with craeting | 01:40:16 |
| @gitter_aschneiderman:matrix.org | (edited) I’m starting to experiment with craeting => Is there a page somewhere that has a list of links to different users’ custom grammars? I couldn’t see one in the docs. I’ve been using Vocola for years and am starting to switch to Caster, so it’d be useful to get a sense of how different users customize Caster. Thanks! | 01:43:45 |
| @gitter_lexiconcode:matrix.org | @aschneiderman You can reference the people on this channel and the following list dictation-toolbox.org. Note you're not limited to just Caster grammars/dragonfly is pretty much compatible inside the mappings as long as the grammars are separate for each file and you have the caster loading mechanism of get rule. | 03:22:44 |
| @gitter_aschneiderman:matrix.org | @LexiconCode Thanks! | 12:26:20 |
 @gitter_patricktouchette:matrix.org | @aschneiderman an idea that comes to mind, since I am working on something similar, is to create a chrome (firefox, etc.) extension which contains your dom manipulation javascript. Bind those functions to keyboard shortcuts, add a rule to caster and then bam, you can invoke those functions with your voice. Eg. "complete task" could find the dom element and click it for you or something along those lines. | 17:21:24 |
