This Week in Accessibility – December 4, 2023

The following transcript was taken by AI before edited by a human. If you notice any errors or have suggestions for improvements, please email delgadilloAccess@gmail.com

[Intro]

Segment 1

Des (VoiceOver): Today on the show, we love the accessibility features on Android, but hackers might love them more. Plus, we’re reacting to all the big news from this week in accessibility.

[Music Plays] 

Des: Welcome to the show. I’m Dez Delgadillo. 

Louis: And I’m Louis Do. 

Des: Let’s jump right in and talk about the news from this week. Google unveiled more about Project Guidelines, an innovative initiative that enables visually impaired individuals to navigate outdoor paths using just a Google Pixel phone and headphones. That sounds awesome. The project uses advanced machine learning to follow paths marked with painted lines. Google has now made Project Guidelines open source, allowing anyone to access and enhance it. Wow. Louis, are you going to start a running career? What’s your plan now that we’ve got access to Project Open Source? 

Louis: No running career. My legs already hurt at that statement. But I’m really looking forward to seeing what other developers do with this. Because one of the benefits to Android is the two words you mentioned, open source. People are able to take bits and pieces of Android to create some pretty innovative and cool apps. And for us as blind and visually impaired users, the marketplace has traditionally, or the market share, rather, has traditionally been with iOS. So it’s great that Android is seeing some love. And I’m really looking forward to seeing what folks will do with this new open source Project Guidelines. 

Des: I’m so ready. I’m an Android user. I’ve got an iPhone as well, but I use an Android in my day to day. And it is just going to be so much cooler to have an option. And OK, so thinking more about it, this might be a little like LIDAR. If anybody thinks I’m off the mark, you can email in and correct me if I’m wrong. But it sounds to me a little bit like what the new iPhones are doing. 

Louis: I would agree. I think that matches my assessment too. And I think what we’re seeing here that’s exciting, that I only kind of thought of now that you brought this up, is that both of these big tech companies, Apple and Google, with their mobile platforms, iOS for Apple and Android for Google, respectively, they’re both competing for the blind and low vision market share. And it’s really neat to see the different approaches that both companies are taking. 

Des: Yeah. Especially the fact that it’s open source so that anybody can tap into it. And speaking of open source, a new screen reader for the Mac could be on its way. Portuguese developer Jaio Santos announced on the Apple Viz forum that the idea is for the project to be open source, similar to NVDA on Windows. The working title for the project? Tell me what you think about this, Louis. Vosh, a combination of Vision and Macintosh. Do you like the name? It’s a working title, but Vosh. 

Louis: It’s a little on the nose, but I mean, it says what it is. So I’m not sure how fun you can get with these naming conventions. 

Des: We’re finally shrugging off the convention of naming our screen readers after sea animals. 

Louis: The downside that I see to this approach in the Apple ecosystem is that Apple is the opposite of Google, right? They are not open source. Open source is something that they’ve never really embraced. So it’s going to be interesting to see if Vosh can offer the capabilities and, frankly, the competition that VoiceOver can, because VoiceOver, since it’s a native screen reader built into the operating system, can access a lot more permissions and underlying stuff that Vosh can’t. And Apple, in the past, if it didn’t like its competition, it either bought out its competition or just created something of its own and just made its competition irrelevant. And I think that term is called Sherlocking. So it’s going to be, I think it comes down to, like it or not, a lot of it will come down to philosophy. So is Giles Santos and whoever else is working with him on Vosh targeting Apple at a time where they’re willing to change their philosophy and become a little more open? Or will Apple just decide that VoiceOver will reign supreme and be the only screen reader on the block? 

Des: Yeah, that’s a good term, Sherlocking. And that’s definitely what I expect to happen, personally. Apple has a really good reputation of just buying up their competition, like you said. So if there’s any sort of upward momentum for Vosh or whatever it ends up being called, I don’t know how well off this Giles Santos is, but I would hope that he is a little resistant. But history has bore it out that there’ll probably be some sort of acquisition coming up. Speaking of Apple, Apple released a short film on Thursday highlighting the personal voice feature for iOS. Now the video was created by Taika Waititi, and it centers around a furry creature trying to help a little girl find her voice. Now this video, I don’t know if you’ve gotten a chance to see it yet, Louis. 

Louis: Give us a summary. 

Des: So the video basically follows a little girl in the woods looking for her voice and a little nondescript furry creature is helping her find it. And they’re looking all over the place under the rocks and in a tree, and they can’t find her voice. And then at the end of the video, we have a heartwarming moment with the daughter and her dad, and he is reading her a bedtime story with the iPhone. So there’s an audio described version for those visually impaired listeners. I wanted to talk about this because it’s a really good example of a polarization, I guess is the word, that I’m feeling about this. On one end of it, I think it’s kind of icky that Apple is marketing using disability as a marketing tactic. But on the other hand, I feel like this is the only way that some people are exposed to disabilities. This is the only way they learn about it, e.g. representation. 

Louis: So just a point of clarification, the iPhone is supposed to be this device that she can use for her voice? They found her voice at the end and it’s on the iPhone? 

Des: Correct. Because she can, I guess, create her own voice via the iPhone. 

Louis: Okay. So one of the biggest things for people with disabilities is what you’re alluding to, right? The dignity of the subject being discussed. And of course, the opposite reaction is that icky reaction of like, ugh, this is belittling, this is whatever negative language you want to throw around it. But heartwarming stories or what people perceive as heartwarming hasn’t changed for all the times for as long as people with disabilities have railed against, like, don’t make videos like this. Videos like this is sometimes is what is accessible to people, right? We talk a lot about you and I, we work in our fields and we talk about digital accessibility a lot. But I think there’s a thing about conceptual accessibility, too. There’s a huge caveat here. And the caveat is that if the world is full of heartwarming disability stories, then the true disability stories aren’t being told, right? So there’s actually an opportunity here. Like Apple can create a series using this character, right? So this is, you know, the first video, as you described, there’s this creature, it helps her and like they find the iPhone and then, you know, she’s using the iPhone now and like she discovered her voice, right? But what’s next? Version 2.0 comes out. Let’s say, I don’t know, there’s a bunch of features. She should come back. They should have a second video of showing, oh, this person’s, you know, this character has grown up a little bit since she discovered her voice, this is how she’s using it. If there’s no connection to real life, to the way integration should be done, to the way people with disabilities want to be seen, then it’s just cheap marketing at that point. 

Des: Yeah, that was incredibly well put. I couldn’t say it any better myself, a very well educated point. And I think I know why, because this next story is actually about your alma mater. UCLA has started to offer a major in disability studies, making it the first university to do that in the state of California. Before, even though state universities here were key in the disability rights movement and helped shape important laws, they didn’t offer a full major in disability studies. Universities like UC Berkeley and San Francisco State were into disability studies early on, but they only had minors, not full majors. So other states like Illinois, Wisconsin and Ohio already have the disability studies programs, but California’s top schools have been slow to fully embrace this field and hire disabled faculty. Now, the cool thing about this is not only are they building disability studies into their curriculum, they’re also going to be hiring tenured faculty who is disabled and can obviously help better educate whoever comes into this. So what do you think, Lewis? Are you going to go back to school for this? 

Louis: No plans on going back to school, but the fact that somebody can declare a disabilities major and put it on their resume and walk across that stage after four years of college and have disability be recognized as a serious, worthwhile subject for discussion and education and innovation, that’s huge. It’s important because it’s showing that the university, the leadership behind the university cares enough to make this a major, that the students are finally being heard. And I know for folks with disabilities, being heard is sometimes the hardest step to get others to reach with you. So I think this is wonderful. I think this is a great step, a right step. And I’m very curious to see where this goes. The class of the first graduating class will be making history. 

Des: Yeah, and I’m really excited also to see what other institutions do in response to this. Obviously, UCLA is an incredibly influential place in terms of higher education. So I think the fact that they’re doing this, we’re going to see a lot of other schools doing this, not just in California, hopefully, but all over the place. But enough about education. Let’s talk about play time. In the world of video games, 2023 has been an amazing year for accessibility. If you’re a blind player, we’ve gotten Mortal Kombat 1, Forza Motorsports, the remastered The Last of Us, and not to mention products like the PlayStation’s Project Leonardo, which is an accessible controller with endless remapping options for players of varying abilities. So this week, Ubisoft announced some of the accessibility features for the new game Avatar: Frontiers of Pandora, which comes out December 7th. CanIPlayThat.com has a great breakdown for all the features in the game. But among some of the ones that really stood out to me, customizable controls, customizable UI colors, menu narration, and a speaker direction indicator. So not sure if this game is going to be fully playable for blind people or anything like that. I don’t think it’s going to be as comprehensive as, say, Forza or Mortal Kombat 1. But just the fact that the company themselves are investing in these accessibility options, I think is awesome. Ubisoft is, of course, a huge game manufacturer. Avatar is a huge franchise. I can’t stop hearing about those damn blue people anywhere I go. So they’re everywhere. Louis, are you going to fire up the PlayStation 5? What are you going to be doing this Christmas? 

Louis: I am finding… I just got myself an Xbox controller and I’ve been connecting it to my PC and playing games that way. So you don’t technically… Well, I shouldn’t say that because I’m not a technical programmer, but from what I can tell, you don’t always technically need to have a console anymore to play, which if you don’t need a big screen or whatever on a TV and you just have a laptop, the way I do, makes it even more accessible. So I’m finding the way hardware and games are being developed, it’s becoming a very interesting frontier for folks who are finding what we do. 

Des: Yeah. The other thing I want to add to the gaming thing is also if you are playing on the PC, The Last of Us Remaster is on the PC. You do need a certain graphics card and GPU and all that jazz. So you can check all that out on the website if your website is compatible. Mine sadly is not. I have a sneaking suspicion, Louis, you being the technical person that you are, yours probably is. 

Louis: Besides the games mentioned, we’ll also add a couple other games that we have found to be accessible in the show notes as well. 

Des: Right on, that’s all the news for right now. Let’s take a quick break and when we come back, we’re going to talk about Android accessibility and how hackers can use it to hijack your devices.

[Music]

Segment 2

Des: All right, Louis, for our first big segment of the week, I thought it would be fun to talk about Android accessibility services, more specifically, how hackers are hijacking Android security services to get into people’s devices. So when you think about Android accessibility services, you think of features like TalkBack, Voice Control or Navigation Assistance, things like that. But the same API that can make our phones talk to us or let us control them with our voices is also a weak spot in every Android device’s security. Do you feel safe when you use your Android? Louis, I just wanted to start there. 

Louis: I use apps and websites that I trust and the way I determine that is I try to only download apps from the Google Play Store. However, that’s, of course, not always possible. 

Des: Right, yeah, it can be pretty difficult and we’re going to talk a little bit more about how Google has kind of tried to cut this problem down, but it’s still a problem. Let’s talk about what the problem looked like before Google took those measures. So Bleeping Computer wrote this back in 2017 and I think it’s still pretty relevant today. They said the API works by allowing an application programmatic access over actions that in normal circumstance require physical interaction. For example, the accessibility service can mimic taps and swipes on UI elements to navigate users through various screens. Sounds a lot like TalkBack and the other apps that we use on a day-to-day basis. 

Louis: Sounds exactly like them, or their principles at least. 

Des: The article goes on to say this is a very powerful feature, one that malware authors also noticed and incorporated into their malicious apps. For years, these malicious apps have relied on tricking users into granting them access to the accessibility service. Once they gained such access, it was game over, as this allowed the malware to install itself as device admin, download and install other malware, and execute various operations in the phone’s background. That is no bueno, Louis. 

Louis: And you know, it’s scary as a screen reader user. How many times have we encountered dialogues where we’re just like, I am clicking okay, I don’t know why I couldn’t read it, but I think this is the right, this feels right, I hope this is right. 

Des: Yeah, because you just want to get to your task, right? That’s the whole point of using the phone is like, you’re just going to click through because you want to get to the task that you’re trying to achieve. 

Louis: And the question that I think screen reader users always ask, and they ask it with more frequency if they are older, someone new to screen readers, you know, a multitude of different situations and scenarios. And the question that folks ask a lot is, is it me? Is it my phone slash device? Or is it my AT? Is it my screen reader? 

Des: Yeah, because it’s really hard to find centralized documentation, especially for Android. For the iOS world, you have places like AppleVis, at least from a screen reading perspective. It’s really hard to find, I mean, accessible Android is probably the closest I could think of to a similar website. But you know, they’re only like one or two people. So you’re not always going to get the most up to date documentation. 

Louis: And we’ll get into this more as the conversation progresses. But with Apple, people on the Apple side of things, they live in a walled garden in a supervised sandbox, where security is already a priority. So they don’t necessarily have to worry about the security vulnerabilities that folks who use Android have to worry about. 

Des: Let’s learn a little bit more about what what kinds of things they’re doing here. There are a lot of different ways the accessibility services can be abused by attackers. One of the scariest is called click jacking. So in my research for this, an example I saw used a lot was Rick and Morty game from 2016. Now I love Rick and Morty, I didn’t get a chance to play this at the time, but it looks like a really fun game. You have all the characters from the show and they’re popping up, and it’s your job to whack them with a hammer. I mean, if that doesn’t sound like fun, I don’t know what does. But the game was anything but. Each time the users were whacking a character in the game performing any sort of gesture, the game was actually passing through clicks to an invisible permission screen. So by the time users got bored of the game, the app had taken total control of their phones using the accessibility services, leaving all of the information on their devices up for grabs. Now, like I said, this came out in 2016. This was freely available on the Google Play Store. 

Louis: The timeline is something that concerns me because it’s, you know, 2016 or 2017 to like 2022 or whenever they fixed it to like 2023. You know what I mean? I don’t know where we are. 

Des: They fixed it in 2021, November of 2021. 

Louis: But yeah, there’s like, there’s a lot of years in between those things. So it’s like, I guess the point that I’m trying to make is, like, are these, these are priorities that impact us as accessibility users, but are they being viewed as such? Or are they being reacted to? 

Des: Yeah. And I will just add, I left it out of the story just for brevity, but Google actually did try and address these issues. So in 2017, Google tried to block apps that were using the accessibility API, but a lot of developers who were using that API for harmless reasons were also going to get penalized. So although the actual ban went into effect in 2017, Google quietly walked that one back And it wasn’t until 2021 that they came back and actually re-implemented their procedures. I think they took the, over the interim, they were developing these new procedures and, you know, everything takes really long in tech. I’m not trying to play like Google’s answer man or anything, but that’s, that’s what I saw in regards to the timeline. Okay. So like I said, in 2021, they decided that they weren’t going to allow people to use their apps anymore for accessibility purposes, without written lists of what they planned on using the API for. And they went ahead and took it a step further and actually removed any apps from Google play that did not provide a written list of, of those reasons. So as far as Google play is concerned, I would argue as of 2021, they’ve done a pretty good job of cutting down on these accessibility attacks. Would you agree with that? 

Louis: I would say so. 

Des: But as much as Google would like you to think so, the Android ecosystem doesn’t exactly stop with Google in their play store, whether it’s third party app stores, like let’s just say APK mirror, for example, or links on social media. There’s still plenty of ways to get APKs out there that are outside of the Google sphere, which means there are still plenty of places for hackers to try and hijack your accessibility permissions. I know you’re not using Android all the time, Louis. Are you one to try and site load an app every now and again? 

Louis: Yeah, I am. One APK I can think of recently is I tried to download the Eloquence synthesizer for Android. 

Des: Oh, yeah. 

Louis: Which is not available in the latest version. It hasn’t been available for the last couple of versions, but I only recently got an Android phone this year in 2023. So I tried to site load that, and for various reasons it didn’t work. So that’s an example that comes strictly to mind, and it’s one of the examples that I think is relevant to the conversation at hand, because the synthesizer, we’re using it for, say, using TalkBack and things like that. But it may not be the direct accessibility service, but you still would need a synthesizer, and if that gets flagged under Google’s rules, then that would be a detriment for us. 

Des: Yeah. Yeah, no, that wouldn’t be good at all. So okay. When I started researching this topic, I figured this wasn’t much of an issue in 2023. After all, only smart people use Android, right? Well, here’s how they can get the smartest of us today without ever touching the Play Store. So you’ve just received a text from your bank. They updated their app, and they wanted to send it to you so you can get logged in and give them feedback. After all, you’re such a valued user, they wanted to send it to you first. Besides a little bit of flattery, most people wouldn’t think twice about this. After all, our banks have our phone numbers and whatever else they could ever want. So the fact that they’re using our phone numbers to contact us, it’s not gonna raise any alarms. This exact type of scam is what started happening in Peru in April of this year. For those who thought they were downloading the app they use to pay their taxes or get the refund or whatever they do in Peru, I’m not sure. They were actually downloading the Zanubis Trojan virus. According to a Kapersky analysis published at the end of September 2023, that’s just two months ago, the Xanubis malware is then programmed to remain dormant until the user opens any of the banking apps that are on a predetermined list in the Xanubis program. As soon as those apps open, the malware will either start locking inputs or recording the screen. As soon as they have access, they can take all of your information and start using it for whatever nefarious purposes they would like. Does that scare you or what? 

Louis: I’m speechless. 

Des:Yeah, it’s not safe out there. The thing that I worry about is you can try and lock these things down, right? You can try and make things safer for people if you’re Google. But that kind of goes in the face of at least what I see as the unique selling proposition of Android, right? As you’re able to get in there and tinker, you’re able to do things that you’re not able to do in the Apple ecosystem. 

Louis: I have to play devil’s advocate here. We’re talking about safety and security and freedom. Concepts that on the face of it, they look like opposing concepts. And Android, I’ve heard Android be described as the wild west. That’s the wild west of technology. You don’t know what’s going on. It’s the frontier. It’s Androids and so many other products, too, aside from phones, right? It’s on Androids on microwaves, fridges, washing machines, all that kind of stuff. And if we’re talking about freedom and the wild west, then should we be expecting, should we have been expecting that it was only a matter of time before accessibility services get targeted? Because one of the conversations we had before the show was that there are people out there whose jobs are dedicated to creating malware. 

Des: You’re right. Absolutely. There are people whose jobs it is to just create ways to scam you and not just like look at the Rick and Morty example from earlier. That was a creative game with fun graphics that match the show. People aren’t going to think twice about something like that. So it’s really, really on us to be super vigilant. But I also think a big proponent of this in a way to solve this issue in the long term when Google did a really good job of sort of plugging the hole back in 2021. But like you said, Android is the wild west in the sense that you’re still able to go in and have more granular control than you would on an Apple device. And a big way to solve for that security flaw or a security issue that exists is education. We talked about the whole UCLA thing at the beginning of the show and you know there’s going to be like a security component to that curriculum. But you’ve also got to think about more short term solutions and ways that we can get people today who aren’t necessarily pursuing higher education to be able to understand that their phones are susceptible to this backdoor attack that people can use. Because you’re not going to get rid of accessibility API. That’s not something you’re going to do, right? 

Louis: You can’t because it’s how people are able to hook into the system in the first place. And I think the dilemma here is how do you build safeguards into the system itself, into the software itself that people are learning and reacting and being proactive within the environment rather than having to take time and sit down and watch a five minute video on security when they have jobs and kids and families and things to take care of. 

Des: Exactly. Good foreshadowing there. You know, there are a few security apps out there right now that you can download to monitor your phone for malware. I personally haven’t installed one yet, but after doing all of this research, I think I’m going to do that this weekend. Google Play Protect is definitely the most popular option. And as long as you’re using Google Play services exclusively, you’ve opted into those features. But that doesn’t cover any of the other apps you’re putting on your phone via APK Mirror or other less reputable places. Basically, if you think of an antivirus program, chances are it has an Android equivalent. Bitdefender, Avast, AVG, they’re all on there and they all have a degree of security that they can apply to your apps, including apps that you site load. Meanwhile, on the development side, this is kind of what you were talking about. Developers have the option of using a secure service SDK like AppDome, which builds itself as a no-code solution to help build security protocols into apps. And that includes safeguards for accessibility APIs. So the way I kind of see AppDome is, A, it’s on the development side, so this isn’t the user’s prerogative, but it’s just adding a secondary layer of security, right? 

Louis: Yeah, and just detecting these malicious actors and preventing them from gaining access and getting more, like getting a grip on people’s systems. 

Des: Yeah, it’s a tough time. And it is truly the Wild West. It’s back and forth between bad actors and companies just trying to keep their OS safe. And the users, well, we’re just the ones caught in the crossfire. And although this is not a combat sports website, we’ll keep you up on this particular fight right here on DelgadilloAccess.com. We’re going to take a quick break. And when we come back, we’re going to talk about our big highlights for the week in accessibility.

[Music] 

Segment 3

Des: All right, before we get out of here, we’re each going to name one thing from this week in accessibility that really stood out to us. Lewis, I’m going to let you go first. 

Louis: My pick is called Face in View. And it’s designed for when somebody is trying to figure out if they are in frame for either a virtual meeting or other video or camera purposes. It’s a small feature, but it works really well. I didn’t think I would ever use it, but then I moved and my work setup changed and my lighting conditions and all that changed. And it was great being able to fire up Face in View, know and get some feedback, and then know that I am looking good for my meeting, that my face is, of course, in view, and then going on about my day. I’ve never used it since, but the fact that when I needed it, it performed flawlessly was great. 

Des: Yeah, I’ve been using it pretty regularly, actually. And it really has helped my confidence of being on camera, making sure I’m looking in the right direction. I think people appreciate that. And just knowing that I’m not completely off to the side or in a direction that people can’t see, it’s really been helpful. I’ve been actually doing more screencasts and stuff like that, trying to get myself on camera a little bit more, and it’s been super useful. In fact, I learned for the very first time that my camera is actually on the left side of the screen. This whole time, I’ve had this laptop now for five years, this whole time, I thought the camera was in the center of the screen. So I’ve been completely just showing people my left cheek for the last five years of video calls. So thanks to face in view, I figured that out. And that’s just my little story about that. 

Louis: The word confidence that Des used is great because for a lot of people listening, you may think, oh, this is not a feature worth talking about. You may think that it’s just as easy to use face in view as it is to ask someone for help. Well, not everybody asks for help or is capable of doing it. Asking for help from some people is the hardest step for them. 

Des: My pick actually kind of goes hand in hand with that. You can ask AI for a little bit of an audio description, or at least here’s the beginnings of that prospect. The idea place did a great post highlighting a combination of using FF MPEG. I don’t know if I’m saying that correctly. I use it, but I’ve never had to say it before. FF MPEG and Be My AI to describe pictures from a video. So they actually take the frames of a video, turn each of those frames into a picture, and then generate descriptions for each of those pictures. So it’s definitely not replacing audio description, but it’s really showing you how the future can look with AI in the mix to create audio described multimedia. 

Louis: The more tools we have at our disposal that allows us to be agents in our own digital journeys and be creators in our own digital journeys in an accessible way, of course, the more of that we get, I think the better off we’ll be. And I’m quite the fan of what we’re seeing happening here. 

Des: I also like with this particular process, they can take it a step further. You can even create the voiceover using 11 Labs. So I saw somebody take photos of themselves, have it fed through Open AI or Be My AI, and then create a voiceover in the style of David Attenborough. And all of a sudden, you have your own nature documentary about yourself. It’s just some really fun use cases for what’s going to end up being a very serious technology. So I’m ready for it. I’m here for it. And we are done for the first episode. Thanks so much for listening. If you have any questions, any feedback that you want to send our way, the email address is DelgadilloAccess at gmail.com. It was available, so I took it. You can also look at the blog, delgadilloaccess.com slash blog. We’re going to have blog content, tutorials, and more fun stuff that I am putting together for you to enjoy. So thanks for listening. We’ll talk to you next time. Say goodbye to the good people, Louis. 

Louis: Stay classy. 

Des (voiceover): And the only thing left to do is credits. Research was by myself and Louis Do. Our intro music is Rollin’ at Five by Kevin MacLeod. You can find more of his recordings at incompetech.com. The statements and views expressed in this podcast are personal and do not reflect the policies of our employers, partners, or any associated third-party entities. Thanks for listening. Talk to you next week. Bye.

[Music]