Problem:

Any product can be more successful when developed through a process that identifies how it will be used, elicits input from potential users, and watches how it functions in real time with real users. Yet, many products fail because they are designed with little knowledge of user needs and capabilities.

GOAL:

The goal of this project was to choose a mobile app or website and conduct a semester long usability evaluation with recommendations for areas of improvement.  T he application my group and I settled on was Be My Eyes, a mobile application that helps blind users accomplish simple tasks. The application connects a sighted user to the video feed of a blind user’s smartphone and allows the sighted user to give directions to blind users.  We put the app through multiple phases of evaluation outlined below before making recommendations on ways to improve its functionality and service. 

CHALLENGES:

Even before our evaluation began my team and I expected some issues in finding active users of the app, both sighted and blind. After testing the app ourselves, we found that there were relatively long-waiting times for interactions. Both of these issues presented challenges that would need to be overcome. 

Process:

1. Identifying the Audience:
   It is important to identify a specific target audience in order to make a product polished and with focused functionality to cater to that audience as best as possible. We identified two core audiences of this application. Firstly, during the time of testing, the application was only available on iOS. Therefore, all users were expected to be relatively tech-savy iOS device owners. From this demographic, the target audience was then further split between blind users in need of help for simple tasks and sighted users willing to volunteer their time to assist the visually impaired. 
2. Competitive Analysis
   To evaluate the strengths and weaknesses of a product or service, it helps to know about how it compares to its competitors. For this competitive analysis I compared Be My Eyes to several mobile applications that used a smartphone camera to help a user identify object around them. The competitors that I evaluated were: VizWiz, TapTapSee, Cam Find, and Color ID
   
   Summary of Results:
   These apps were evaluated across several categories including accessibility, UI responsiveness, learnability, and availability on multiple platforms.Be My Eye’s strength lays in the detail and accuracy of its search results. Because the app utilizes live video, Blind users can ensure that they receive exactly what they need. Other competitors merely used photographs to interpret the needs of blind users.   Where the app falls short in comparison to its competitors is in the speed that results are returned. While other apps could return results in seconds, Be My Eyes took up to several minutes to receive a connection. This is an area that is severely needs to be improved. Users will be extremely dissatisfied if they need to constantly wait for long periods of time to receive help. This hurts the overall usefulness of the app and is the primary reason the app scored a relatively mediocre average rating of 2/5. Click below to view the full report.

     3. Heuristic Evaluation:

For this report a heuristic evaluation was performed on the app that tested it on many levels to assess its overall usability. Among the many things that the app does well are firstly, using common icons (such as a gear for “Settings”) to denote certain functionality. Secondly, the visibility of system status is also well integrated for blind users when attempting to connect to sighted users. However, there are a handful of usability problems that need to be addressed. the statistics banner on the main screen for blind users must be eliminated. The banner can easily be confused with a button and does not work well with OS voiceover functionality. Lastly, the ability to remove languages must be denoted clearly. As of this testing, a user must go to “add languages” in order to remove or change the default language which is completely unintuitive.

    4. Interviews, Personas, and Scenarios:

My group conducted several interviews across a wide demographic to better understand the target audience. These interviews helped in the formation of three personas and their respective scenarios.

    5. Interaction Map:

View the Interaction Map

    6. Surveys:

Surveying potential users of Be My Eyes presented our group with a enormous obstacle. What was the best way to reach out to blind and visually impaired individuals in a manner that would be appropriate. Fortunately there are many online forums dedicated to blind individuals. I posted our survey on popular forum site Reddit where we saw a spike in the number of blind respondents. View the survey here .
 

   7. Live Usability Testing:

We conducted a series of live usability tests recording both user body language and their interactions with the application. Doing so provide insight into ways to improve interaction within the application. 

ReSULTS and Takeaways

•  Sighted users thought the app was far more centered on visually impaired users, despite the fact that sighted volunteers play a valuable role in interacting with blind users

• Long wait times for call connection can be discouraging to blind users in need of immediate help

• Rare and infrequent calls for help to sighted users makes these users less invested in the service

• Sighted users complained of a lack of control over what times they could be called

Overall, this project gave me a lot of insight into usability testing and application evaluation post-distribution. We were also faced with many obstacles during this project (such being able a sufficient number of blind users) which we were able to overcome. I also gained much insight into the design process that goes into a successful application and these are processes that continue to inform my approaches to current projects. 

Tools used: Adobe Illustrator, MS Word, Google Drive for collaboration, Qualtrics Survey Generator, Voice Memos app