Keep: App Research - Insights & Feedback


This document is a summary of member research & feedback meant to inform opportunities to improve the app experience and explore ideas to increase the value of Keep's overall value prop and offerings.

The purpose and approach of the research was to understand:

    What is missing from the current experience (unmet needs & challenges), that would make the app more useful?
    What should the future app experience (and overall set of services) be and what things does it need to do to meet the needs of members to make it more valuable?
    What other ways might people want to be able to interact with KEEP?


Approach

Step 1
Step 2
Step 3
Collect feedback from users on the current experience and use this to inform the development and testing different working prototype concepts (using Bolt)
Present different members with different experiences / concepts in an unguided way in order to inform overall approach as well as desire for specific features.
Test and Refine on UX to arrive at an experience and set of features that members felt would make the app and overall service more valuable as well inform future ideas to explore.


Key Takeaways


** If there was a single important takeaway it was that users want the ability to both understand their compliance / adherence over time and be able to easily take action on it by providing the information they need to do so - this has driven much of the direction of the research and overall direction of the app.

Key Takeaway 1
Key Takeaway 2
Key Takeaway 3

Move from usable to useful

Increase ease of use of core actions

Serve the main use cases

While functional, the app needs to provide a level of value that would give members a reason to rely on it daily and weekly.
Increase efficiency of frequent actions such as marking doses as taken, or modifying med schedules.
Focus on serving the 1 user, 1 device, 1 med schedule use case in order to simplify the experience.

Move from usable to useful


The Key Takeaway from feedback was that while the app is usable - in other words it provides the basic functions, it lacks key things that would make it more useful and valuable.

This would include:

  • A cleaner, less cluttered home screen to make it easier to scan and understand the status of their doses, including more robust categorization and intuitive information hierarchy.
  • A home (schedule) screen oriented around a "Today" / "Weekly" experience (vs. a feed) that aligns better with their daily / weekly routines.
  • Helping members understand not only what they need to do, but also how they are doing by providing key adherence information and insights into their routines.
  • Creating a simpler to understand Medication section with better information organization, including making it easier to modify schedules (a key pain point).
  • A dedicated Circle of Care section to make it easier for members to find and interact with a key feature and value prop of KEEP.

** This includes removing the current toggle system which confuses members and doesn't serve value to the main use case.

Increase ease of use of core actions


While there is definitely more value to be created in creating a better app experience, because members are continuing to struggle with core interactions, it means that they experience friction that could lead them to abandon using the app as intended.

This includes:

  • Reducing clicks and increase ease to mark doses as taken.
  • Making functionality such as modifying schedules more intuitive.
  • Making the add medications process more guided and streamlined (suggest the use of default schedules based on med requirements and common daily routines).

"If there were somehow a way ... to interact with the app indirectly while marking a medication as “taken” without directly needing to go into the app each time (*which I think is why I ultimately stopped using the app quite so much, because it got to be a hassle going into the app multiple times a day to track each of my medications when I took them). That would be a surefire way for ME to interact with the app more consistently than I have been!"

Serve the main use cases


While it's important to be scalable, the experience should focus on the main use case of 1 device, 1 med schedule, 1 user (even in the case where there is a user and a dependent - it's essentially still a "single schedule" experience).
This includes:
  • Removing functionality such as the toggle system.
  • Implementing the new add medication flow in onboarding to streamline interactions and reduce complexity.
Note: Newer interactions have been designed to better accommodate multiple user / schedules in the same view.


Prototype Concept Testing



Members were presented with different home concepts (above left to right) that were then iterated on based on feedback.

The main concepts explored were:

  • (v1 / Left): Desire for simpler card based system(s) to add focus, create more clear actionable UX system and reduce clutter.
  • (v2 / Middle): Categorization of dose schedules as well as introduction of insights and chat as a way to create / modify medication schedules.
  • (v3 / Right): Experience oriented around "Today" and "Weekly" views along with new Stats, a cleaner Medication section and more robust Circle of Care feature.

Included in the study was:

  • Summary adherence stats and more advanced insights to understand what data members want to see about themselves.
  • Providing more robust medication section to make it easier to modify medication (a key challenge) schedules.
  • Providing a new dedicated Circle of Care section to make it easier for people to find and interact with this key feature.
  • A new chat based system that would allow members to interact with KEEP in a more conversational way (including adding medications).
  • New navigation system(s) to make it easier to understand and find information.


Version 1: Card View



The first prototype concept used a card system. The idea behind this was to allow people to present members with what "now" or "upcoming" doses, simplifying the home screen and letting them focus on the most immediate actions.


Key Learnings:



Card View:


While the card system had value in it's simplicity, and members liked the idea of a card based system, some didn't understand how it would work as they had more meds and more doses.

"I’d be curious to know how that would work with multiple meds that have to be taken at the same time - will it give me multiple boxes, groups the names, or just pick one?
I do feel that textbox takes up a lot of space. "


Status:


This version seemed to lack the clarity and peace of mind in terms of status that some rely on for pill boxes and blister packs. It's not only important for them to understand what they need to do now, but also what they have already done.

"I like the added peace of mind (of organizers): by filling the organizer, I visually see exactly what I’ve taken."


Version 2: Categories View, Insights & Updated Nav




In version 2, the feed was replaced by "Today" and "Full Schedule" views. The single card system also changed to more of a "focus" based multi-card system to make the home screen more scannable.

This included:

  • Navigation labels and different icons to more clearly represent the sections.
  • A more robust hierarchy of categories including a "Due Now" / "Overdue" top focus card that would allow members to see what actions needed to be taken right now.
  • More "states" to dose entries so that members could easily scan their schedule.
  • An "insights" section to understand what kind of information members might want to see.
  • A chat feature that people could use to create medication schedules.
  • A dedicated Circle of Care section to would bring this feature front and center.

** The device section and open / close was purposefully left off to get people's reactions to what they thought should be there and how and why they use it.


Key Takeaways:


  • Overall this version of the app was much closer to a possible new experience and provided a direction to the new home screen.
  • "How am I doing" became the #1 question people were seeking to answer as that over time is what they want to know - am I on track or not?
  • Chat was interesting to people and they liked the idea, but it was clear more testing needed to be done, especially if chat were to be used for critical things like adding or changing medication schedules.


Updated Navigation


In general members liked the navigation as well as responded to new features that are not in the current app very positively. The navigation also followed a layout that helped convey more of a clear flow for the experience which members found useful.


The goal of the new nav bar was two fold:

    Provide more clear icons and labels so people could understand what was in each section and make easier to navigate.
    Introduce new sections and information that might be prove more useful and increase the likelihood of engagement.

"The new “menu bar” at the bottom is much more accessible and the list options are nice (*as opposed to just icons as they are currently)."

"I found the app to be very easy to use and understand. The flow through different screens made sense to me as well."

Next Steps:
While an insights section is extremely useful, this test informed a more polished navigation for the next version which moved the insights into the home screen and added back a settings tab in the nav.


New Today / Full Schedule Views



The current app view and feed makes it challenging for people to not only manage doses as taken but more importantly to understand the overall status of their schedule.

On the most basic level it turns out there is still challenges in taking doses with the current UX. This means that people might not be always using the app as intended and / or that their actual compliance or adherence might be affected by this.

Adding focus cards and more clear and present action buttons (as well as future ideas of marking doses as taken via push and sms), could increase the likelihood of accuracy and use.

I have not used the app in a while because of the hassle of marking each medication as “taken” when I know I’m diligent about not skipping my meds."

Across the board members preferred the new views as it was easier to read and the horizontal layout made the full schedule easier to navigate.

"Love the access to the full schedule and how it looks. Definitely easier to read vs. currently. "

Having doses both organized by category and by status helped people understand where to focus and how to understand how they were doing in terms of late or missed doses.

"The reminders of “overdue doses” and “upcoming medications“ are handy, as is the ability to look week by week by scrolling side-to-side as opposed to up and down like it is now."

Next Steps:
  • The categorization, color coding of statuses and providing different views drove the overall direction of the next iteration of the app and the direction that the app design should take.
  • New updated categories to the next version as well as additional information to "Taken" statuses to reflect actual times taken.


Medications View



While some actions and information were still missing, especially the ability to modify schedules, generally members liked the cleaner design that made the section more clear to understand due to color coding and layout of information.

"I love this view. It’s simple and very clear. Tells me all I need to know and makes it easy to change anything if need be."

Being able to look into the past proves useful in people confirming that they had in fact taken medications or in certain cases skipped others.
This creates a level of reassurance that KEEP is in fact keeping track of this and providing ways to easily reference this - which was the intention of the Weekly view.

"For past days, it would be great to have a confirmation of meds taken (on time, late) or skipped on there (should be editable though in case the app makes a mistake)."

Medication schedules are not simply about dose times but about routines, and some members eluded to this in terms of how they think about their day. A few are even self categorizing (which actually means they are creating multiple schedules for the same medication).

"One way I used my keep right now if by just creating a "morning meds" category and "night meds" category because there's too many to list and check off. When I went to add a medication i noticed it searches names. Is there a way to create your own name? I think this is especially important for rare diseases or people who take too many to list."

Next Steps:
    Adding categorization, status and today / weekly views seems key to people using the app more often and better managing their medications with Keep.
    Modifying schedules was a key pain point of the current app and was something users felt they wanted to see in a more final version.


Insights Section



Members indicated that while some of this wasn't always clear, overall providing information was very useful in helping them better understand how they were doing and for some could help them know whether or not they needed to modify their schedules.

"This is an awesome treasure trove - so much info here."

"I like the idea of the “Doses Taken” tab and “Adherence tracker” tab in the “Progress Overview” section, and the stats for each individual medication."

"I really like the Insights page. I found it super helpful to be able to look back at the overall compliance rate and think that would be really helpful for both patients and medical care teams. I like the little blurbs at the bottom that give you a big pictures and congratulate you on your successes."

Providing the right data at the right time in the right way is also important so that it doesn't feel overwhelming. This feedback informed the next version to contain more summary information.

"When you’re on the day-by-day page, will the insights and medication be adapted to the day itself? If not, I wouldn’t repeat, might cause confusion."

Next Steps:
While a robust insights section was interesting, more testing would be needed to create a dedicated section. However, adding a few stats on the home screen is definitely a value add.


Circle of Care




Circle of Care was added in this version to test whether people knew it existed as a service or what they thought of it, and to design it in a way that highlighted different key actions.

Circle of Care had been fairly hidden from users in the past during onboarding and was hard to easily find and engage with this key feature. For some people, this was even the first time they even knew it existed.

Even people who didn't need or want support, understood the value, so it is something we should highly consider adding to the main navigation, especially if we want to drive more engagement and use out of CoC.

"I also like the idea of the “Circle of Care,” and notifying caregivers in the event of a missed medication dose. I don’t think that I would need to use it, because I am so used to taking my medication at a certain time every day, so I’m not sure how helpful that would be for me."

Members even suggested features to expand the CoC to for members, such as using it to send alerts based on patterns so that CoC could become a more robust intervention service.

"For the Circle of Care, have you thought about alerting the Circle of Care member if there's a pattern of missed doses vs a single missed dose? Say maybe someone misses 3 doses in a week (or whatever time frame) and then the alert would be sent? I think giving the option might be a good idea for someone like me who doesn't want anyone on my back all the time but would like some accountability. "

Next Steps:
This is a key feature and value prop on our website and landing pages, which also drives adherence.
  • While some design updates still need to be made, this should be included in the main nav as a section in a more final version of the future app.
  • Develop and test future prototype iterations that would include more robust interventions and alerts based on adherence trends.

Chat Feature (Add Medications)



The chat feature tested people's desire not only generally about being able to interact with KEEP via chat, but also as a more guided way to perform key actions such as create and manage medication schedules.

While people overall liked the idea, and were posi it wasn't completely clear if people wanted to do things such as create or manage schedules with it.

We should however keep testing different chat concept within the app, including more support type content to see what sticks.

Next Steps:
  • Design and test different chat interactions and content.
  • Update the create schedule feature from the new onboarding to make it more accurate to how it might work and re-test.


Version 3: Overall Refined Design, Native App Build, "Today" and "Weekly" Based Home Section




For this iteration, previous feedback was incorporated into what could be a more final version, as well as building this as a native app and orienting the main home around the new Today / Weekly views.

The goal main of this was to increase average daily and weekly usage rates by giving people a reason to check the app by providing value beyond simply helping them "take" their medications.

This build is also more in line with a KEEP look while making the overall look more clean.

Next Steps:
  • Roadmap releases and plan for an app refresh to the current feed and home scree,
  • Follow this with starting to redesign the home screen to the new Today / Weekly view(s) and modify medications feature
  • Build "Add Medications" into this version to test how this would work.
  • Update CoC to include updated features from past iterations as well as current features (types of notifications and subscribing to specific dose times).
  • Continue to test newer iterations.
  • Continue testing chat.

This included:
  • A more refined "Today" and "Weekly" view(s) structure with toggles and a more refined weekly schedule
  • A focus card for "Due Now" category with functionality that allowed for multiple doses / medications.
  • Adding details to medication cards such as "time taken" to improve understanding of dose compliance.
  • More finalized category structure and information hierarchy to make the feeds more scannable.
  • Weekly summary stats on the home screen. These could eventually be interactive but for now just having this information as a baseline would prove to be useful.
  • The ability to "modify" schedule on medications screen to show how this would work in a simple way (most don't want to have to go through the whole add process again, they want to update frequency, time
  • An updated nav structure.


Home: "Today" and "Weekly" + Stats


This was rebuilt around the key feedback of previous rounds of making the app better help members not simply see a feed or take medications, but better track medications over time.

  • The current feed was replaced with this new structure.
  • Summary stats were added with the eventual goal of expanding this into something more robust.
  • A new Open / Close feature replaced the current device section to make it quicker for people to engage.
  • The existing toggle system was removed as this took up space, didn't provide enough value to customers and was confusing to use (in fact none of them noticed it was missing from all the prototypes).

Next Steps:
  • Roadmap and plan releases toward the new Today / Weekly view with new categories.


Medications View




The main goal was to design a system that made it easier for people to modify their schedules (vs. what they do now which is delete / archive schedules and add new ones).

Next Steps:
  • Update the prototype with "Add Medication" feature currently in the onboarding (Chaz).
  • Plan for build and implementation of the new Modify Schedule feature.


Circle of Care



The next version of circle of care reflects more closely what a releasable version could be including incorporating functionality that exists currently in the new onboarding and app (subscribing to specific meds and dose times).

Making this important feature more prominent would increase the chances of its usage and allow us to start building in more robust features to increase the overall value of this service to members.


Settings / Profile



The settings section has been the least tested but there are some new items and updates that would be important to further test and are shown above including:
  • More robust profile section, including being able to access profile from the upper left image on the home screen. This could open up the potential to make the profile more robust.
  • More robust notifications section including organizing them into times of day to align with feedback on how people think about their daily routines (morning, noon, night vs. specific times). *** we should also incorporate this concept into medication schedules as well.
  • Privacy & Security settings - especially when it comes to data people want to be able to have control over their data and how it's used.
  • More easy to understand layout and a single drill down interaction (such that we don't have too many nested navigations in the settings section).



Future Considerations:


  • Enhanced insights & visualization for adherence and compliance trends as well as in situ recommendations in real time for optimizations.
  • SMS pattern based interventions (late, skipped) so we can begin to create smart ways to intervene and course correct should we see "out of range" patterns.
  • CoC interventions similar to above but would make CoC more robust as a true support service.
  • Refill reminders for prescriptions.
  • Syncing with health records (e.g., Apple Health, EHR integrations).
  • More robust chat features including for more guidance, support and intervention including potentially program based AI agents.