Podcast: Why some Mobile Design Patterns Fail – test with your users
Podcast: Why some Mobile Design Patterns Fail – test with your users
In this podcast series we ask experienced Appreneurs for one success story and one fail. You get actionable ideas for your Apps in just 20 minutes (well…26minutes this time!).
Here I speak with Henry Cho, a UX guy with over 10 years in Mobile UX and has been the design force behind pureplay mobile startups and large corporates – I forgot to ask him but I suspect his current work with Commonwealth Bank’s Apps would have millions of active users. Henry’s twitter is here.
Henry shares learnings about where user testing is critical and how sometimes the Mobile Design Patterns (User Experience – UX) prescribed by Google or Apple are a complete fail. Henry also gives us some insight into the current best practice in banking apps globally.
This is Episode 1, there was so much good stuff in here that I’ve split it into two podcasts. – enjoy!!!
For all our interviews we ask the interviewee to score their own insight – Henry self-scored his learnings and experiences as 0,9,6 because of the usability requirements in App design.
Let us know on our Twitter account how you score it.
Mobile Design Patterns prescribed by Apple or Google might not be as good as you think. Also how things like push notifications get lost and how you should engineer for that affecting the user lifecycle. The key is iterating your testing to surface roadbumps in the user experience or user lifecycle.
David: Gidday. This is David Jones from Street Hawk, and today’s guest is Henry Cho. Henry is a guy who has worked around the mobile industry for a number of years and done many things. I’ll let him tell you that story. Where I met him was in Startmate 2011. It was our first batch of startups and these guys were a startup that was kind of like a dream come true. They were three technical guys now called IRL Gaming, very smart, so we had a great UX guy, we had a great scale guy, we had a great developer.
I think Dan from a scaling perspective had worked at Yahoo, so he knew that they could actually go big if they were successful, and of course, they had this idea which was really fashionable at the time. There was a lot of stuff happening with mobile apps, and Foursquare was on its way with massive amounts of funding, and there were other companies out there like Koala and Scavenger that were doing little bits of gamification.
The IRL Gaming guys came to us and said they wanted to do a Zombie Apocalypse Meets Foursquare or something like that. We thought that was just too good to be true. I think you guys have met at gym board, something with the martial arts things. There were some tattoos so it just looked like the perfect startup all-round.
Anyway, that’s an introduction to Henry. So, Henry, tell us what you’ve been doing lately.
Henry: Lately I’ve been working mostly with CBA, heading up all their mobile products.
David: Okay. That’s Commonwealth Bank of Australia.
Henry: That’s right, yeah. We’ve recently redesigned their major banking app and that’s got seven awards so far, so we’re hoping to take that a bit further next year. It’s really trying to drive innovation in payments and personal finance. That’s our goal for that.
David: Yes. You guys, probably even before your time, maybe when I was with Vivante, there was kind of like the (NFC) scanning or the cardless POS concepts and stuff like that.
Henry: That’s right, yeah. We’ve got a lot of concepts – well, we’ve had a lot of things in production now, like you can get money out of an ATM just using your phone now – that’s a big one. We’ve got NSC payments. We’ve got CHIP on foreign payments. There’s a whole bunch of stuff. You know, you can control your credit card from your phone; you can lock it if you think you’ve lost it. We’ve got a lot of really good innovative stuff coming through there.
David: That’s right. By comparison, do you think Comm Bank, even though it’s an Australian bank, is right at the front of everything that’s happening with banks globally?
Henry: I would think so. I mean, there’s a few that you sort of talk about as being out there in the front: iGaranti, BBVA and Barclays. And beyond that there is not… does not masses happening. There’s SimpleBank and Movenbank but they haven’t really got to “scale” yet. So they’re being like the exciting child that people have been talking about for a while but haven’t simply grown up in any sort of big way. BBVA bought Simple Bank, so we’ll wait to see what they do with that.
David: Right, another Australian connection.
Henry: Yeah. I mean it’s interesting that the ones that are out there, so you got Barclays from the UK, BBVA is a Spanish bank and iGaranti is a Turkish bank. There’s simply nothing huge seeming to be coming out of America at the moment.
David: Is Tesco doing anything because they’ve got -?
Henry: Tesco is quite big as well, yeah. Tesco is quite big in… I mean, I think that’s where your Coles and your Woolworths are looking at. Looking at a supermarket, you can actually start to play in other channels. So they’re a very interesting one to watch. I think they created their own device even, their own tablet. They’re trying to sell that and sell that almost in an Amazonish model. So they’ve been trying to own the whole journey, if you like. Yeah, it’s pretty interesting.
David: Yeah, pretty aggressive. Yeah, you see a lot of that stuff that’s sort of shifting across into financial services.
Henry: Yeah. That’s a real thing for sure, you know, if I could buy insurance off the shelf while I’m buying cereal, you know. [Laughter]
David: It’s happening.
Henry: Yeah. I think that will be a thing. They’re obviously also getting into payments. I think Coles definitely have phone payment, they definitely have credit cards. We’re going to see challenges from lots and lots of different areas, let alone your Apples and your Googles, which are obviously there. But I think the maturity, there’s something different about the Australian market being that it’s very centralized versus American market which is very fractured. That’s why you see all those payment disruptors, if you like, over there because it’s very difficult to just send each other cash, like we can just pay, say “I’ll send you some money,” and it gives you BSB and they could do it.” They can’t do that. Even [5:35] sorts of other things, they’ll send you a check. So they’re much more ripe there for innovation, if you like, because the problem’s bigger.
David: Yeah. I guess the really interesting things that I’ve seen in the US have been mostly in the retail sector. I think obviously Walmart, we’ve got some of these Startmate alumni – the Grabble guys we know they’ve become a centerpiece of the Walmart strategy. Walgreens kind of with the chemist shop or the drug store do a lot of the sort of pre-order of your drugs. Yeah, so there’s a lot of interesting stuff there but I haven’t seen that much in banking, probably because of that fragmentation.
Henry: I think that’s really what it is. That’s why, you know, you had two banks that were able to start up like Simple and Moven (bank) and you go “Hey, this is the problem with the whole industry.” It’s a little bit analogous to what the phone companies were like maybe in the early 90s there. In the early 90s, you couldn’t send an SMS because of that fragmentation of the networks; they just wouldn’t do the interchange. So when you get something big that changes then messaging can explode. But it took them a long time to catch up.
David: It’s a good illustration of how things radically change as well too.
Henry: That’s right.
David: Okay. So tell us a little bit about what you’ve done before as well too. I know you’ve been involved with telcos?
Henry: Yup. I’ve done a lot of work with 3-Mobile back in the day when they were a big thing; they were quite an innovative company at the time. I think they really started to push into smartphones and getting content down to a phone, so I worked a lot with them for a long time, and left them when they became taken over by Vodafone. I think which was probably a good move. I worked a lot with Telstra right at the beginning when they were trying WAP products and i-Mode and all these other really wonderful…
David: Oh, you have been around a long time.
Henry: Yeah, early versions of the mobile web. I’ve also been doing some work with some startup accelerators in Australia, with Slingshot, so doing coaching days for them. I’m working with another startup in San Francisco that’s doing online education, so just finished writing the UX course for them. We’ve got 30 people through that, so that’s an exciting one that’s different.
David: Yeah, I think I know the one.
Henry: Yeah, they funded out of Techstars – there called DesignLab.
Henry: And their point of difference is that they provide you with a mentor. They’ve got lots of good industry mentors on board so you get to actually talk to someone in the industry that you’re trying to get into. That’s a very interesting one.
David: Great, very good. Okay, so let’s crack on with sort of the centerpiece of the talk, which is we want to look at perhaps one or two experiences that you’ve had during your career where you’ve had like an “aha moment”. You’ve been struggling with something and then you tried something and it worked. You cracked the challenge.
Henry: Yeah. Everyone talks about patents in design, so the UX patent, design patent, and they often look at the manufacturers like Apple and Google. If we used their patent until we find, we don’t have to think about… we possibly don’t even need a designer if we just do what Google or Apple does. There are a lot of cases where what they’ve done just doesn’t work and you might find that they then six months down the track they actually changed that.
So one of the examples that we had a strong fail on was the android implementation of the hamburger menu, which is a very, very strong patent, you would assume, globally pretty strong. But the way Google implemented it was instead of having the three bars, if you like, surrounded by wide space, they connected to the left-hand side, so it’s almost like you got half a hamburger or a hamburger going off screen.
So we thought, “Okay, well, we need to –” when we developed the android initially, there was a lot of backlash about, well, it’s too iOS-y. So that’s something you need to be aware of is there are some zealots out there and if they feel that you’re just giving the android guys the Apple version on their android phone, they don’t like it.
David: Right, right.
Henry: So went to great pains to make sure that we used android patents.
David: The back button for example?
Henry: Yeah, the back button for example. But this particular patent in about eight different rounds of testing now has failed, so it’s a really strong indication where you can say, “Just because Mom and Dad do it doesn’t mean that we should do it.”
David: Yeah. Just to drill in on that, you know, you say eight rounds of testing it’s failed. How exactly do you, as a UX expert, actually say what’s a success and what’s a failure there?
Henry: That’s a good question. We talk about hard fails and soft fails. If I was giving you a task to let’s say if we’re talking about shopping app before and I was saying, you know, “You need to find men’s board shorts,” and we’d give you that task and we’d observe you going about your business trying to complete that task.
A soft fail would be, well, he goes to the hamburger first, then he goes to the content link, then he looks for shorts, can’t find shorts, then goes to search, puts in “board shorts”, nothing comes up, which is pretty bad, but then goes to “men’s” then drills down through it. We might say, “Well, that’s kind of a soft fail, like he didn’t see where to go straightaway, but he recovered, so that’s all right.” But we would take note of that go that it’s not the best. We’d rather that didn’t happen. We’d rather that didn’t happen – we’d rather that they work out how to do it straight off the bat.
A hard fail is, you know, the testing session where the user abandons the task. They’re given the task, in that example, they might have looked in the hamburger nav, tried search, clicked in to a few different other areas, never clicked in to “men’s” and just said, “No, I can’t find it.” That’s reasonably rare but it happens.
David: But it must qualify these days that, you know, from a mobile perspective and attention deficit life cycle that even a soft fail must becoming a hard fail now if you actually say you kind of achieved it in ten seconds or five seconds?
Henry: Yeah, that’s right. I think the only other qualifier for that is where you have content that’s… let’s say, back to that shopping app, if you had a disputes area, that disputes area might not be super front and center, so that might take… of course it’s not a primary task, it might not be, well, be a priority.
David: Sort of like try and change your privacy settings on Facebook.
Henry: Yeah, that’s right.
David: You have to give up the week’s work to actually…
Henry: Yeah, get a manual and… Or another example I guess is the iOS is turning off location services or any of the settings things, you know. You go to settings but where is it in settings? Then it might be a little bit trickier, and that’s because you have to get some prioritization on the tasks. So if it’s something that’s important but infrequent, they’re the more difficult ones to prioritize.
David: Yeah, right.
Henry: So that’s tricky.
David: If there are some people who are listening who are small startups, those examples that you’ve found might grant some testing; obviously that’s people with deep pockets that can do extended UAT. What’s kind of your best hack for small companies with small apps? You know what I mean.
David: Small budgets.
Henry: Small budgets, yeah. I guess you can always do guerrilla testing, which is not going to the zoo. That’s just chatting to people in cafés, then buy them coffee to do tasks on your app. That’s very common. If you work in a business where you haven’t sort of got any budget yet, you can just walk around the floor; you just need to be aware that… don’t test with engineers. Unless your Atlassian and your building “Jira” – if you’re just building consumer products, it’s really best never to test with anyone from a technical or design background., anyone involved in web dev, app dev, not those people, but there’s often other people around the building as well. Maybe it’s the people at the front desk, maybe it’s the people in HR or legal or whoever. There are other people often in businesses that aren’t directly related to creation.
David: One of the experiences that I had with a consumer-facing application was that I’d go out and sit with people and do a run-through on that. There’s something to do with like if you don’t set up the environment right and it sounds like it’s the way of framing the UAT or the tasks, that you get people to just kind of… they want to be nice to you, they want to be able to actually have a pleasant experience. It was so kind of like, I guess, the “aha moment” for me in one of those things.
I would say, “Well, how much would you pay for this?” They’d say, “Oh, nothing.” You know, I’m just testing it. I said, “Well, why wouldn’t you pay for something? Doesn’t this have value?” They go, “No….maybe I’d pay off the six months.”
So how do you actually put people on the spot? So if you do go up to the receptionist or the girl in the coffee shop or whatever and ask them to try something, how do you make sure she’s just not being nice to you?
Henry: I guess there are a few things you can do. The first thing is lie. It’s not really a good advice but it’s true. You just say that you’re not actually involved in [16:06]; you’ve got no skin in this game at all.
David: “My friends are doing this. What do you think?”
Henry: Yeah. Or “this guy I know” even. This friend might still be like, “Oh, I don’t want it to be friend.” Just take yourself out of the picture. So, that puts them a little bit more at ease, because you’re right, as soon as it’s “Hey, what do you think of this thing that I made?” it’s going to be a particularly hard person that says, “Man, that’s ugly. Your baby’s ugly.” That’s not something that people generally want to do. So the first thing is to take yourself out of the picture and say, “Some people that I know are doing this. I don’t really know them that well. They’re a little bit worried about whether it makes sense or not. Can you give us some of your time?”
The other important thing to do is—and this is really important; it’s a real subtlety, I guess, and new to testing—you’re not really asking for opinions. A real danger is that you can go out there and say, “What do you think about this?” Really statistically it’s not enough data. You’re really looking at behaviour. So when you give people tasks, you’re watching how they perform those tasks. You’re watching more than you are listening.
That’s really important because, you know, I’ve done hundreds and hundreds of tests now, and the disconnect between what people say and what they do is massive. They’re just, you know, “Oh, that was really easy.” Man, you watched them and they struggled for 15 minutes to do a 5-minute task, and you go, “How was that?” “Oh, it was cool.”
If you took that quote and you put it in your report without that context of the observation, it would seem like everything’s going fine. And if you’re going out there with a bias because it is your baby, then you can go out there listening, you know, it’s all “Blah-blah-blah, yeah, I like it.” Oh, they say they like it. You can miss all that other good stuff. That’s the danger, because you’re just going out there to confirm and validate, not to learn.
David: Yeah, confirmation bias.
Henry: Yeah, that’s right. And I go to all my guys. You know, your goal is not to validate, it’s to learn. If we fail and you learn, I’m just as happy, if not more happy.
Henry: I’d rather learn than just… I’m almost suspicious if everything keeps on going through smoothly. It’s a little bit like, hmm, what did we miss? There’s something here either really incredibly intelligent and lucky or we’re missing something and we’re not asking the right question. We’re not seeing the bigger picture. There’s something that’s not quite right here because we haven’t got any snags. And it’s super simple. I mean that can also be the case, but…
I’ll give you another example of where you can have misconceptions about phone use feature, that notification bar, the slide-down drawer in iOS and Android. We’ve done some testing in that. And even within mobile agency teams—and this is a shocker—people even in delivery teams didn’t know about the swipe down to review all the notifications. So product end is going, “Well, I’m going to send this, push here and then it will be in the notification bar.” Then what happened as you go to testing, the thing pops up, they go, “Oh, yeah, what was that?” Let’s say it was a PIN code you needed; then they go, “Okay, where was that thing?” And it’s gone. So they’re looking, okay, oh, if it’s an SMS, they go into this, and this is not there.
David: Right. They don’t even know where you find it.
Henry: Yeah. They don’t even know what – I don’t even know what else there is outside SMSs, so now I’m just… now, I’m stuck.
David: Yeah. And it’s different. On iOS, you can go back and possibly find what you’ve missed, but on Android, it just disappears altogether. I’ve written a couple of things on this. I kind of feel as though that both Google and Apple are really sort of putting a little priority on the notifications centre metaphor, and iOS8 certainly and Lollipop sort of changed that a bit. But it seems as though in order to push to ultimately being a decent displacer for email, for reengagement or transactional type stuff, they do need to actually do a lot better job in terms of that experience.
Henry: Yeah. I would totally agree, Dave. It’s not enough. If you’ve got an app and you think that sending people notification is enough, it won’t be because depending on your interest, there’s going to be a large proportion of people that either miss that, you know, the phone’s down, they’re talking to someone else, and they just completely miss it and can’t action it. If there’s any information in it that they need to perform another task, then it’s even more critical.
David: Yeah, that example of the one-time PIN is a really good example, where you end out with a broken process.
Henry: We had a massive number of fails in that piece of work that we saw as this extra small turned out to be, I think, six sprints’ worth of work to get it right.
Henry: Yeah. That’s a massive assumption that that thing works, that the notification bar works. The bigger one there, I guess, that you don’t take into account is that people have a strong sense of connection with that and SMSs. They understand that “Hey, John just sent me an SMS and it comes up there and it said, ‘I’ll be late for dinner and I’ll meet you at blah.’ Oh, okay, I have to go to my SMS to see the rest of that message.”
They understand that connection and then they take that and then they infer to… well, what we’ve seen is that they infer that across a lot of their messages, so anything that comes up in that pop bar they will then think, “Oh, that’s an SMS,” and if they can’t find it in their SMS thing, then they’re quite confused. So sometimes they might open the app that it came from if they noticed which it was, but you need to sort of continue that flagging the whole way through. So if you do that, you know, the badge has to go on the app, then you have to provide a way in the app to display that message, which is all additional cost, right?
Henry: And if you hadn’t assumed that, then you might be surprised now that, oh, we thought we’re just going to help solve this whole messaging thing in the app with push, but now we’ve got to build the thing to show the message in the app anyway.
Henry: That’s an interesting one.
David: All right, that’s good. So what me will do is just go back and revisit two of those three to give it a rating. I had talked to you before about giving things ratings based on user experience, acquisition, engagement and retention. So if we take first the hamburger menu stuff, I think that’s probably obviously very user experience focussed.
Henry: Yeah, that’s right. Straight up, that’s user experience 100%.
David: 100% on that and not much on acquisition or retention in that sense.
Henry: I guess you would say, look, user experience and retention are pretty strongly linked. If you can’t use it, then that’s where you get the drop-off. A really great example for that is – I don’t know how many different calendaring apps or reminder apps or to-do apps that you’ve tried, but they’re the ones where you see, if it’s not better, if you don’t get it right, then they just fall by the wayside. My phone is littered with them.
David: Yeah, same with note-taking and stuff like that.
Henry: Exactly, yes. So they’re a pretty good example of if you don’t get it right, then your retention and engagement drops off. So, they are linked.
David: And your learnings in regards to the push notification and those six sprints, can you rate that as well too? Obviously no acquisition there but…
Henry: No. So I guess that’s… that’s a user experience thing and, yeah, it really… well, it can be a blocker in acquisition. So for any sort of security-based service or payment-based service or anything that requires the so-called two-factor identification or step-up authentication, that can be a blocker, you know. If you actually can’t get that code and then you can’t use the app, then you’re stuck in registration. That’s a problem as well.
David: All right, very good. Well, there was a mix of both “aha moments” and fails in those particular stories you talked about, which is the right way it should be, I guess. All right, well, that is fantastic. There’s so much good stuff in here that I think what I’ll do is I’ll split it into two different podcasts.
David: There’s so much there. I feel as though we could keep on talking for ages.
Henry: Yeah. I’m never accused of being short on words. [Laughter]
David: Well, they’re all good words as well too.
Henry: Thanks, Dave.
David: Where should we look for you next? What should we be looking out for? What’s the name of that education startup again?
Henry: That’s DesignLab. Look out for that. We’ve got three courses coming up. The first one’s just got out there. Look out for me in the General Assembly; I’m always teaching there. So, thanks too, David, that’s been an ongoing great thing there.
David: Oh, yeah. I did the intro.
Henry: That’s done really well. That’s going really well. A good one for startups is if I’m doing a whole-day workshop or a Lean UX workshop, that’s a good one for how to get a good process using a variant of the Google Ventures “Design Sprint”, so something that you can just take away. There’s a whole bunch of tools in there that you can use in your startup to get validating your ideas, to generate ideas, and to try and think of it differently. So you use that over-used word, like to try and work out how to come up with innovative solutions. So that’s a good one.
David: Great. Well, thanks so much for your time. It’s been fantastic.
Henry: Okay, have a good Christmas.
David: Ciao! Bye.