S2E3: Eduardo Sonnino – How to design tactile hardware experiences remotely

How do you ensure new hardware and the software on it meet customer needs and work together as a seamless experience? And how do you test an experience that’s been designed in the hands of real customers in the middle of a global pandemic?

Eduardo Sonnino is a Senior Product Designer at Microsoft, and talks to us about designing for hardware experiences remotely.

Design Untangled | A UX & design podcast in plain English
S2E3: Eduardo Sonnino - How to design tactile hardware experiences remotely
/

We’ve partnered with ProtoPie, the future of interactive product design, to help you navigate through uncertainty and overcome the challenges today’s unprecedented conditions have brought to the industry. Join us for Season 2 – Designing for a new level of uncertainty.

If you are a digital designer you may be used to designing experiences for existing hardware your customers may use, such as a smartphone or tablet.

But how do you approach designing for an entirely new device?

How do you ensure both new hardware and the software on it meet customer needs and work together as a seamless experience? And how do you test this experience that’s been designed in the hands of real customers in the middle of a global pandemic?

About our guest

Eduardo is a Senior Product Designer at Microsoft and was most recently involved in the development of the new Surface Duo and had to come up with different ways to respond to new customer needs due to the Coronavirus pandemic. 

The Surface team sits at the interjection between software and hardware and involves both hardware and software engineers and designers. Eduardo talks to us about how these disciplines come together to create unified experiences for customers. 

What you’ll learn

  • How do hardware and software come together to form new experiences?
  • How do you prototype experiences that involve hardware and software elements?
  • What techniques can you use to test both hardware and software when there is limited access to users for usability testing given the Covid-19 restrictions?
  • What teams and skill sets do you need to create these combined experiences?
  • What challenges are there in testing hardware with customers during a pandemic and what research approaches can you use?
  • How have customer needs around personal technology and home tech changed as a result of the increased time we are spending at home?
  • How does a hardware and software team come together to make design decisions around the hardware itself?
  • What things can you learn from users only when a product is post-release?
  • How do you test for human factors such as neurology or ergonomics during the design process?

Show notes

Transcript

[00:00]

Chris Mears: I’m Chris Mears.

Carla Lindarte: And I’m Carla Lindarte.

Chris Mears: We’re two UX designers.

Carla Lindarte: And we hate jargon. So, we’re here to help you untangle the world of design.

Chris Mears: Cut through the crap and talk about what really matters. 

Carla Lindarte: Yes, solving people’s problems. 

Chris Mears: Welcome to Design Untangled. 

Carla Lindarte: Hello, everyone, and welcome to Design Untangled with me, Carla Lindarte, and Chris Mears like always. And today we have Eduardo Sonnino. Eduardo is currently in the US and he is a senior product designer at Microsoft. And he’s got a very, very interesting background, as he was part of a team who worked on the Surface Duo. So, welcome, Eduardo, thank you for being with us today. 

Eduardo Sonnino: Oh, thank you so much for having me. And I’m super excited to join you guys today. 

Carla Lindarte: Great. So, just tell us a little bit about yourself and kind of how you go into your role at Microsoft. It’ll be interesting to know.

Eduardo Sonnino: Sure, yeah, absolutely. So, I’m born and raised in Brazil. And, and in Brazil, I studied computer engineering because design like doing design in Brazil, it’s it’s a it’s a rough path, honestly, like, hopefully today it’s better. But, back in the day when I when I graduated, was really hard to have a career in design. So, my dad skewed me into computer engineering. And that’s what I what I studied. And throughout my college time, I always participated in competitions for software development and interface design along with my brother, my brother is a hardcore developer. So, he codes way better than I do. So, we paired up really well, him doing the code, and I used to just do the design for our projects. So, year after year, we we competed in those competitions. And some of those competitions were sponsored by Microsoft. And we did really well on a few years. And after a few years competing, and after I graduated, I got an internship at a company in Texas called Telerik. But, as a, as a designer, not as a developer, because of all the projects that I’ve have worked in the past, as a designer I was able to gather kind of a good design portfolio. And I was able to land an internship as a designer in this small company, and that spring boarded me into getting a design internship at Microsoft in the Windows core team that used to do search and start and test bar. So, that’s how I got my food on the door at Microsoft. Got a job and moved to North America to join the company. 

Chris Mears: First question, I guess, is just how have you been coping with the pandemic just personally and workwise as well? 

Eduardo Sonnino: Yeah, definitely it has changed quite a bit. All the meetings are online and my my role of Microsoft I work on the Surface team right now. And our team works with software and hardware. As and as you can imagine, doing hardware there’s a lot of physicality in it, there’s a lot of touching the the products, touching the materials, making physical prototypes, carrying them around. So, it’s been quite challenging to, to work on the hardware space remotely, mainly because of the physicality nature of the of the trade. 

Chris Mears: So, how have you tried to overcome that? Because yeah, as you mentioned, it’s very reliant on I suppose both you and customers being able to physically kind of touch and hold devices, as you say, like, have you found any remote workarounds for that or is it just something you can’t recreate? 

Eduardo Sonnino: So, because we work on the software and hardware space for our software solution, we always prototype our solutions and demo them in, in the hardware that we’re designing for. So, for that specific solution, we we, in our remote meeting instead of just sharing my screen and showing a PowerPoint deck or a figma artboard I I bought a tripod with a camera and I put it in an angle to shoot, to shoot, me using the device over my shoulder. So, it can showcase people, me using the prototype on the device, other than just sharing my screen and showing what’s on my screen. And that’s on the software side of things have helped immensely like, instead of just sharing my screen and, and, and showing the pixels, I actually can showcase me interacting with the device, interacting with the prototype that I created. And then on the software space that helped a lot. On the hardware space, it’s still very challenging, you still need to get your 3D-printed prototypes and touch them and feel them. So, we’ve been doing kind of a rotation for going to the office in the safest way possible to check on prototypes. There are some other prototypes that we mail to people. So, we mail directly to their homes so they can try it out and then give it give them back. So, that’s that’s how we’ve been coping with the situation. 

Chris Mears: So, how do you record the feedback from like, when you post a device to someone? Is it like a diary study kind of thing? Or do they film themselves using it?

Eduardo Sonnino: For the software prototypes, we we we get into video calls and that does the trick usually, like because we’re doing instead of just turning on my camera, I turn on this tripod camera that shows me playing with a prototype and we have a conversation around the experience that I’m showing now that has been shot over my shoulder. For the hardware ones is definitely more of a one one at a time feedback, we we sometimes we have to wait a full week for everyone to have experienced the physical prototype. And then we can have a conversation around the the specific model. So, it gets a little bit slower the feedback process, but I think still, we managed a way to to get everyone get everyone’s input and make sure everyone has a voice on the on the process of hardware making. 

Carla Lindarte: It’s interesting, interesting to know a little bit more about because obviously you as you talk about how you do the prototyping and testing, you’re kind of explained in your role, but it’d be good to know what your role was in the process and what kind of like team setup you had, like what kind of skill sets and and people you had in the team. 

Eduardo Sonnino: So, my my team at Microsoft, so I’ve been working on the Surface Duo project for the last couple of years. And my team specifically deals with the with the space between where hardware meets software. So, trying to get that the the, it’s basically in the studio, we don’t have quite a differentiation between a hardware designer and a software designer. We’re all product designers. And but my team specifically deals with a lot of prototyping and bridging the gap between design and engineering. So, you can imagine that the skill sets of the folks in my team, they are all around prototyping, coding and translating UI from from from figma or sketch into code, into XML or zamel. So, and also translating the motion design from After Effects or ProtoPie, or Principle into into real code. So, we try to my team specifically tries to digest as much as possible to design assets into the engineering team. So, we can have we can bridge the gap between the design process and engineering process. 

Chris Mears: As I’m interested to learn a bit more about those conversations you have with the the hardware engineers. So, like could you give us an example maybe of one of the maybe design problems that you had to tackle and come to a solution with them during the development of the device or any more of a general examples you can think of?

Eduardo Sonnino: Oh, yeah, for sure. So, on on Surface Duo, for instance, we have a finger print reader on the side of the device. So, right below the power button, there’s a fingerprint reader right there. And we had many, many studies of like when the fingerprint reader should be on or off so it doesn’t activate in your pocket, for instance. Or if what’s the hinge angle, for those who doesn’t know, Surface Duo it’s a two-screen device that opens up has a 360 hinge. So, as you open the device, you can peek on, on notifications, and at the time, so also the fingerprint reader ability to to like when the fingerprint reader reads your fingerprint, and it’s either correct or not, how do we show that to the user? So, all those experiences that are hardware, software related, we had deep conversations between hardware engineers, software engineers, and the software, the software design team and the hardware design team to to come with the proper solutions for those. 

Chris Mears: Cool, awesome. So, yeah, I had a question, I suppose a bit more specific to the the Duo. Do you feel that because of COVID, and the way people’s lives have changed, and their work has changed, do you feel like the purpose of that device was originally intended for as maybe shifted because of the pandemic? Or do you feel it’s sort of, of its time almost, and people are looking to get a lot more productivity out of their personal devices? 

Eduardo Sonnino: Yeah, that’s a really interesting question. Because definitely, when we started the concept of this product, we had no intention of it, of being a great, a great device for you to use during a pandemic, it’s something that definitely caught us off guard. But, as you as you mentioned, like the concept of productivity in a mobile device has been something that many, many companies have been trying to achieve for quite a long time. And Microsoft is, of course, one of the companies that have, have tried to heavily invest on that, and try to pursue that mobile productivity. And I feel that, although there are many interesting concepts on how to achieve mobile productivity, we, we we think about two screens as being an amazing device that you carry with you all the time, it can serve many, many purposes in your life, and it can definitely make you more productive wherever you go. Mainly because multitasking is built in into the core experience of the of the device, it’s not something that we added on top, as a lot of other devices have. So, because multitasking and doing multiple things at once is is the core of the device, it really shines through when you use the when when you go through the experience. And when you see it on your couch, and you do two or three things at the same time with this very thin device on your on your hands, it’s it’s something that I don’t think at other concepts were able to achieve in such a seamless way. Like usually the the mobile devices, they start with one task, and then you can do something to achieve a half screen or split screen. But, on Duo, the the nature of the device is to do multiple things at once. Doing two or more things at once is is core experience of, and again, it’s built in into the heart and soul of the device, which is something very different for for a mobile device. 

Carla Lindarte: This sounds interesting, because, you know, other companies have tried to do that in the past, then, you know, I remember the beginning of you know, mobile devices and people you know, people do one thing at a time, but actually, as you said, people are doing multiple things at a time. So, how did you guys kind of perfected it? Did you do lots of testing, user testing, or you know, what was that design process? 

Eduardo Sonnino: Yeah, we did. We did a lot of user testing, and luckily Microsoft has a a lab called the Human Factors Lab, where we have all types of machinery to understand the responses of the human body to technology. So, in terms of ergonomics, in terms of neuroscience in terms of light, and you can imagine, like, how many good insights we, we got from from studying the human factors of of tech. So, it’s basically it’s easier on your brain to do two things at once, and having that seam in the middle, and doing two activities at once with Duo, it’s easier on your brain than doing two things on on any other mobile device, for instance. The the cognitive load and the the brain activity shows that the mental model and the experience that we achieved on Duo, it’s really, it’s just easier on people’s brain than other multitasking approaches that other companies tried to do. 

Carla Lindarte: Yeah, interesting. I mean, as you said, before, you’re obviously we’re planning to design a device that worked in the middle of a pandemic, right. And we, as designers, we always work with the level of uncertainty on you know, product roadmap, or features, etc. We kind of like, in the process, we keep making changes, etc. But, how has the design of the Surface Duo changed, you know, as a result of COVID? I mean, for example, the latest Google Pixel have removed the has, sorry, has brought back the fingerprint ID because people are now with face masks, you know, it’s harder to do face ID etc. So, was there anything in the kind of feature, you know, roadmap of, of, of the Surface that you guys had to change because of COVID and what’s happening in the world? 

Eduardo Sonnino: Yeah, when we, when we were developing Duo, I believe that pandemics hit us in the latest phases of development of the product. So, hardware wise, definitely we we were locked and loaded. And we we didn’t, we weren’t able to do much to pivot from our hardware design standpoint, because of the pandemic per se. But, on the software side, on the other hand, we saw amazing, interesting opportunities in different use cases that we never thought before being used with Duo. Duo has an interesting concept called app pairs, that in one tap, you can open two apps at once on both displays. Well, I mean, one on each display. And it was really interesting to see how people were using all our users were using the that feature to create different app pairs and app sets for their different needs during the pandemic. So, you can imagine a Zoom call on one screen while you’re taking notes. Or also, like a video call, why while you’re watching a video, or while you’re reading a book, so like, with a textbook. So, we couldn’t do any hardware changes for the pandemic, per se, to inform the design of the device, but for sure, the way that people are using the device pivoted quite a bit. So, the software experiences that we didn’t anticipate while designing the product they’d really changed for because of the pandemic, which is super interesting. 

Carla Lindarte: Yeah, that’s super interesting. I mean, it’s also like quite exciting that you guys were able to deliver something that is going to be useful for people. So, that’s, that’s really cool. 

Chris Mears: Yeah, I think the other interesting thing about it is, you know, this whole season is about uncertainty and designing within it. And now even before COVID, I think maybe some people thought if you did enough user research up front and enough for iteration iteration, sorry, then once you release your product is going to be fairly fit for purpose. But, I think this situation you just described has proved that sometimes you will just never know things until it’s actually in the hands of customers in the real world, right? 

Eduardo Sonnino: Yeah, exactly. And that’s, that’s the best way to learn. You just put something out and see how people perceive and see how people use it, and you learn from them, and you learn to iterate and the whole fail fast mentality that we embrace so much. That’s, that’s very true in in this times, you can, you can’t foresee what’s coming. So, putting something, trying to foresee creates more effort and more churn than trying it out and learning from your, your mistake or from your, from whatever you get right, if you’d like.

Chris Mears: Mhm. So, just following on the thread about how people’s usage just changed. Have you seen any other kind of trends around, and the technology in people’s homes in general, so like, you know, smart speakers or Internet of Things type devices, because I know, I’ve been chatting to Alexa a lot more just for someone to talk to. And yeah, getting a lot more involved in, I suppose setting up ways my home can help me do work at home more effectively. So, you know, routines and all that kind of stuff. I’m just wondering if you’ve seen any of that kind of behavior? 

Eduardo Sonnino: Yeah, absolutely. So, being part of the Surface team, it’s, it’s incredible, because you can not only you’re not isolated into your problem space, but you also have access to other designers working in other programs. So, you can see what people are doing with sound, what people are doing with the the new laptops and new tablets. So, it gives you exposure to a lot of interesting projects and makes your head spin quite a bit. So, you can definitely see, like how, of course, the conference calls and video calls now are are vital for for the work that we all do every day. So, the investments on on displays that are are are kind of a not utilities, like utilities like your, your fridge or your stove, are they called utilities?

Carla Lindarte: White goods.

Chris Mears: I think we call them white goods.

Eduardo Sonnino: White, yeah exactly, like thinking about devices as white goods?

Carla Lindarte: Yeah, I mean, you call them white goods doesn’t make any sense anyway, my fridge is black, so.

Chris Mears: That’s how they call them in the UK, but in the UK, they always have weird ways of calling things. 

Eduardo Sonnino: So, white goods. Back in so there are two big things that changed, right, like one was that when when you’re working in in a, you know, in an office with your team, people that are online are often second citizens, which is very unfortunate, unfortunate. But, it’s like it’s very true. If you have, I feel that if you have five people in a conference room and three people online, those five people will have way more advantage because of body language, because of well, everything that comes with being in the same room with other people. And the unfortunately the the other three people, a lot of times become second citizens in a meeting like that. So, reversing reverse engineering that and thinking about that problem. It’s something that we’re very worried about, not worried about, but we’re actively investing on that whenever the the office opens again. And some people are going to be comfortable going back to the office, but some people are still going to won’t feel as comfortable going back to the office, and how we can not go back to that situation of some people feeling like second citizens in the meeting when they’re online. So, we’re thinking a lot on the hardware space about things like that, for sure. Like this, this coming back to the office and this mixed workspace where some people are going to work from home remotely and some people are going to work from the office, how we can do things differently moving forward. I feel I feel that’s a huge theme that we’re we’ve been talking quite a bit. And the second part of it as as you were mentioning, Chris, we’ve been spending too much time at our house and this makes us reflect on how we can make our house smarter. Like our home is not just a place we wake up, and, and then we leave, we leave our home. And then we come back at seven o’clock, six, seven o’clock at night, and we have some food and sleep again. We spend all our creative time and our active time in our homes, and that definitely makes us think more about how we can make our home smarter, how we can make our home help us on our daily tasks, how we can make our home more cozy and make us feel more not so lonely, you know, and what tech can make us how tech can and devices can can accomplish those things as well. So, I feel those two things. One is working remotely how we can do work remote work better, how we can improve this mixed works workspace situation that we’re going to get into very soon. And also how we can make our home like looking back at our home and reflecting more about our, the place where we live, how we can improve on on on our homes. 

Carla Lindarte: I have a question like, based on what you were saying, and you know, how people’s like consumers’ behaviors is changing so rapidly and you know, going back to the point of uncertainty. How, like, do you guys prototype, let’s say prototype lots of different features, and then I’m just kind of interested in the design process. And then just kind of go with lots of ideas first, and then test those with, you know, with customers and also talk to the engineering team to see if it’s feasible, or you kind of refine–so it’s more like, I just want to understand what the design process you guys follow internally. 

Eduardo Sonnino: Oh, yeah. So, uh, because a lot of our projects in the Surface team, they are, they run on devices that sometimes do not exist, or they run those software projects, these run on devices that are like they’re they’re new form factors. So, as much as we can, we love designing on the devices that we’re designing for. So, prototyping is huge for the software side is a huge part of what we do in in our team. My personal experience is like, usually, I don’t even go to pen and pencil, I go straight to my prototyping tool with like, we use ProtoPie quite quite a lot, which is a prototyping tool. So, ProtoPie is almost my, my pen and paper, I, I go when I have an idea, I go straight to ProtoPie and I draw a few boxes, make them make them dense in my prototype, put it into either either to device that if we have the device already put it into the device or put into something close to that device. And we showcase to ourselves to in the design team the prototype to evaluate if the, if the idea is good. And we also show that to the engineers to start getting an assessment of feasibility of the of the of the idea. And we do have a very close relationship with our engineers, our engineers where every step of the design process, we have meetings with them to validate if something is kosher or not, if something’s expensive or not. So, we have an assessment of of what what like how, how we can get around technical constraints and still have a very polished and and conceive the best experience that we want. So, we yeah, we start we start with prototyping from the get go. Like our wireframes are ready prototypes. And we we just take the fidelity up a notch for for the for the ideas that we have the the blessing from our design leadership, our design team and and the engineering team all together, and we just take those prototypes to the next level to the next level, and so on. 

Carla Lindarte: Great. Well, that sounds good. That sounds like the ideal. I think with now, all these tools available is much easier to do that than before but yeah, sounds really good. So, just to wrap up now, and we always ask this question to most of our guests and I hope I’m not putting you on spot there, but I just wanted to see if there’s any books, podcasts, resources or something interesting that you recommend junior designers or just designers, like you, experienced designers, who are looking for advice to navigate today’s, like, uncertainty and complexity of of the design world. Is there anything that you can tell them to do? Or action that is a bit more tangible? It would be great. 

Eduardo Sonnino: Yeah, no, that’s, that’s always hard to recommend. There’s a I’m going to do something a little bit not so orthodox. It’s my dad gave me a book called Finish. It’s a it’s not design related, but I think it brings a lot of good topics. Let me the author of the book is Jon Acuff, I think that’s how you pronounce his name. And basically, it’s kind of like a it’s a book that talks about it’s called Finish: Give Yourself the Gift of Done. And my dad gave me that that book as a, like a backhand slap compliment. I don’t know how, how the, the, the term goes.

Chris Mears: A backhanded compliment, you mean.

Eduardo Sonnino: Backhanded compliment saying that, like, I do a lot of things, but I don’t finish them because a lot of times, I’m too perfectionist on the things that I do. And I think that’s, and then he gave me the book, I read it. And it’s kind of a, it’s an interesting book that mentions basically, that well, done is better than than it’s basically you give yourself the gift of done. And by that, it means that you give you give out your perfectionism to, to have realistic goals and achieve them and finish whatever you whatever you started. And that’s for, for designers I think that’s really an interesting concept because a lot of designers start their their journeys thinking about, like, every task needs to every step of the process needs to be perfectly polished and perfectly presentable and it needs to be high fidelity and needs to be exactly how it will be in the end. And they give out they give away the the scrappiness and the the nimble of wireframing, and prototyping and being and having tons of ideas and scrapping them and failing fast because they they they attached themselves to perfectionism. So, that book actually sparked me something interesting, which is basically given so letting letting that perfectionism go and and making sure that you finish whatever you try to what you started is [inaudible 33:58]

C:. Yep, yes. Yeah, I know, many people who could probably use a bit of a read of that book. 

Carla Lindarte: Yeah. Especially me. Eduardo, thank you so much for being with us. Really interesting insights for our audience. And yeah, good luck with the future products that Microsoft are gonna release. 

Eduardo Sonnino: Thank you so much for the invitation. 

Speaker 1: Search and subscribe to Design Untangled using your favorite podcast app and leave us a review. Follow us on the web at designuntangled.co.uk or on Twitter @DesignUntangled.

[34:46]