We’ll make a wager that no matter where you’re reading this from in the world, you know Candy Crush. Whether it’s familiar because you’re a subway over-the-shoulder snooper or because you’re a fan yourself, the colorful and hypnotizing layout stands out from many games. King has, in part, made Candy Crush such a success because they’re committed to localizing and globalizing content for their players.
In this episode of The Loc Show, we go in depth about video game localization and why Miguel’s work with King as the Global Localization Manager stands out. Tune in and learn:
- How he actively works against the prevalent thought within the industry that localization is an entity in and of itself.
- Why the localization team at King does serious research into how gamers from different cultural backgrounds perceive everything from font type to beauty.
- How Miguel and his carefully selected team members folds in many considerations to maximize localization efforts, resulting in a seamless gaming experience for players by asking important questions:
- What is the message and who is it going to?
- What are the preferred game modes?
- What cultural nuances should be considered?
- How can we help the marketing team find customers by localizing keywords?
- Is there different terminology used in games for different markets?
Join us for this conversation as Miguel takes us through his 20+ years of experience in the industry, gives us the rundown on how he has bolstered his team to be the glue at King, and shares snippets of his successful blog Yolocalizo.
Subscribe: Apple Podcast / Google Play / Spotify / Stitcher
On this episode, you will learn:
- How Miguel has built his crew to become the glue for the company and how he encourages team members to ensure they’re silo breakers.
- How Miguel embraces the RACI framework to enable his team and colleagues to understand localization is a shared responsibility and the most effective way to keep projects on track.
- Questions to ask yourself, your PM and your team to keep your projects moving seamlessly.
- Miguel’s thoughts on the ROI of localization and globalization and what other insightful questions should be discussed with your C-Suite to evaluate success.
What to listen for
[04:05] How Miguel came to work in the industry and about his current position at King.
[08:55] The localization process at King and the importance of building a strong and balanced team.
[15:55] Understanding that localization is a continuous process.
[17:39] The RACI framework: What it is and how it helps to keep teams on task.
[21:23] The results Miguel has seen from the implementation of the RACI Framework at King.
[23:15] Why the player experience is a more valuable metric than ROI.
[28:34] Measuring customer satisfaction: The single unifying metric.
[31:42] Addressing internal feedback regarding localization processes.
[33:12] Why Miguel is proud of shifting the localization process to an internal model at King.
[35:46] Taking on talent: The importance of creating a balanced team and investing in people.
[40:50] Is ROI important to localization and, more importantly, can you survive the market without it?
[46:30] Understanding the added value localization brings.
Keep on learning, bookworm.
- Yolocalizo, Miguel’s website
- King’s site
- @King_Games on Twitter
- @lifeatking on Instagram
- Smartling’s site
Full Transcript that almost certainly has typos (forgive us!)
Announcer: You're listening to the Loc Show presented by Smartling.
Adrian Cohn: Hey everyone. Welcome back to the Loc Show. I'm your host Adrian Cohn with Smartling. My guest today is Miguel Sepulveda, the Global Localization Manager at King. If King doesn't ring a bell, let me jog your memory. King is the video game developer that's brought Candy Crush and other notable games to the market. And I know you've heard of Candy Crush. Miguel has also got a blog called Yolocalizo where he shares well-informed perspectives on translation, ROI, Internal Project Management best practices, and how to assess the efficacy of your translation program. We dive into all of this and more right now on the Loc Show. Thanks for listening. Miguel, thanks so much for being on the Loc Show. Welcome to the podcast.
Miguel Sepulveda: Sure, no problem. Thanks for having me.
Adrian Cohn: It's great to have you, and I want to start off by reading a quote. That is, it's your quote.
Miguel Sepulveda: Oh, really?
Adrian Cohn: Globalization, localization, internationalization trans creation. Our industry is quite complex and we do much more than translation. This is a quote that's on your blog, Yolocalizo. Tell us a little bit about you and how you've come to this perspective.
Miguel Sepulveda: The reality is that, I mean, I've been for many years in the industry and there is still a misconception that this industry is just translation and the session with Google Translate and machine translation and everything is not helping in that particular part. Miguel: What I'm continuously thinking is that a good language professional, a good organization professional is much more than that because when you take all these different subcategories that you just mentioned there, the globalization, the internationalization, all these different services is what you really need to have grow Global product. Because if you just do the known part of translation, you are missing a big part about how to adapt to the culture, to the specific audiences. What I was thinking with this, and I was always be solace in this like a sub puzzle, but with different pieces and localization is just one of those. But before you could have some content, some key strings to translate there are many things that should have done before. That's why I was coming up with a kind of equation and a kind of combination between all this together is when we do really have a global product, if you have only some of the services, you have something, but this is not a full global product. So that's how I ended up with this train of thoughts.
Adrian Cohn: You're certainly an informed person in the industry. This has been a long standing career for you. You've been with King, the video game developer now for over six years. And you're currently the Global Localization Manager there. Tell us a little bit about your backgrounds and how you got into localization.
Miguel Sepulveda: Many people from my generation accidentally, because back in the days I used to have in the mid 90s, and back in the days there was not like a formal localization career or nothing even similar to that. Basically the criteria back in the days, it was that you speak well language English, and maybe you have some knowledge of computers and then you are in the either translator path or functional QA or linguistic testing. In my case, it was with linguistic testing with Microsoft. In the mid 90s in Dublin, they were setting up their industry. And they were basically looking for a Spanish linguistic tester for Microsoft Office sometimes for Windows. That's how it started. I mean, I was not a linguistic myself. I studied computer engineering, but speaking a little bit of English and being native speaker and being somehow knowledgeable with the scripting and macros and everything that was everything that it was required back in the day. That's how I started and many people that are in the industry that they started in the mid 90s, early 2000 is pretty much the same story because now there are universities, now there are resources. Now you can have a certification with the Localization Institute and you can have also some certifications like in UX and all these things. But back in the day, it was just accidentally you studied either computer engineering and you just see what's going on there or translation career, and then everything is coming together. So that's how I started with Microsoft, that they were basically moving from Seattle, from Redmond, all their operations to Europe. And they opened a headquarters in the 90s in Dublin. I was working there. Then I was working for Lionbridge as a bender, working with different clients. I have very good memories of my days working for Nokia. When they were... Almighty and powerful and working for Lionbridge to provide the translation to all the mobile phones that they were doing back in the days and the LQA. It's been accidentally at the beginning and more intentional in the second part of my career, because there was one moment that around 2007, something like that. That I decided that working from a multilingual bender it's a great experience, but it's hard, it's a lot of pressure. Then I wanted to try to see, okay, it must be nice to work in a final client with not all the pressure off the working during the week and everything. That's when I started to reconsider everything, and it came an opportunity to work for Electronic Arts. That was in the year 2007. That's how I started in the games industry. Because before that it was the side and serious side of the industry, but around then 12 years ago, it was more moving to be the way I'm seeing now, electronic arts and working for FIFA and Full Speed. And it's the best of all worlds because you're like video games, you're like localization and it's like, wow, what else? And I've been working in the video game industry since then.
Adrian Cohn: I have fond memories of playing FIFA with my brother and friends in the 2000s. That's a great game. But it's amazing, the localization industry has changed enormously since the '90s. And you mentioned how it was accidental that you got into it and now it's a whole different ball game than it was before. The digital transformation that took place in the early 2000s, I'm sure played a big role in that. Now you're at King. Tell us about King. Tell us about your responsibility set and what does the localization program look like at King?
Miguel Sepulveda: So again, I am leading the globalization team. And what we do is, it's been the glue for other departments. And we are ensuring that we are silence breakers, that not everyone is working alone in the corner. And this is related with what I mentioned earlier, because nowadays, to create a mobile app or a mobile game, you need to have many different functions all together. And it's really finding the way to know with whom you need to talk and in which moment you need to talk, that's the key. So, in King, what we do is that we will cover all the spectrum of services needed, since a game is in the prototype phase until it's in releasing the market. And that could go from market research, because I mean, with the app store and with the Play store, we could localize into hundreds of languages, but actually, that's not the busiest, as you know. So just deciding, "Okay, what's the language coverage that we want for Candy Crush? Or what's the market privatization that we want for Farm Heroes or Bubble or all these games that we have?" Actually, that's something that you need to do some research and to be able to know, should we go to Vietnam or should we go to Thailand or should we go to another country? So continuously looking for on top markets and providing that to the game teams is one of the things that we do very early in the process. So, we are working hard to become like SMEs in the company and just not perceived as translators, because actually the added value that we give is the expertise that we are bringing in. I mean, in my team, we are 10 in total. We have people from different nationalities, eight different countries, if I'm not mistaken. So we have a good mix when it comes to give a recommendation from localization expertise. But also, we have in the team, native Japanese, Korean, Chinese. So we are also able to get feedback from a cultural perspective. So, we guide the game teams from the beginning until the very end. And at the beginning, it's more like with their research, which are the right markets? Which is the right font for a game? Which is the practice culturalization that we could do, because we have different perception of beauty in Europe or in Asia? We have different preference of game modes, type of games and all these kinds of things. So there is one part which is really becoming a by-source, in a way, and giving feedback. And then it's more like the execution itself. It could be the traditional localization process. And we are also paying a lot of attention the marketing campaigns and all the marketing assets that is top notch. So we do really pay a lot of attention again in the quality. As the company, their review is very positive about games in the app stores and very high; 4.7 average stars, so it's really good. And what we do as a company is to ensure that all the different pieces are working fine, because in the end, for a player, they will not say, "Wow, this information is amazing." I mean, it does not work like that. For a little bit, the other way around, we only know when it's not good. But this one, we have the combination of one game with art that visually is appealing, that there is no bads from a QA perspective: functionality is good, good music, good content. When you put all the different things all together is when you have a greater stream as supplier. And this is what we do. We want to ensure that our part in that whole process is really well done. And from the marketing part and for all those assets, they have a lot of visibility. We ensure that the videos are really good, that the test and the description in the game is really engaging, that we do good work providing localization for the keywords, because that's something that nowadays in the app store with so many games and apps, how do they find that for the different markets? So it's something that I also like doing, like localization, keywords, depending on the market, depending on the term of the game, because otherwise, how do they find you in Turkey if they don't really put the right wording? So when you think about it, it's many different things here and there. And when we are able to come all together and become the glue between all these different things, some of these different services, that's when we have succeeded as globalization team. So, my role is to ensure that all these things are somehow coming together and that it's working as much as possible.
Adrian Cohn: I love that you're positioning your team as not just the group that will execute on the translation, but also the team that does the research on the markets. The recommendation setting, or as you mentioned, you all are becoming subject matter experts on the different markets. So you're recommending the type base, that's awesome, by the way, all the cultural nuances that are going to be relevant to that market. You were just also speaking about how do you, how do you help the marketing team find customers in the different regions? By localizing keywords and figuring out what terminology would be used in searches to find your games and in different markets. That's really quite sophisticated, and it must be pretty rewarding as well to have that type of responsibility set with so many different business units in your organization. One of the things that I learned from your blog is that localization... There are a lot of complexities to it, and it is a team effort. And in a recent blog post, you posit how wonderful it would be if localization teams can use a Jedi mind trick to convince everybody to work in the way that you'd like to. But you came up with a more... Or you introduced a more practical system, a system that's widely known called the RACI system, or RACI framework rather. To enable your team and your colleagues, the other business units that you work with to understand that localization or globalization is a shared responsibility, and that it is the most effective way to keep projects on track. Tell us about your use of the... Tell us what the RACI framework is first of all, but tell us about how you're using it at King?
Miguel Sepulveda: The good thing with the RACI is that it gives a clarity about who does what and by when basically. So this is really helpful when you have cross functional teams and the matrix organizations. Because it may not be clear who has the ownership of some specific parts. And if that's not clear, it's going to be a difficult project. Because if it's not clear who owns the content or who owns the sign off of this part on that part? It's going to be a really difficult project. With RACI, basically what you do is... Who is responsible, who is accountable, with whom do you need to inform, with whom you need to consult. Because you don't need the same level of granularity with all your stakeholders in one project. You may be easily in a game, interacting simultaneously with 50, 60 people, and you don't need the same time to deliver and give the same type of information to a producer, than to a game designer, than to a marketing specialist. So when you get into discovery to say, "Okay, in this specific project, who is responsible to make a decision if there are conflicting priorities?" And you just put the EDR, the different goals, the different roles there. You put them in there. "Okay, if I'm talking about this aspect of game development, who has the sign off?" And you just cross mark that one and then you do keep writing about this. If things go in the wrong direction, who has the ownership of this? To whom we are keeping accountable? Because, that's important also, to keep people accountable of their actions. It's thinking, "Okay, who has the accountability for this part of the process?" And when you enter in the habit of asking yourself this from a private management perspective is when you start have some granularity. And then if you bring together everyone in the meeting and you say, "Hey, this is how we are going to work, this is what we are expecting from the different roles." Because we don't like any of FACI's unclear roles, unclear specifications, what's the role of this person versus this person? So when you take the time to plan that carefully, and when you reviewed that together with the rest of the stakeholders and you say, "Hey, are you comfortable with this subject, comfortable to be just in the informative?" Because we believe that this is something that due to the nature to make a decision yourself, then that's bringing a lot of clarity, so that's always good.
Adrian Cohn: When you use this at King, what results have you experienced? Is it something that you use literally every single time you get into a new game or are a new development? When and how do you use the framework?
Miguel Sepulveda: It's specifically useful with new games or with new teams? Because with the people that we've been working for a number of years, then it's clearer who does what basically. But when it's like a new game that is starting, it's a very good practice. To just start with those kickoff meetings, when we are starting to gather all the information and understand what we are doing and how we are going to work on the roadmap and all these things? It's the perfect moment. So new games and new teams is thel idea combination to put this technique into motion.
Adrian Cohn: Yeah, that makes sense. I'm particularly interested in the utilization of this framework. I actually have used it a few times myself for internal projects, like when we've scheduled or organized our global conference, or if we're trying to build a series of presentations. Internally, that will be used for external purposes. We typically use this framework to delineate responsibilities. I found it to be super helpful. One of the things that I feel that you also articulate on your blog quite eloquently, and I'd love to dive into in more detail with you is how globalization teams can think about their return on investment. That's the broad topic here. But you introduced this topic on your blog, first by talking about translation quality and how translation quality metrics are typically what localization teams look at to start evaluating the efficacy of their work. Why are quality metrics not always as useful as they may appear on the surface level?
Miguel Sepulveda: I think that we are looking... Traditionally, localization teams and the language specialists, we are looking at it from a very narrow perspective. Because, we tend to focus on like linguistic issues, or number of typos, misspelling or things like this which is not good to have. But in the end, what it does really matter is like in the case of Kim, the player experience. And the player experience? It's something that we need to find a different way to measure that. Because you could have one text that's from a linguistic perspective. It's totally okay, but that is not engaging. And maybe you have all the grammar rules okay, and maybe you have all the punctuations and everything there. But I think that one of the problems that is that we just keep ourselves liking this a model of this. This has no title. And we don't generally think about is this content really engaging? Are we transmitting feelings and emotions? And traditionally, this is not something that the localization team has measured. I mean, we would have all the KPIs that you might imagine about delivery on time and number of errors, number of words, all these metrics that the big guy wants. But at the end of the day, for all the companies now what is important is the customer, the player experience. So I believe that what's important is to find a way to capture that. To capture... What we do, what we translate, actually how is that perceived by a final user? By a final player. Because to be honest, I am taking for granted that the translators in normal circumstances, they don't make many mistakes to be honest. Especially we are making an effort to provide them the context of what they need to do. So when you are working with translators for a number of years and you provide the localization kits, reference material, context, they are quite, quite, quite reliable and they don't make many mistakes. So then the measurement of error rate, I don't really find it that useful. It's something we have to do, yeah, but ultimately the real metric is, is this content something that is engaging the players? And I believe that when we are able to find this kind of metrics, it's when we are having a different... an attentive discussion with the product things. Because for a product things, they don't really care either about the linguistic accuracy of Italian. What they would like to know is how we are doing in Italy. Are they enjoying what we are doing there? So we need to keep working to somehow come together from our traditional very nerd metrics to the product metrics.
Adrian Cohn: Yeah, think that is an interesting way to think about translation. Because while those detailed analytics are interesting, they really aren't what make the purpose of your business unit successful. It may determine failure of a business unit if you're not hitting those marks, then clearly there is something wrong with process and maybe vendor whatever. But the things that actually matter are are the customers buying your product? Are they using your product? Are they happy using your product? If you're able to tick those boxes, then it's easy to recognize that the efficacy of the globalization team, translation team is on track. And you talk a little bit about, in your blog, the importance of a single unifying metric, which is customer satisfaction. How do you approach measuring customer satisfaction at King and what can we learn from you as it relates to measuring customer satisfaction for localization?
Miguel Sepulveda: Actually, that's a product we are working as we talk. Because that's an area that we want to go there. I mean, at King, the player support team is quite, quite, quite pro. They are really good in the relationship with the players. And what we are trying to establish there also is to get sentiment feedback. So this is something that we are discussing with the teams to get feedback from international players. Because we have more research and more feedback for the English-speaking players. A lot of focus in the States, and what we are trying to do and to build now is a model that we get also some feedback from international players. And even asking them questions about the localization quality. Or even questions around, "Would you play this game if it's not in your native language?" Because when you go into those type of questions is when you really... Even if you don't have a error rate model, like a very solid one, if you have some way to show to the business to say, "Hey, in Japan eight out of ten people said that if the game is not in Japanese, they will not play the game." That's like an 80 percent. You don't really need to go to the ROI to do the tests if you don't want to because those results are quite powerful by itself. So it's getting this feedback from the players. It's also getting feedback from the internal customers. Something that we pay a lot of attention again is the surveys for our internalistic holders. So ever quarter, we run a survey with the different things that we are working and we ask them how happy they are with our services and what recommendations they have for us or which new services they would like to see that we offer. So we do that every quarter and we use that to monitor our performance on how the others are perceiving us and the combination of those both things is what I believe is telling you if you are doing a good job. Because if the stars review in the app store, in the play store is okay. If you have good feedback from this sentiment analysis and if you have good feedback from your internalistic holders, then things are moving into the right direction in my opinion.
Adrian Cohn: In the past, have you received feedback from internalistic holders that concerned you and that you had to address to improve on the way that your team was working with other teams?
Miguel Sepulveda: We have had some feedback to go deeper and explain more and better all this area of internal denationalization and it's one of the areas more unknown. And for many people they don't really know what they have to do to denationalize the code. So those are some comments that we have received, "Hey, it would be great if you can go deeper in this area or if you could run a presentation about this, about that, and if you could help with the market research of some specific regions of the world. Or if it makes sense to go to India or if we are fine with English in India." So those are the type of... In the other comments of the survey that sometimes we get feedback like this. And for us this is very good to have because actually it's much easier and we just hear what they would like and then we see how we can provide that. So I like the internal surveys. It's a good model in my opinion.
Adrian Cohn: Yeah, I love that. And I think I could actually learn from that myself. At Smart Learning, there's plenty I'm sure we could be doing more to listen to different teams at the company. What sort of experiences have you had at King where you walked away and you were like, "Wow, we completely killed it. This was a good result." What are you most proud of, I guess is the question, of your work at King?
Miguel Sepulveda: From a processes perspective, I'm proud that we are pushing localization as well as globalization activities that we push to the left. Because traditionally, when you start making the game development process, you will have the prototyping. Well, this is the same for any digital product. So, you would have the ideation and the brainstorming and the prototyping, and then some play-test. And then maybe some self-launch in some markets. And usually that's where localization teams are involved, when you start there. Which is more or less in the middle, if you take the axis, it would be in the middle from ideation and curation to the release. And we used to be more in the middle a few years ago. This is exactly the type of thinking about our services as an afterthought. They do something and when they need translation, they will send it to someone else. And we've been working to push there, and being involved in the co-creation phase, and tell them, "Hey, this is how we can have you in this specific area." And not being perceived as a kind of internal service provider, because that was the perception. I'm really happy that we were able to position ourselves. And the value, the impact, if we are engaging with our stakeholders much earlier. So that's nice. I like that.
Adrian Cohn: I think that's a great results. When we started this conversation, you were speaking a lot about how you've developed the team to be more than just an execution group for translation. We really are the thought leader inside the company on how products may or may not succeed in different markets and what needs to be done to achieve those goals. When you hired people to work on your team, did you have to look at different skill sets for that? Or have you developed skill sets, and people you work with, so that they have that expertise and they are not just translation or linguistic experts?
Miguel Sepulveda: One of the best things, again, is that they do really invest on us in education. We have access to many resources, many books, many workshops, many opportunities to grow. Certifications, if they are useful, we are also covered. Every year we have our own internal PDP, Personnel Development Plan, which is something that we do together with our manager or with our teams. And we say, "Hey, what would you like as individual grow?" And to come up with a first iteration with that. Then you try to see and align that with the needs of the team and the need of the company. So, a lot of the skills that we are getting now, it's a polishing, because in the last couple of years, we didn't hire anyone, and what we are doing is continuously evolve our team and try to expand our skills more horizontally. And in the end, we know by now how to do localization. It's more around the other areas, and we get keen support for that. When we identify a scale that we are missing and we believe that that's something that we should pay attention, we get the budget for that. So it's amazing. On that front, Kim is really helping us. And it's super useful because then we can... One good example of something that is getting a lot of traction in the world, it's everything UX related. Now, there are many areas related with UX, either with the design, either with, either with UX design for international audiences. So, for many of us, maybe we have done some kind of UX things, but we don't really know, or it was named in a different way. Because during all my life, the content house is Brighton, but we didn't know that that was called like a UX Brighton. We really don't know what's the difference between a content creator and a UX Brighton. And we don't know how a UX Brighton could impact the copy for the international markets. So this is a scenario that myself, I'm also investing time there, to get more knowledgeable in the UX part, because that's something that it's important. So, when we see a need and when we see an opportunity, and now we are identifying the opportunity of investing in UX, I'm thinking, "Okay, UX for English" But then how we are going to bring UX design of the different mock-ups, the different pop-ups? How we're going to translate this into the international experience? So I think that's a good example of something that is working fine in the way that we as internal team, we see a need. We also recognize, we are not really a strong in that area. But in the end, it's not that different from localization. Is just a different approach, but everything's related with player experience and how the players are perceiving, again, from different culture. There is a lot of common sense with the localization industry. And that's how we work, identify a need, and then create a PDP and then go for it, for getting more knowledgeable in those areas.
Adrian Cohn: That's awesome. And I'm sure your team is super appreciative of that as well, because the more skills you can add to your toolbox, the more productive you can be, and the more gratifying your day-to-day experiences. That's really cool. When you, when you taught Miguel with your colleagues about the value of translation, we were sort of hovering around this a few minutes ago when we were talking about C-SAT and the metrics that you look at. You also talk in your blog about how translation ROI is really not simple, and that you are still trying to crack the code on this, despite many efforts. Is ROI important for translation?
Miguel Sepulveda: I think it's not the right question to answer for a localization team, because there are so many variables that they have affecting them, that in the end it's putting every local professional trained to prove that, into a very difficult situation. Because it's not the formula, it's not that if you invest this, you get that. It's not. Because it's related with the player experience, and that player experience is very, very, preferential, and subjective. So it's more the question around, is player experience in this market important for this company? Yes or no? Because in the end, for most of the teams, if you compare the cost of localization with the cost of development, it's really small. I mean, really, really, really small. So it's going to be always positive from that angle. If you compare one mobile app or one game in one sprint, maybe you have 300 words, you know that translating 300 words is super cheap compared with everything that is required to create a digital product. So it's going to be positive, always. But the question is, can we succeed in this market without having localization? And is localization really enabling the business in that region of the world? Because if you want to target one country, in traditional industries, I think that you would not send salespeople to sell something to one country that they don't speak the language. That's not done. So if you want to sell in Spain, you're not going to send people that they don't speak Spanish, and you're not going to send people to a factory going to sell services if they don't speak the language. It doesn't work. I don't know why in this digital world it's a different approach because it is not like that. If you are selling pharmaceuticals, you have to speak German if you go to Munich to sell something, as simple as that. And if you don't do it, you don't make business. And in our industry there is a perception in web, in data products, mobile data, well, it might be nice to have. The reality is that it's not nice to have. I mean it's true, but just... If you just only choose and you don't do anything else, like more marketing and more support, it will not work, it's not a miracle. But the absence of localization is what is going to open or close the doors. So that's why I believe it's not really more making to the because there are many valuables and it's a difficult conversation, but it's more around . If in the Middle East, the English proficiency of the country in Saudi Arabia, if it's very low, why we don't translate into Arabic, they're going to understand it. So we are going to help the company to be in a better position to make money. So I prefer to have this kind of conversations, rather than to say the price per word, the price of healthcare hour, because that's... I don't like that. In the end I think that is just having bias vision of the industry.
Adrian Cohn: And it's also interesting because you write about how product teams have a lot of interesting metrics like retention rate, churn rate, active users, sessions. All of these could actually be looked at from the lens of translation as well, to get an understanding for whether or not the investment in a language is paying off. Let's say you translated a game into French, and you're seeing that those French users were continuing to use the product. You'd be able to say that the translations are effective. Whereas if those French users are starting to drop off the product, maybe you'd have... And other languages had users that were maintaining the service. You might have a way to look at, and evaluate, whether or not it's a successful investment.
Miguel Sepulveda: And Adrian on top of what you said, I think that another measurement very good to have is the A/B test. And that's my favorite one, much, much, better than AOI, because AOI is only telling one side of the story. But if you say, "Hey let's launch two products". One is localized and the other one is in English, and you ensure that it has the same functionality and you give them the same timeline to test. That's super powerful because then the differences is the added value of localization. So it does not really miss that much if we... If I'm investing one, if I'm getting 100, I don't think that's important, that's my opinion. But if you have the results of A, and the results of B, and you see that there is a big difference. And in all the A/B tests I've done during career, the localized version has performed better. Sometimes more, sometimes less, depending on the country, depending on the type of product. But I didn't see any single situation that we did one A/B test, and the English was outperforming the local. And those A/B tests, they are super powerful. And it's a shame that they are so difficult to run because it requires time, and it requires the commitment of a development. We are going to have the same features in these builds, we're going to segment users. So we distribute the A to these group of users, and the B to these group of users. So it requires a lot of internal coordination but when that's done, it's super effective because that's giving you really, the added value of what we do.
Adrian Cohn: How long do you typically run an A/B test for, what's your ideal timeframe? Is it two weeks, is it a month?
Miguel Sepulveda: The longer the better, because what is important is to avoid the false positives or the BS conclusions. If you do it one or two weeks, maybe you do not have a good look of the trend. So if you can afford four weeks, that's good. If you can afford six weeks, that's even better because there are spikes and, you need to normalize the results. So it does really depend on the team, but the longer, the better to avoid false positives and avoid coming to conclusions super early.
Adrian Cohn: Miguel, I feel like I've learned so much from you and I really appreciate that you joined the show today. We're going to make sure that in the comments... People, you got to go and check out Miguel's blog show locally, so we'll leave the URL in the description so you can find it. But Miguel, keep up the amazing work, keep on writing because we're learning so much from you in that channel. And I can't wait to see what you do next to King.
Miguel Sepulveda: Thank you so much Adrian, it was very nice. I really hope that this can be useful for the community and yeah we need a new, so let's keep doing it.
Adrian Cohn: I hope you enjoyed that conversation with Miguel as much as I did. If you haven't done so already, check out his blog yalocolizo.com. That's Y-A-L-O-C-O-L-I-Z-O dot com. Subscribe to his newsletter, you will not be disappointed. And hey, if you liked this episode of the Loc Show hit the subscribe button so the next episode will be waiting for you. If you loved the podcast, it would make my day if you left a review. Five-star reviews means more listeners, and more listeners means we'll continue attracting amazing guests to the show. If you're not ready to give a five-star review, send me an email, locshow@smartlink.com. I'd love to hear your feedback. Or give the next episode a shot. Thanks again for listening. See you next week.
