Nexon’s commerce system was beginning to show its age after operating for over ten years. Our commerce platform served as a billing service and a shop for game studios to sell exclusive in-game items. If game studios wanted to sell products, they would have to route all traffic through our global site instead of their dedicated site because the billing system was not modular. Gamers needed to leave their game to make a purchase.
Users needed to leave their game to make in-game purchases. If users wanted to reload their funds from inside the game, they would have to leave their game to go to our website to add funds. This would cause the game to freeze or crash. We were receiving 15-20 tickets a week regarding game crashes. Furthermore, we were launching a shop page for game studios to sell in-game items on our website, and the current checkout was scrapped to build a new checkout that would work for both the web and in-game.
Before I started to push pixels, I needed to understand what was wrong with the old checkout and why it was scrapped. I interviewed game teams and asked what were some of the issues that they commonly experienced. It turns out that the old checkout system had backend issues and could not integrate balance information of prepaid game cards. This meant I had the opportunity to redesign it from the ground up. I started researching how other game platforms were handling their checkout and comparing it to our capabilities.
After implementing the revamped pages, we gathered analytics data to see how it performed.
This was good news, but we wanted to dig deeper into the data to understand the usage for each payment method. To better understand if users were trying to learn more about payment gateways, we looked at the data for users who looked at an FAQ.
I hypothesized that we could add short messages on our payment gateway buttons to give a context to users on what forms of payment methods are accepted on the payment gateway so that they would rely on the FAQs less. If users relied on FAQs less, then there would be a lesser chance of them contacting customer service.
We conduct an A/B test with three variants:
Contrary to our hypothesis, variant A performed the best as shown in the results below. I hypothesize that this could be due to the amount of time it takes users to read each description and possibly cause them to change their mind. The new checkout also eliminated the tickets that came in about users' game crashing because they were now able to checkout while playing the game.
better performance over other variants
probability to be the best variant
probability of beating other variants