Server-side testing is a technique. It is also a philosophy. The technique is straightforward. You can A/B test anything that is directly pulled from or run by the server, whether it is different web pages, different price points or service offers, or different search algorithms. The philosophy is sublime. Server-side testing can cultivate a culture of experimentation in the organization. Any small change made to gain an incremental improvement can be tested upon release. That makes experimentation routine, developing an experienced core of marketers and back end developers who can question and test effectively.
EX for UX
Digital marketing ultimately comes down to the user experience. Improving it should lead to more customers and increased sales. Some will do this through client-side testing, where changes are made to the appearance of the web page to see what gains are triggered. In contrast, by testing on the server side, “we’re able to test richer types of ideas, do more than change the visual user experience,” said Nick So, vice president for delivery at WiderFunnel.
Client-side testing only requires a change in the web page’s code, which can be done with a WYSIWYG editor. “Server side is all code-based built into the app,” added Tom Davis, senior web developer. “You’re implementing with a tool set, software development kit, or your own solution, and there is a lot of testing throughout development,” he said. “There is more in-depth testing of the code because you need clean code for the solution to work. Even the roll-out is treated as a test,” he added. You are testing more deeply on the server side, noted Jon Noronha, senior director of product at Optimizely. Performance APIs, chatbots, conversation interfaces, pricing, mobile apps — all are possible from the back-end. But this is the realm of the back-end developer. “They are expensive, and it’s hard to get a lot of them,” Noronha said.
“[W]hile this workforce is not cheap, we find many customers and prospects see this investment has already been made or is in the process of being made, as they cannot build any modern, cross-platform solution without such a team,” said Omri Mendellevich, CTO at Dynamic Yield.
A culture of experimentation
“Most sophisticated development teams make it a rule that there is no no new feature or change unless it is experimented on first,” Noronha said. He contrasted this approach with his previous experience at Microsoft, where a “waterfall mindset” resulted in years between major product releases. It was not until a product was released that anyone could see what happened.
Constant experimentation changes that view. “It’s a cultural change, more time-related. Small changes are frequent — a week or two weeks,” Noronha said. The mistakes, when they happen, are smaller, and more likely fixed easily. “It’s the essence of agility, more than scrum is,” he said. “The process of server-side testing is very baked into our product and engineering processes,” said Nate Stewart, head of product strategy at BigCommerce. “[T]he step of performing a split test isn’t just the last box to be checked with shipping a product — it’s a checkpoint as we iterate that helps signal whether we’re on the right path. The key to a culture of experimentation is viewing every test as a learning opportunity.” Stewart continued. “Everyone likes to be right, but what’s more important is being open to observations that produce a better product in the long run.”
From hypothesis…
Server-side testing allows a marketer to test everything and anything. That does not mean they should. “You need vision and a point of view,” Noronha said. “A/B (testing) is misunderstood as mindless tinkering without a clue,” he said. “What it should do is provide validation and guidance.”
“In terms of server-side product testing, the question that you ask should always tie back to that user need.” said Stewart. “It’s also critical to build a hypothesis out of a well understood funnel. You should be tracking performance/completion rate of the user journeys you want to improve before you start setting up A/B tests.”
Taking a different angle was Matt Gershoff, founder of Conductrics, who suggested a different tack: it’s not a clear choice between client-side or server-side testing. “What is the task? What is the problem?” he asked. “Server-side versus client-side is the wrong way to look at it.”
Gershoff admits to being agnostic about technology, preferring to view a marketing challenge as a problem to be solved. “Can I solve it in an environment that is probably live?” he asked. “Definition is most important. The solution lives in the problem.” The approach used by Conductrics is an intelligent learning agent that will make decisions based on a goal, Gershoff explained. The system is not learning, but optimizing its decisions to get to the goal.
…to synthesis
Digital marketers, with their affinity for data, are naturally drawn to server-side testing, but lack the tech skills of a full-stack developer, who can best access data to craft tests. That means turning to the company’s IT department, noted Paul Bernier, director of product management at Sitespect. “We’re seeing developers getting more involved on the marketing side of the house.” he said.
“You need to go into the server to alter some logic into the code base to test a strategy,” Bernier continued. There is a risk to this, since you have to find out if the new change works better than the process previously in place. “Testing can solve that,” Bernier said.
For the developers, there is the added plus of using their skills to solve a problem, which adds value to the IT department that goes beyond the simple “fix what’s broken” mission, Bernier added.
“Unlike classic client-side testing, which fits “over” an existing stack with very limited interaction, a server-side solution needs careful planning and oversight – by R&D management, architects, and team leads.” added Mendellevich. “Without a strong bunch of developers, it’s very hard to deeply integrate server-side solutions robustly (not just for testing).”
