By Linda Bustos
Our friends at Wider Funnel are running an Ask Me Anything event this week. Ask your burning question at this link.
I got a head start with my question, gleaning some insights from the trenches from Wider Funnel’s Chris Goward and Alhan Keser on “what test cases are most difficult to test due to testing tool or ecommerce platform constraints?”
Chris shares:
Shipping options. (e.g. UPS vs FedEx vs USPS) are tough, but not impossible.
Product pricing. The challenge isn’t presenting different prices, but ensuring the ecommerce system charges the customer what was shown on the product page according the test variation. It’s possible, but very tough with most ecommerce platforms.
Category page filtering. If products aren’t already grouped in ways that we can get to them via the front-end, the testing team is limited to what is available already on the website.
Value of product reviews. If your product doesn’t already have reviews, it’s hard to estimate whether the effort of getting more reviews will pay-off via testing.
Increasing AOV with offers. For example, is it better to offer $15 dollars off shipping to everyone, or offering $20 off shipping if you buy over $50? Testing offers like this is difficult with typical front-end testing tools, but can be done with some back-end elbow-grease.
Social impact. It’s still difficult to track the full impact of social sharing. It’s easy to track the conversion rate impact of moving social buttons, but once a tweet or status update is posted, what’s the total economic value of that through the network effect? There’s no good technology for A/B testing this yet.
Alhan says:
Checkout step order While it is possible to test reducing the number of steps and simulate a single-page checkout using client-side a/b testing tools, it’s impossible with most ecommerce platforms to re-order the steps within your checkout process as ecommerce elements are hard-coded and cannot be re-arranged. (Linda’s note: one of the capabilities of Elastic Path Commerce’s Cortex API technology is the ability to load ecommerce elements directly into a CMS as individual components [vs hard coding] that can be swapped around in such a test case).
Shipping estimates. Bringing shipping estimates to the product detail page (or cart). Depending on the ecommerce platform, moving the shipping cost calculator to a page higher up in the funnel can be impossible.
Addition of a shipping option which encourages subscription of some kind. For example, say you wanted to test giving away free shipping in exchange for signing up to club… à la Amazon Prime. The complexity of such a test makes it impractical through conventional A/B testing tools.
Chris says “nothing is impossible, but some things are impractical when considering the amount of effort involved. WiderFunnel uses a PIE prioritization framework that considers the “Ease of Implementation” and de-prioritizes test ideas that may take more effort than easier quick-wins.”
Have a burning A/B testing question for Wider Funnel? Ask Me Anything closes this Friday.
Via:: Get Elastic eCommerce
