Dishonest or incompetent company
We worked with Conversion Rate Experts on conversion rate optimization during 2024 and 2025. Because our website receives a high volume of traffic, they decided to run most A/B tests for only two weeks.
Many of the tests appeared to produce strong results, and several changes were declared winners and implemented across our website. However, despite some tests showing substantial improvements over the control, we did not see any corresponding increase in overall sales. As the expected benefits never materialized, we eventually ended the partnership.
We later began running A/B tests internally and discovered that test results can fluctuate significantly over time. In particular, results that appear strongly positive after two weeks can later reverse and become neutral or negative.
We also became aware of the novelty effect, whereby existing and returning users may temporarily respond positively when they notice changes to a familiar website. This does not necessarily mean that the same changes will improve conversion rates among new users or continue to perform well over the longer term.
In retrospect, we believe that several tests presented to us as clear wins would likely have shown neutral or negative results had they been allowed to run for longer. As specialists in conversion rate optimization, Conversion Rate Experts should have had the experience and expertise to recognize this risk and account for it in their testing methodology.
Instead, we were led to believe that a large number of tests had produced genuine and sustainable improvements, when our subsequent experience suggested otherwise. In our view, this reflects either a serious lack of competence or a failure to present the results transparently.
For these reasons, we cannot recommend Conversion Rate Experts.
