Top Online retail giants such as Amazon, E-bay, Alibaba are trailblazers who defined digital commerce. From a pure website-based commerce more than a decade ago to mobile apps, commerce platforms customer experience has gone through a big upheaval. Customer journey is only getting complex and it’s a great challenge for online retailers to keep up with the shopping behaviour and expectations. This is forcing the digital commerce retailers to act like a software company specializing in retail. Software is eating the world and it is also gobbling up the IT budgets & resources.

The online customer journey gets split into many journeys. It begins from the customer researching/browsing the online store, showing interest on a product, payment, returns/exchange. At each stage different software, systems, platforms come into play. For a CIO or the e-commerce head, it is important to have a holistic view of the QA testing strategy. What this means to have a test automation framework specializing in online commerce that starts with the customer facing aspects of the systems such as device functionality, UI/UX and back end systems like inventory, merchandising, ERP, payments to name a few. Once an integrated testing approach is determined, there can be an expectation of seamless customer experience. Since customer journey is across systems, testing becomes key at each stage of the process.

What does digital first testing mean in a digital commerce context. It means developers design with gadgets in mind meaning mobile devices & screens of all kinds, engaging UIs, AI based devices such as Alexa, Google Home. Bigger screens such as desktops & laptops come next but priority is given to gadgets. So testing needs to be more comprehensive across digital devices due to variety and varying customer journey across these devices.

So, what are the 3 things a CIO or a QA director needs to keep in mind when deciding for digital commerce testing strategy

Continuous delivery testing.  Functional test automation should be across channels and needs to be integrated with CI/CD cycles easily. Functional test assets need to be leveraged with performance testing automation in the same pipeline.

Smarter testing and at scale: Automated test orchestration, test execution at speed & scale. Bring AI & ML into the picture with smarter test designs.

  • CX Testing: Bringing in testing criteria and scenarios for customer experience evaluating both developer and business use cases. Shift left scenarios by leveraging production data/information.

Now let’s go through the key customer journey stages and how testing comes into play

Digital asset testing:

The e-commerce experience begins and ends with devices/gadgets and/or an ecommerce site. These include many hardware, software and other technology configurations which include devices, OS versions, browsers which keeps changing. A testing strategy using the right digital QA framework is very key.

Compatibility testing

Testing compatibility between browsers, OS, mobile devices including customer journeys which break and continue in different devices or OS or browsers.

Functional testing

Testing basic features such as menus, links, and forms. Complex features like database connections, vendor or seller integrations or partner integrations.  All the ecommerce web pages touched by the customer journey needs to be comprehensively tested.

Performance testing

Performance testing needs to be performed early in the development cycle to ensure smoother customer experience. It should be comprehensive and structured. Key is pin down the variables, scenarios to test and analyse the right data. For example, making sure website is tested during peak shopping hours, season sales.


Payment testing is one the most important aspects for any ecommerce sight. Insufficient payment testing can lead to cart abandonment and can direct affect business performance. It needs to even accommodate user behaviour scenarios wherein user can make errors. The shopper needs to feel secure and sage while entering payment information safeguarding data privacy.

Security testing

This is an integral part of e-commerce testing. Both mobile and web applications should be tested for potential vulnerabilities.  Maintaining proper security practices is very critical as online merchants put themselves and their customers at elevated risk for payment fraud. Some of the key security threats online retailers need to aware of are phishing attacks, Credit card fraud, cross site scripting, DDoS attacks, Bot attacks.

Big data testing

Big data testing can produce results only when quality of the data is good. It is imperative to have a big data testing plan by recognizing and identifying defects and inconsistencies in the data early on. This will contribute in reducing cost and achieve business objectives. One of the major challenges for big data testing is the expertise to write big data test scripts. Finding skillsets to write this is very difficult to find and retain.

Regression and automated testing

Regression testing can be carried out either manually or through automation. When code changes are frequent then it is recommended to go automation. This is the case of for most of current digital commerce sites due to the frequent changes that happen to the site. Some of the key functionalities that needs to be checked are login functionality, third party integrations, modules that are impacted by the frequent software releases. Automated regression testing helps in saving time between releases and increases ROI.

Usability testing

Only half of online retailers report they use Usability Testing to optimize ecommerce performance. According to a recent article by eConsultancy, global ecommerce sales will be over $4 trillion by 2020. Online retailers who optimize their user experience will have a better chance to get ahead.  Online retail is getting transformed with the advent of modern technologies and increasing customer expectations. Usability testing is an integral part of providing a superior user experience.

For the CIO, it is imperative to choose a test automation framework to achieve the above objectives as a huge amount of money and efforts are invested on new technologies and tools. It is critical to objective track development and testing process to achieve the digital objectives. Test Automation frameworks like AFTA not only reduce testing efforts but ensures maximum test coverage. Making sure the performance, functionality, security, and accessibility of the application while leveraging digital technologies is very much required but QA/testing needs to be at the core of any digital commerce business.