Live trialling

Maeña Twomey, programme manager, Lloyds Banking Group, looks at the issue of enhancing assurance on large cross divisional change programmes

The growing trend for larger, more complex change programmes calls for an elegant approach to test subsequent target operating model impacts. Live trialling is a way to do this by enhancing standard testing to prove the operating model post-implementation in a low risk manner

Traditional testing assurance

It is widely acknowledged in change programmes that issues are exponentially more expensive to address the later in the life cycle they are found. None are more costly than those problems found post-implementation. While this principle commonly refers to system defects the same applies to operating model issues. Operating model issues include SLAs, team-to-team interfaces, organisational flow of data and service and critically customer experience.

Traditionally organisations will replicate production, as far as possible, by creating a pre-production or test environment. This aims to prove the technology and usability. However, a production environment has uniquely ‘evolved’ and therefore only in rare/costly circumstances can you fully replicate this. In addition, what is generally overlooked is the requirement to prove any new processes end-to-end and therefore the target operating model.

Traditional testing assurance:
• Gather requirements
• Design and agree the solution
• Build the technology
• Test the solution
• Train and communicate to the users
• Roll-out

Shortcomings in the traditional model

While widely used, this model has shortcomings when it comes to large, cross divisional change programmes where there are a significant amount of cross department dependencies.

Key shortcomings are:
• Limited proving of inter/cross divisional hand offs
• Limited proving of SLAs (both internally and externally with third party suppliers)
• Limited proving user and customer experiences
• Limited proving all of the above in a production environment - certain aspects of a production environment are inherently unique e.g. reference data and batch processes.

Issues resulting from these are generally customer facing and therefore have an impact on an organisation’s reputation and worst case scenario its regulatory compliance. Take, for example, retail banking account opening for a new product. Looking at the components involved in this:

Customer expectation:
• Open an account via telephony, online banking or in branch.
• Account opened within agreed timeline
• Card, cheque book, welcome letter, PIN etc issued

Now look at this same process from an operational perspective:
• Open account is initiated via different systems\routes and passed to a single back office team
• This team must then deal with the opening of this account and pass it onto additional teams e.g. fraud
• This account is opened and then internally new letters are printed and issued
• At the same time data is sent to an external third party to produce the card
• Within this process you have numerous internal and external dependencies. Teams are all dealing with material (the product) and data which must be seamlessly passed, worked upon and understood all the way through the process.

This is a complex array of change, all of which is customer facing. In order to be successful there is one major dependency, all departments and third parties have worked seamlessly throughout the lifecycle and understood all requirements exactly. As we know, this is simply not the case, hence the need for traditional testing assurance. However, this model falls short when proving customer and operational aspects end-to-end.

A better way - live trialling overview

Standard project lifecycle exists because we will always find issues with our design and build of technology and operating models. The traditional project lifecycle and associated testing assurance is not entirely aligned to addressing issues realised when the technology and operating model are brought together in production and the level of change is significant.

So what can we do to address this? The answer is to build on a simple idea commonly used in small single change projects - the pilot. Rather than simply pilot a system in a ring-fenced simulated manner, look to extend this concept to include a customer and user experience in the real world.

Look not at the minutia of the detail, for example, the screen layout etc, look instead at the bigger picture, treat the systems as black box. Operate in a controlled post-implementation situation and monitor this, i.e. carry out live trialling.

Implementing live trialling - live trialling principles:
• Your production environment is inherently unique and simply cannot be exhaustively replicated:
• All traditional testing assurance (system test, UAT etc) is completed;
• All traditional implementation proving is complete e.g. training;
• Your live trialling is outcome focused - you identify key outcomes of a process and prove these. The key assumption is that all underlying functionality is stable.

Live trialling approach:

Step 1 - Maintain the status quo
• Deliver your change in line with the traditional testing assurance model
• Fully test and implement your change

Step 2 - Know your ‘moments of truth’
• Identify your critical business processes, inter divisional hand offs and corresponding SLAs. Use your frontline and business subject matter experts to identify these
• Identify the outcomes you wish to prove to gain assurance and the checkpoints in the end-to-end process

Step 3 - Manage the risk
• Create the ability in your production environment to operate the system end-to-end in a ring-fenced manner (e.g. in a retail bank, create a new sort code against which customer accounts can be created and easily managed; alternatively creating a data format which you can easily manage)
• Confirm the readiness and support of your third parties in doing this.

Step 4 - Start small and build up
• Use the real users and not testers to carry out the proving
• Carry out your business processes in a BAU manner
• Start with small and simple levels of assurance and increase the breadth and complexity as your assurance increases

Step 5 - Monitor the outcomes
• Monitor the outcomes in three areas: Customer and user experiences, technology outcomes.

In summary

While there is no substitute for traditional testing in carrying out live trialling you will have realised extra assurance for:
• The integrity of your target operating model design
• Your cross divisional hand off and SLAs
• The integrity of your production environment when new data is introduced
• Your colleagues’ readiness to accept this change
• The readiness of your third parties to meet your requirements

Live trialling is inherently simple, low cost and low risk to implement as you leverage existing implementation activities to deliver this. This phase will bridge the gap between traditional change assurance and what is required for large cross divisional change programmes.

    Share Story:

Recent Stories

New Business Frontiers
FStech’s Mark Evans discusses the future of financial services with Liu Jianning of Huawei, covering the limitations that current thinking can impose, how financial institutions can embrace technology to be both agile and resilient, and making space for the organisation to focus on the job of creating innovative business models and on delivering business value for their customers.

The Future of Intelligent Finance
FStech Group Editor Mark Evans sits down with Jason Cao, President of Global Financial Services Business Unit, Enterprise BG at Huawei ahead of its Intelligent Finance Summit which was held on 3rd and 4th of June in Shanghai. This Q&A delves into key trends in digital transformation of the financial services industry as well as a look at how data, robotic infrastructure, intelligent storage and innovative technologies are shaping the future for FSIs.

Cracking down on fraud
In this webinar a panel of expert speakers explored the ways in which high-volume PSPs and FinTechs are preventing fraud while providing a seamless customer experience.