Testing of agile software development | KPMG | SE
Share with your friends

Testing of agile software development - Is automation a solution?

Testing of agile software development

The days when clients could wait for years between projects and upgrades are long gone.



Partner, Head of Treasury & Investment Management Team

KPMG i Sverige


Relaterat innehåll

Clients want to be agile and need the ability to react quickly to changing business needs, implement a new policy and comply with the latest legislation and regulation. Test automation is an essential element of being able to execute regular cost effective and repeatable upgrades.

At ION, as part of our development lifecycle, test automation and acceptance test-driven development are rigorously implemented while we build our software in continuous integration environments. Furthermore, our clients can use the same test tools and Test Automation Service for their own testing; this minimises the need for manual, error-prone and labour-intensive testing, which in turn significantly reduces the number and duration of downstream test cycles.

This is how David Atzeni, Director of Research & Development at ION Trading – now the owner of many treasury and commodity management systems – summarises the effect of transition to agile software development on testing. But it is not only ION clients that face the challenge of this change; more and more software manufacturers are converting to agile development, also in the area of treasury management systems, such as SAP or FIS.

For example, Dirk Joachim Henn, Product Manager for SAP Treasury, and Jean-Philippe Lombardi, Director of Agile Transformation, describe the approach of SAP as follows: 'SAP started its agile journey to succeed in the cloud many years ago. First and foremost, SAP applies SCRUM@SCALE in all programs in order to receive early feedback – also from end users in the course of design thinking projects. Further, the teams put criteria in place to ensure not only external, but also internal quality of the software. The latter is particularly important to keep productivity high.'

What is agile software development?

You can find a detailed description of the characteristics of agile software development and its effects in our December Newsletter entitled 'Impact of agile software development on TMS projects' here.

The overall conclusion is that manufacturers strive to respond more quickly to client or legal requirements and to be able to offer new functionalities by shortening their development cycles (sprints), which are combined into blocks. Usually, a sprint takes two to four weeks. At the end of a sprint, two things happen:

  1. The development is presented to the client, allowing clients to give their opinion on it right away.
  2. On the basis of existing development capacities and the pre-filtered topics ('user stories'), the manufacturer decides what will be developed in the next sprint. In the process, adjustments or changes are also considered based on client wishes expressed during the presentation.

For instance, six such sprints are combined into one block and supplied to clients. That means that businesses receive a bundle of new functionalities every few months (e.g. in the case of six sprints of two weeks each, every three months), which should be ultimately integrated into the production operation. For this reason, major upgrade projects for treasury management systems (TMS) will become increasingly rare in the future. These will be replaced by ongoing activities resulting in the TMS always being up-to-date. That means ongoing testing – essentially also for the specialist department, which needs to accept every new functionality.

What has testing involved up to now?

Before a new functionality can be adopted into the production operation, it has to be tested. In conventional projects, the following test phases are differentiated and run:

  • Developer testing at the manufacturer: this ensures that the software is supplied with the commensurate quality. All additional test phases take place at the client's location.
  • Functional or unit testing, in which only the different functionalities are tested on a modular basis.
  • Integration testing to ensure that all overarching processes, incl. interfaces, continue to function smoothly.
  • System performance and technical security testing, in which system capacities are verified on the one hand and the proper response in the case of system failures is examined on the other.
  • Acceptance testing (User Acceptance Tests) which ultimately decides whether the new version and/or separate functionalities of the TMS proceed to production. This final test phase is frequently also used to train users.

It is apparent that these conventional, comprehensive test cycles cannot be run every three months – at least not if they are primarily manually conducted, which is the case at many businesses.

What changes are thus necessary to testing?

The requirements arising from the transition of TMS manufacturers to agile development coincide with the frequently made good intention by businesses to improve testing (generally speaking this is made when it concerns creating the necessary test cases). In the course of a TMS upgrade or launch project, it has been shown time and again that businesses do not plan adequate time for testing and, in particular, for the preparation for this. As a consequence, test documentation and the associated reusability suffer in particular.

In order to be able to take up production of new functionalities at short intervals, standardised test cases are imperative. This cannot be manually worked out each time as this would tie up too many resources on an ongoing basis. Furthermore, new test cases need to be supplemented owing to newly added functionality. As a result, the prerequisite test catalogue is constantly being extended. Automating testing is thus a logical consequence. A reduction in the test scope or even waiving individual test phases increases the risk that errors occur in the production operation, which is to be avoided where possible.For the time being this means an increased effort in the form of a separate project to launch a tool to automate testing. This is divided into three steps:

  1. A decision is made on the test tool to be used. The TMS manufacturer, to the extent that it uses automated testing, is often the first point of contact. Otherwise there is the risk that the treasury management system is not compatible with the test tool. As a consequence, significant additional effort can be incurred through the integration of these two components.
  2. It is then necessary to define the functional business transactions which need to be depicted. This is made according to clear criteria, such as the frequency of occurrence, the significance for the entire business operation and the complexity. These functional business transactions are then turned into separate test cases which are automated. Examples of this are recording the deal and its processes in separate steps through to the activation of payments and posting, issuing key performance figures, analyses, exceeding limits, hedging, etc.
  3. In the final step, these test cases are prepared for the selected test tool. Each tool has its individual requirements, such as information, e.g. on the transaction (instrument, counterpart, amount, price, etc.) for which automated processing needs to be prepared. In general, these systems produce reports which show in colour (green or red) whether it was possible to successfully complete a test case. In doing so, the result is compared with the defined criteria, depending on the aim of the test. As an example, criteria can be the processing of a deal without an error message, calculated key performance figures compared with expected values, limit warnings being triggered. As a result, one needs only to analyse the test case containing errors during actual testing in order to find and remedy the causes.

It is clear that the testing of units, integration and performance can be automated, as a minimum. As already mentioned, acceptance tests are also used to train users. Here too a fundamental rethink is necessary. Owing to the short intervals in which new functionalities are made available, there will no longer be major changes between the old and new release of the treasury management system. This is an ongoing process consisting of many small steps. As a result, learning constantly takes place on the running system. It is useful if the users concerned already participate in the manufacturers' presentations. Naturally commensurate documentation and communication by the manufacturer are imperative.

Why is Treasury affected by this?

The earlier mentioned trend for switching over to agile software development will not spare manufacturers of treasury management systems. Treasury should accept the changes that this brings at an early stage. The introduction of test automation is a separate project which ideally should not be combined with an upgrade project or a new launch as this would significantly increase the project risks.

Andrew Owens, Group CTO for treasury solutions at FIS summarises the current situation and approach. His statement can also be understood as a call to treasury to actively get to grips with this topic: “FIS uses agile development across our Treasury and Payments development groups. We also have a high degree of alignment across these groups. The combination of agile processes with our ‘Continuous Integration’ frameworks is helping us to increase software quality and development velocity, whilst providing our product management team with a large amount of flexibility on setting and adjusting the product roadmaps. FIS has a mature ‘Continuous Integration’ framework in place for our treasury and payments solutions. The scope of this framework includes automated build and testing, where we run many thousands of tests at least daily. We use a variety of tooling to assist with unit testing, GUI testing, integration testing, end-to-end testing and security vulnerability testing (static code scanning). The FIS testing framework is in place internally and we are also offering it to our clients in case they want to utilise it for their testing requirements.”

Standardisation and automation of processes and end products require a fundamental rethinking of how treasury is organised: which products shall treasury offer? Which processes produce these products? Who is responsible for these processes? Which resources are required and used for this process? Which data and methods are necessary? What benefit/use will it produce? Ultimately, the treasury operating model is generally facing a comprehensive realignment (more or less depending on current setups). Crucially, processes will be structured depending on their character into strategic or managing, executing and monitoring processes. Compliance aspects (e.g. separation of functions and internal controls) will be taken implicitly into account. In the second step, the potential for standardisation and automation will be evaluated and then assigned to the corresponding platforms, consisting of databases and systems. By also taking into account local processes and specifics, a group-wide, clearly-structured operating model in accordance with Treasury 4.0 emerges. In this way, it will specifically become transparent which treasury services will be offered as standard products. Or on which products the treasury consultants have to individually advise their clients in procurement, sales, HR etc. with highly specialised knowledge.

This has similarly important implications for the skills profiles of employees. If processes are digitalised, operating, manual activities are hugely reduced – whereas IT skills and analytical abilities become the new factors for success. Structuring and evaluating data, using data-analysis methods and deducing actions to be taken will become important areas of responsibility.

Social skills for communication, conflict resolution and managing discussions and leading negotiations will also be of great importance. The treasury employee's role will require a lot more communication with colleagues from different regions and cultures. Focussing on these topics is thus of significance, because the training and development market for treasury management seminars continues to prioritise technical issues. Qualification and training, exchange within the network and analysing technological developments are high up on the development agenda for treasury employees.


Digitalisation is one of the game changers of our age, and it will have commensurate significance for treasury. Clear, structured treasury processes and products represent the basis for digitalisation. This goes along with appropriate investment in technology and methods. If responsibilities for processes, the end products and their costs and benefits are transparent, change requirements and their impacts can be clearly defined – and more easily implemented. Treasury will become manoeuvrable and will be able to act faster – from phasing in new products to mapping new reporting requirements. At the same time, treasury will be made fit for the requirements of the future – in whatever form this may be in the next decade.

Kontakta oss