Software bug in vendor’s server caused disruption to SingPass system

SINGAPORE — Last month’s service outage on the Government’s identity authentication systems SingPass and CorpPass was caused by a software bug in the authentication server, parliamentarian Janil Puthucheary said on Monday (March 19).

The Senior Minister of State for Communications and Information added that the bug was not detected previously and only showed up after enhancements were made to SingPass and CorpPass in January.

On Feb 8, the disruption to the online systems, which enable users to access government e-services, lasted about six hours. Users were also locked out the next day for about three hours before the services were restored. The Government Technology Agency of Singapore had said then that both incidents were not related to cyber-security issues.

Member of Parliament Tan Wu Meng (Jurong GRC) tabled a parliamentary question on Monday to ask about the outcome of the investigations and the lessons learned from the incidents.

Dr Janil said that the bug was found in the server provided by Amsterdam-based digital security company Gemalto. While the enhancements to the systems in January complied with all technical specifications and were properly tested, the “interaction between the enhancements and software bug caused some records to persist in the systems, instead of being automatically removed 30 days after they expire”, he added. This was the root cause of the problem.

Explaining that the hardware backup could not address the unknown internal software bugs of this nature, he said: “We will review the system design to improve all-round resiliency beyond just hardware resilience.”

While the bug was elusive, symptoms such as the slowdown in system performance could have been detected earlier.

“Our early detection and warning capabilities can be improved and will be improved,” Dr Janil told the House, adding that the authorities will step up software checks and diagnostics so that engineers can act on the issue before users are affected.

The episode showed that for critical systems that rely on products from commercial providers, the authorities should work more closely with the providers to better understand and ensure that the product operate as intended, Dr Janil noted. This would allow them to improve the system design and the early warning mechanism.

“We will take these lessons and apply them to the development and maintenance of other Government systems,” he added.

On whether the contracts with commercial providers provide for liquidated damages, Dr Janil said that the authorities are reviewing the contracts with Gemalto as well as those with other commercial providers.