Singtel fined S$1 million over 2024 landline outage that affected 500,000 users and emergency call services

SINGAPORE: The Infocomm Media Development Authority (IMDA) has imposed a financial penalty of S$1 million (US$770,000) on Singtel for a landline disruption last year that affected half a million users.

The fixed voice disruption on Oct 8, 2024, affected about 500,000 of Singtel’s residential and corporate users for more than four hours, during which the public’s access to customer service lines for some government agencies, healthcare organisations, banks and companies, and emergency call services were affected, said IMDA on Thursday (Dec 11).

Singtel said in response to CNA's query that it accepts IMDA’s ruling and the financial penalty.  

"We recognise the seriousness of the disruption to our fixed voice service last October and the concern and distress it caused our customers and members of the public," said Singtel Singapore CEO Ng Tian Chong.

Mr Ng added that the telco took "immediate remediation action" after the incident by improving its network design and hardware configuration, as well as enhancing monitoring processes. 

"We also worked with our clients to strengthen their customer service hotline resiliency," he said. 

"Network reliability and resiliency (are) top priorities for us and we are committed to strengthening our systems, so that we can continue to provide reliable essential connectivity services to our customers."

INVESTIGATION FINDINGS

IMDA's investigation found that Singtel hosted two separate virtualised firewalls on the same hardware - one for its fixed-line voice system and another for the monitoring system for home broadband routers and Pay TV set-top boxes, read the media release.

“This meant that both virtualised firewalls shared the same hardware memory resources,” said IMDA.

“As the monitoring system’s virtualised firewall did not have adequate filters installed to protect the hardware against high-intensity traffic, the shared memory resources of the hardware were overwhelmed when the monitoring system faced increased traffic intensity on Oct 8, 2024. 

“This caused the virtualised firewall of the voice system to also malfunction and operate intermittently,” said the authority.

IMDA said that voice traffic from the affected voice system should have been fully and seamlessly redirected to a separate voice system at an unaffected site through an automatic failover mechanism.

However, the failover did not happen seamlessly due to the intermittency of the affected voice system’s virtualised firewall, leading to calls being dropped intermittently as voice traffic alternated between the affected and unaffected voice systems.

“The incident was resolved only after Singtel fully swung all voice traffic to the unaffected voice system,” said IMDA.

In determining Singtel’s fine of S$1 million, IMDA said it took into account the scale and impact of the disruption, as well as the time taken to restore the network.

It said that the impact of the incident was "significant", as it affected the public’s access to customer service lines for some government agencies, healthcare organisations, banks and companies, and emergency call services.

“The potential impact on the safety and security of the public could have been very serious,” said IMDA about last October's outage. 

“IMDA’s investigation concluded that the incident was within Singtel’s control to prevent,” the authority said, adding that the incident was not due to a cyber-attack.

Since last year’s incident, Singtel has implemented separate hardware for its voice system and monitoring system; and an intervention mechanism to stop network traffic from alternating between its systems during failovers.

“IMDA holds telcos, which are key service providers, like Singtel, to a high level of service reliability,” said the authority.

“These service providers are required to plan, design and operate resilient networks, and put in place measures to ensure speedy recovery and minimal inconvenience to end-users in the event of a disruption.”

Other key service providers are also required by IMDA to conduct checks on their systems to avoid similar configuration weakness and to remediate where necessary.

“IMDA will validate the checks and remediation undertaken by the service providers,” it said. 

Under the Telecommunications Act, IMDA may impose a penalty of up to S$1 million or up to 10 per cent of the service provider’s annual turnover. 

“IMDA will not hesitate to take strong action under the Telecommunications Act, including imposing financial penalties, should any lapses be identified.