DSE clarification and our reply

The Dhaka Stock Exchange (DSE) has sent a clarification to a report published in The Business Standard on 2 November 2022 under the headline "What causes frequent trading disruptions at DSE."

Following is the full, unedited, text of the clarification and our reply:

Published report: The country's main bourse is running its huge operation with only one data centre. The trading could have resumed within a maximum of five minutes of the crash if there was a backup server – secondary hardware that has the same configuration and function as that of the main server.

DSE response: There was no server crash or hardware failure happened on 24 October. The trading system runs on multiple servers where we have reserved and work as failover i.e. system is self-resilient. As a result, we have not faced any hardware failure since December 2014. If resiliency fails, then it is possible to restore from DR, if it exists. On that day, unusual behavior from one of Broker Hosted Order Management System (BHOMS)suspended/halted the Matching Engine (ME) as it is designed as such. After investigation, Nasdaq Engineers has found the Killer transaction and they trimmed the transaction and rectified the ME that took longer time than usual. So, if we had DR, we had to do the same action and more time would be required to start the market. There is no example in the world that trading system can be restored in maximum 5 minutes' time.

Our reply: This was the second time the DSE faced trading disruption due to user overload and technical glitches in the Nasdaq matching Engines. The backup server could have helped the DSE restore trading soon after the main server had crashed. DSE trading remained halted for 152 minutes on 24 October this year and for 111 minutes on 18 July 2021.

Ziaul Karim, chief technical officer (CTO) of the DSE, told The Business Standard that "We have been trying to set up a backup server since 2016, but the board approved it in 2021. Its work is ongoing."

The Chinese consortium of Shenzhen Stock Exchange and Shanghai Stock Exchange offered DSE to replace the matching engine with their platforms but DSE is yet to respond.

Despite repeated technical glitches, the DSE neither took action against Nasdaq nor took any initiative for a replacement.
 
Published report: The DSE continues to use an outdated server instead of replacing it with an upgraded version, leading to frequent server downtimes owing to different technical glitches.

The stock exchange procured 120 Dell servers with very high configurations for Tk20 crore, but it has not installed those in a year, according to sources at the DSE.

DSE response: DSE already updated OMS system with new 25 DELL servers for OMS System at Motijheel and shall continue with existing servers of ME as they have warranty with HP.

It is here noted that DSE almost completed the Data Center facility and powered on all the server systems. We have housed all the remaining 95 DELL servers at Nikunja Tower Building Data Center where we would be doing implementation of ME & OMS system in collaboration with Nasdaq and Flextrade.

The DSE's ICT division favored a particular vendor OneWorld InfoTech (OWI) while awarding work orders of all technology investments, including implementation of IT projects.

Our reply: Until September 2022, the DSE did not install 120 new Dell servers even after nine months of delivery. Those servers were procured for replacing existing old servers at Motijheel.

On 1 November, the DSE chief technical officer told TBS that they could not install the new Dell servers due to delays in the DC project.

But, now the premier bourse in its rejoinder claims that they installed only 25 Dell servers. Still, 95 Dell servers remained idle.

DSE response: The DSE's ICT division didn't favor any particular vendor. We have many other vendors working with our DC and nDR project. ICT Division prepares the specification of the goods or services for IT project, then the Procurement Department and DSE Purchase Wing independently select vendor to issue purchase order following the procurement manual of DSE.

Our reply: We mentioned that the ICT division of the DSE was favouring a particular vendor in many technology investments including the data centre project.

For instance, the work orders for building Control-S in the data centre project, Busbar Trunking system, LAN Cabling, matching engine and order management system, purchasing 120 Dell servers, purchasing Security Information and Event Management (SIEM) cybersecurity solutions, and cyber threat solution were awarded to OneWorld InfoTech (OWI).

Moreover, in some cases of awarding work orders, the ICT division of the DSE twisted RFPs (request for proposals) eligibility sections to allow OWI to participate and eventually get the deal, according to a show cause notice served by DSE's former managing director Tarique Amin Bhuiyan on CTO Ziaul Karim.

For instance, for the RFP for 120+ servers, the ICT division allowed the bidder to just submit the letter of authorisation from the OEM (original equipment manufacturer) as opposed to regular and more standard practices of having recognised partnerships with OEM, according to the DSE document.

With this eligibility criteria, OWI was eligible to participate in the tender just by submitting the letter of authorisation from Dell for such a huge deal of over Tk18 crore when OWI didn't have any experience with Dell's support and services. According to the DSE document, for purchasing six servers the eligibility criteria for authorised partner/reseller for the offered brand was to be the OEM Certified partner/ reseller. However, OWI only submitted just an authorization letter. The authorization by Huawei doesn't cover any liability of Huawei. Despite not having OWI was awarded the contract without any protections on support and warranty, etc. from Huawei. The very purpose of requiring a certified OEM partner is to ensure DSE will get the very best of post-implementation warranty and maintenance support. In fact, there is no SLA (Service Level Agreement) with either Huawei or OWI which is absolutely normal and a necessity for any such hardware deal, according to the DSE document. For the RFP for an advance cyber threat detection solution ICT division chose Darktrace Enterprise Immune System supplied by the same OWI. However, they didn't have this solution implemented anywhere in Bangladesh which was a requirement. Despite that ICT division gave a "Comply" rating to OWI which eventually help the company to get the deal.

Published report: The Chittagong Stock Exchange, the Central Depository Bangladesh Limited, and all banks have backup servers, but the DSE is running with only one data centre.

The DSE invested Tk150 crore for establishing the main data centre after shifting its office to the DST Tower in Nikunja, but the data centre project has already seen its deadline extended two times.

The chief technology officer is the chairman of the steering committee of the data centre project.

At present, the DSE is using its old data centre in Motijheel. It also took an initiative to establish a second data backup server on the fourth floor of Abdul Monem Tower in Dhaka city, but it could not start work yet.

However, the DSE has been paying Tk4.5 lakh as rent for the floor in the tower for several months, according to DSE sources.
 
DSE response: DSE has issued PO to a Consortium on June 06, 2021 for Data Center (DC) Construction at Nikunja Tower Building and the project work started on July 15, 2021. Former MD, Mr. Tariq Amin Bhuiyanjoined DSE in July 25, 2021 and after his joining the DC implementation work was halted by him for 2 months due to lack of knowledge and incapacity and also mismanagement during his tenure that triggered further delay.The work then resumed on October 17, 2021 and now, the DC facility part is almost completed and servers and other devices are mounted in the DC racks. Server updating and OS deployment work is going on. Hopefully, we can go live from the DSE Tower DC,Nikunja very soon.

For second data backup (DR), the work is in progress at Monem Building District (MBD). The civil work has been completed and the other work like installation of Cooling system, UPS etc. are going on.Presently, DSE has its main data center (DC) at Motijheel and have a contingency setup at DSE Tower and small scale trading would be possible during disaster.

Our reply: The information that Tariq Amin stopped the project for two months is correct and the result of that DSE has saved Tk6 crore from data center project expenditure.

Even with that two-month delay, the delivery of the data centre was supposed to be the 3rd week of Feb 2022. The DSE still did not get the data centre completed.
 
Published report:  This was very dangerous for a company to depend on one vendor for all technology platforms because if the particular vendor suddenly ceases to exist, the DSE will have nowhere to go.

DSE response: The fact is not true. We are not depending on one vendor only. Currently DSE has been taking technology support from various local and foreign vendors. OWI is not only one supplier rather we have other suppliers who works with DSE on different technology solutions namely NDE, Ctrl-S, MyTech Partners Ltd., Smart Technologies Ltd, DataEdge Ltd, NNS Solution Ltd., Thakral Information System Ltd., Nasdaq, FlexTrade, Trapets AB, Microsoft, e-Generation, and many others. We usually keep 10% BG of the total PO value as security for any goods and services supply. The warranty, license and their maintenance is the responsibility of OEM where the Supplier will make liaison with OEM for DSE. We have information security policy and we can identify ill motivated vendor through our practice. Again, we have the option to change any vendor or partner of OEM as we procure warranty & service support from OEM.

So we don't find no risk if the Supplier cease to exist or collapse don't cooperate DSE. In this case, we can go to OEM and chose another enlisted Supplier/partner of OEM.

One should not compare or mixed up DC facility part (those are electrical equipment) and the cyber security solution with CISO and Chief Information /Technology Officer. In the DC facility procurement, we mainly procured electrical systems from OWI led consortium (OWI, NDE and Ctrl-S) like AVR, UPS, CRAC, electrical distribution panel, computer racks, civil works; and network cable system that includes fiber & copper raceway; pest control system, fire suppression system where cyber security solutions are mainly software technology. The warranty, license and their maintenance is the responsibility of OEM where the Supplier will make liaison with OEM for DSE and there is no correlation between electrical goods and cyber security solution in terms of maintenance and support. Therefore, there is no high risk predicament for DSE. As there is no risk, there is no need to rise any flag.

Our reply: Tarique Amin Bhuiyan who served the DSE as its managing director for over a year raised the concern over depending on one vendor. In his show cause notice to the CTO, he commented that "this was very dangerous for a company to depend on one vendor for all technology platforms because if the particular vendor suddenly ceases to exist, the DSE will have nowhere to go."