[RndTbl] TD network failure

Shewfelt, Doug DShewfel at city.winnipeg.mb.ca
Tue Oct 30 12:15:33 CST 2001


Gilber Detillieux wrote:
> According to author:
> > Does anyone have any specifics as to why the 
> > Toronto Dominion Bank had that major failure
> > this Saturday, October 27 ?  All I hear in the
> > media is that it was some kind of
> > 'hardware' failure. Nothing specific said.
> 
> I'm in the same boat.  I listened to various news reports on 
> Saturday night, but they just referred vaguely to a "computer
> glitch" or a "computer bug", without getting into any details.

There's a brief article on it in The Globe and Mail today
(http:.//www.globeandmail.com).  The full URL that I got when
I found the article is the elegant and easily remembered
http://rtnews.globetechnology.com/servlet/RTGAMArticleHTMLTemplate/C/2001103
0/wxtdbank?tf=tgam/realtime/fullstory_Tech.html&cf=globetechnology/tech-conf
ig-neutral&slug=wxtdbank&date=20011030&archive=RTGAM&site=Technology.

It doesn't go into a lot of detail. (I think that banks are
traditionally quiet about data processing problems.) Here are
a couple of paragraphs that give a little information:

    The crash was caused by the failure of a single
    "motherboard" in one of the bank's central computers
    at about 11 a.m. Saturday. This "gradually started
    to shut down the system" to "protect the integrity"
    of the data already there, Mr. Livingston said.

    "It was a purely random event," he said, adding that
    hardware failures are rare. "This has never happened
    before, and it will likely never happen again."

    Although the bank's credit card system "stayed on the
    air pretty much all day," and its debit card system
    also continued to function "intermittently," Mr.
    Livingston said, everything else went down. This
    included its telephone and Internet banking operations,
    in-branch computer systems and approximately 2,000
    automated banking machines. Customers also were unable
    to gain access to their money through other banks' ABMs.

    [...]

    As TD sought to identify and fix the problem, "a few
    million transactions" were rejected by the bank's systems,
    which, on a busy Saturday, process up to 500 transactions
    a second, he said.

> > I'd really like to know what happened to
> > crash their whole system. I thought the banks
> > had 'bullet-proof' networks what with all those
> > billions of dollars they have to spend.
> 
> [...] I would
> suspect it was not a network problem, per se, but rather a failure of
their
> account database, for it to affect all modes of account access like that.

Gilbert's theory sounds reasonable, especially since the credit
card system was working for most of the time. I assume that the
credit card system has an independent account database.

 - Doug



More information about the Roundtable mailing list