New Orleans has declared a state of emergency as it battles a city wide ransomware attack. The attack was first detected at 5am on Friday morning. By 11am, the city took all its computers and servers offline. While the attack is believed to have started through compromised employee credentials, the city has stressed that no other user credentials or data have been stolen.
Key to preventing the spread of the attack and the loss of data is the speed with which the city reacted to the attack. Kim LaGrue, Chief Information Officer for the City of New Orleans told press that: “The attack was detected during routine checks of activity that starts early in the morning.”
The first notifications of suspicious were noticed at 5am. LaGrue’s team began to investigate and confirmed the attack by 11am. At that point all affected systems were ordered to be shut down across the city.
LaGrue also confirmed that the initial access into the city systems was through compromised user credentials. What is not known is who that user was, what level of access they had or how they were compromised.
New Orleans response was fast and public
The reaction was not confined to internal systems. The communications team were quick to push details across their various social media feeds. For example, on the NOLA Ready page the city posted the following statement:
“At approximately 11am today, the City of New Orleans detected suspicious activity on its networks that indicated a potential cyberattack. Out of an abundance of caution, all employees were immediately alerted to power down computers, unplug devices & disconnect from WiFi. All servers have been powered down as well. http://Nola.gov websites will be down.
“Emergency communications are not affected. Orleans Parish Communication District-OPCD 9-1-1 & 3-1-1 services are up and running. New Orleans Police Department, New Orleans Emergency Medical Services & New Orleans Fire Department are fully able to respond to emergencies as normal.
“The City of New Orleans has activated its Emergency Operations Center & is working with cybersecurity resources from the Lousiana State Police, FBI New Orleans, Louisiana National Guard & the Secret Service.”
The city also posted recordings of the press conferences given by Mayor LaToya Cantrell to ensure that all information was public.
New Orleans capable of working without Internet
Ransomware attacks against municipal centres across the USA have become commonplace. The impact for some has caused chaos and extended over several weeks. Some have even resorted to paying to get their data back. Part of the problem is the lack of preparedness for this type of incident.
Collin Arnold, Director of Homeland Security and Emergency Preparedness, City of New Orleans said: “The positive for a city that has been touched by natural disasters and essentially brought down to zero in the past, is that our plans and our activities from the public safety perspective reflect the fact that we can operate without Internet, without the City network. It makes things obviously more difficult but from a public safety standpoint we will go to marker boards, we will go back to paper, we are doing that right now upstairs. We can backfill that when things are normal again.”
That message was echoed time and again by police, fire and EMS. While their systems were still working there were issues with access to street cameras. Interestingly, Arnold pointed out that those cameras would continue to work and record locally. If there was a need for video footage then the data would be retrieved by sending someone to the camera.
Cantrell pointed out that city workers were using their own Internet access and devices to continue working. However, she made it clear that ALL emails and communications would be reintegrated to the city’s own systems post recovery. This will deflect any post incident concerns over the potential for data theft.
Effective Incident Planning and recovery in progress
In Cantrell’s press conference on Saturday she told journalists that New Orleans was in recovery mode. She stressed that there was no evidence of personal data being lost but did confirm the scale of the attacks. She said that 4,000 computers need to be scrubbed along with 400 servers. Over 7,000TB of data is affected and the attack has compromised 20 separate systems.
One challenge for LaGrue’s team will be the scrubbing of the computers. How deep do they scrub the drives? How long will that take? Will they rip and replace the hard drives to speed the process up? Will they rip and replace entire systems, especially those used by critical parts of the city government?
LaGrue also clarified claims over data loss. She said that there was no evidence of data being stolen and loss meant data that had been encrypted by the ransomware. That data is not necessarily lost permanently. The IT department uses a layered backup approach. That means some systems are backed up daily while others are backed up more frequently. Some systems run in parallel using real-time backup and failover. It will be interesting to see if the latter systems survived the attack.
One thing that is known is that the finance systems were protected because they were stored in cloud servers. It will be interesting to see if New Orleans will now look to use cloud computing for many of the affected systems as they are restored.
The next stage for LaGrue will be the full forensic investigation. That has already begun with external teams from the FBI joining LaGrue’s team to handle this.
Enterprise Times: What does this mean
There is much to be admired here from a response perspective. Cantrell and her various teams took control of this story and have driven the information agenda. This is evident from the openness of the press conferences and the way the various teams have spoken publicly. Keeping control of the story is an essential part of incident response that is often overlooked. When one journalist attempted to add to the story Cantrell responded saying: “Don’t create a situation because you’re adding things that no-one up here is saying.”
The speed of response is also a positive here. LaGrue and her team didn’t wait to see how the attack unfolded. Once confirmed, they shut everything down. They then proceeded to investigate before any data could be lost and implement their recovery plan. Simultaneously, the city activated its own emergency plans.
This attack is a problem for New Orleans but also a significant opportunity. It’s response and behaviour is far better than most private companies and way ahead of other US cities, towns and even states. Once the dust has settled, there is much for others to learn about how to be prepared for any disaster, cyber, natural or man-made.