Pioneering Proactive Approaches to Power Protection

December 31, 2018 • Amanda McKeon

Schneider Electric is a global energy management and automation company headquartered in France, employing over 144,000 people around the world. With a history dating back to the 1830s, these days Schneider Electric enjoys success in industrial control systems, industrial safety systems, electric power distribution and grid automation, smart grid technology, and data center power and cooling.

Our guest today is Andrew Kling, senior director of cybersecurity and system architecture at Schneider Electric. He shares his professional journey, his experience pioneering many of the security measures we take for granted today, the shift to being proactive in his sector, and the importance of threat intelligence.

This podcast was produced in partnership with the CyberWire.

For those of you who’d prefer to read, here’s the transcript:

This is Recorded Future, inside threat intelligence for cybersecurity.

Dave Bittner:

Hello everyone, and welcome to episode 88 of the Recorded Future podcast. I’m Dave Bittner from the CyberWire.

Schneider Electric is a global energy management and automation company headquartered in France, employing over 144,000 people around the world. With a history dating back to the 1830s, these days Schneider Electric enjoys success in industrial control systems, industrial safety systems, electric power distribution and grid automation, smart grid technology, and data center power and cooling.

Our guest today is Andrew Kling, senior director of cybersecurity and system architecture at Schneider Electric. He shares his professional journey, his experience pioneering many of the security measures we take for granted these days, the shift to being proactive in his sector, and the importance of threat intelligence. Stay with us.

Andrew Kling:

Many years ago — gosh, I’ve really stopped counting, but it’s over 10 years — the topic of how do we authenticate users, how do we authorize the certain functions that they have access to, started to come up while I was working at Schneider, then Invensys. We started working through what the strategy would be. I had made some recommendations, though I wasn’t in charge of it at the time, I made some recommendations about adopting some industry standards and moving in that direction.

Well, as organizations go, they thought they could do a better job of this and they went and tried to invent a security layer of their own. And after about a year of that, they came back and said, “This isn’t working Andy, so we want to put you in charge,” and really, it kind of took off from there. We started addressing finding vulnerabilities, addressing them, building in layers of cyber features into our products, and I’ve just been picking up steam ever since.

Dave Bittner:

So, let’s explore that a little bit. Can you describe to us, what was the lay of the land there at Schneider? And what were some of the specific challenges you were up against?

Andrew Kling:

So at the time, we had both a Windows offer and the Linux offer. Actually, it was Solaris at the time, a Unix offer, Solaris. It was primarily focused on our DCS space, although we did have a safety offer, we did have skate offers. They really hadn’t come into my picture at the time. And really, they were developed as many legacy applications were, without an SDL, without a secure development lifecycle, without really acknowledging that cybersecurity was going to be something important.

I could clearly see that it was going to be something that was going to be of critical nature going forward in how we developed applications. You know, it was really about bringing awareness to the organization. So we were very lucky to have a few champions who were at that senior level — not quite executive level, but at the senior level. They brought an awareness to the organization that there at least has to be some attention that’s paid here. So I think we were very lucky to have those early champions because they allowed us to get a program started. Those early, early days, we really got the seeds planted and we really started to make some good progress. And it may sound very strange to say, good progress. “What was good progress, Andy?” We got rid of a fixed password. That was something very significant at the time.

Everybody thought it was fine to have fixed credentials to log into the application, everything running under admin. These are very early wins that we got and really, we’ve gotten a lot of mileage out of getting just those very straightforward things accomplished.

Dave Bittner:

Can you give us some insight? What was it like getting buy-in for these sorts of things? What was the diplomacy like on your end?

Andrew Kling:

Buy-in. Okay. So, in an R&D organization, often one of the misunderstandings about how R&D works in many large companies is that R&D actually controls their budgets. R&D doesn’t control the budgets. R&D actually gets direction from other parts of the organization, typically an internal marketing organization attached to sales and external marketing, and they set the product direction. They set the strategy and R&D executes on that strategy.

So, really, the early buy-in, while yes, there is buy-in at the R&D level, the big wins are when we got some of our marketing organization to understand the need to start to put some of these cyber features to bring cybersecurity to our product offer. And then what you could see literally were light bulbs going off all around the organization.

Even today, we all experience people who acknowledge cybersecurity as something important and execute it as part of their jobs, but then there are people who get it, where the light bulb has gone off and they understand that cybersecurity is more than just about building security, preventing people from having access to your product. It’s about bringing a different type, a different way of thinking about quality, the quality of the product you’re building, the robustness of the product that you’re building. And those early wins were those people where these light bulbs were going off.

Dave Bittner:

And so, is it a matter of shifting from, I suppose, a culture of being reactive to being proactive and building things in from the get go?

Andrew Kling:

Of course. We have standards that help us along, understanding that you have to be proactive in order to bid now, and that’s important. There’s still a large amount of reactivity in the market. Whether you want to lead or follow is a strategic decision, and sometimes, committing too early is as important as committing too late, and so there’s a balance in what we look at when we talk about cybersecurity.

It’s not a balance of, is this enough cybersecurity? Because honestly, it’s kind of binary. Either it’s enough or it’s not. And if it’s not, you’re breached. It’s really an amount of how much effort, how much do we continue to focus on this as we go forward?

And it means that we’re developing a program that isn’t just about bringing features to the product, it means that we’re developing a cybersecurity program that reaches all the way back to where the product was conceived and all the way forward to how that product is supported in the field, how we are managing that product as it goes through its lifecycle, how we’re managing that service, those services that we offer around it, whether it’s training or installation or cybersecurity and services. It’s really about the maturity of the program that we bring as we’re looking forward.

I guess your question was, is it proactive or reactive? There’s a balance there, is really what I’m trying to say.

Dave Bittner:

And I suppose it’s also something that at this point in the game, security can’t be grafted on after the fact. It has to be part of the process from the get go.

Andrew Kling:

Correct. That even when you try to graft it on, or as some of our very large oil and gas customers have pointed out, they call it “bolted-on security,” it’s very obvious when you’re bolting on security to a legacy product and the product itself hasn’t had security essentially built into it from the very beginning. One example to look at is, it’s part of it but really a little bit adjacent, and that’s GDPR privacy.

A lot of the products, a lot of the systems that exist today don’t have information models built into them, don’t have information security models built into them. And so, GDPR was something that was bigger than it probably needed to be, if we had these information security models built into our product offers from the very beginning.

And when I say “our,” I mean across the industry. There were a lot of people scrambling around GDPR. And really, it’s coming back to, how do we control the access to the information? A very cyber-centric principal.

Dave Bittner:

Now, I suppose over the course of your career, you’ve really been witness to this process of these industrial control systems. That side of the industry, I guess for lack of a better word, being hosed up to the internet because it wasn’t always that way on that side of things.

Andrew Kling:

That’s a direction that was very predictable. At some point, people started labeling a lot of this connectivity as IoT or computing at the edge. And we all know that in the industrial control space, a lot of this had already existed. No, we weren’t possibly using IP-based protocols, we were using more ICS proprietary protocols, but the concept of having a compute capacity at the edge with communications that are inherent in it has always been around in the ICS industry.

Having that evolve, that compute capacity now to include the ability to communicate another part of that IoT definition, to be able to communicate out to the internet, possibly even bypassing the control system altogether, is something rather obvious, that this is where things are, where things are going, and will continue to evolve in this direction.

As the concept of computability at the edge gets cheaper and cheaper, people are going to want to do more and more with it. Those people aren’t necessarily always defined by operations. They’re going to be defined by the business side as well and they’re going to want more direct access to that information that’s going on in their plants. This will allow them to pick up the speed of their business. So, it’s very natural, this progression. You can continue to play it out forward and really sort of anticipate where it’s going to be going.

Dave Bittner:

And where do you suppose it’s going?

Andrew Kling:

Where I suppose it’s going is, you’re going to be looking at layers of compute resource. You’re going to start to think about running your applications where it makes the most sense, not necessarily where it’s driven just by process efficiency, but perhaps driven by other factors like cost. We’ll just pick cost or security. You may execute your application in a more secure location or more cost-effective location because you have more direct access to those inputs coming from the process itself.

Now, this isn’t to say that process control goes away — of course not. Process control is very fixed in our world. But the fact that, say, a business may want to build a thermometric model of a reaction, and they may want to do this using thermal imaging cameras instead of a series of sensors, that may be something that the business may want to do and they may want access directly to this information. So they may be reaching directly down to the process to get access to this information, bypass the process control system. Now, they may then make a business decision about how to affect, how to tune that process. They may want to say the cost of energy is going down or these raw material costs are shifting in price — let’s adjust the plant based on a business decision, not just the traditional efficiency reason.

Dave Bittner:

That’s an interesting point because I think you can’t discount the fact that these are businesses, and so there are competitive forces at play as well.

Andrew Kling:

And this brings in a whole different line of security challenges, then. Who has access to this information? How fast do you need it? One of the challenges that we have right now is, how fast do you need access to these inputs, yet you still have to secure them. We all know that there’s an awareness that’s being driven across the ICS industry about layer zero, layer one, basically inputs and outputs, and the cybersecurity that exists at those levels.

And what can we do about them? If you just start to apply traditional cybersecurity thinking, you start to get in trouble really quick because you can’t drop hardware-based encryption on an input sensor that requires sub-millisecond response times. You simply can’t drive it that fast, but you do need to bring cybersecurity to it.

These are the kinds of challenges that IoT, that this connecting of information to other parts of the business, say it’s the internet or not, doesn’t matter, but other parts of the business. And you’re going to be really challenged with having to come up with some unique ways to bring cybersecurity to this new world.

Dave Bittner:

So, I want to switch gears a little bit and talk about threat intelligence. Let’s just start off at a high level. What part do you think threat intelligence plays in the kind of work that you do?

Andrew Kling:

It’s interesting. Each year, when I work with my team, I try to pick a theme that I’m going to be focusing on for the year. And one of the things that I picked this year, one of the top things I picked was to stop focusing on just the SDL to help us find vulnerabilities, to help us understand what it is that we’re supposed to be fixing in our products.

Yes, the SDL is a very, very powerful tool in helping drive consistency in our product development across our organizations, but what I wanted to think about was, think like the attackers. They’re not saying, “What’s the most severe vulnerability that they have? I’m going to attack there.” They’re saying, “What tools do I have at my disposal? What techniques do I use? And I’m going to come in wherever that vulnerability is at whatever level.”

So an SDL drives you to think about the most severe vulnerabilities first, addressing those. And there’s a certain logic to that of course, but you have to think about threat intelligence as part of the story. You have to think about where your adversaries are, where the advanced persistent threats are, where the emerging viruses are, and what they’re taking advantage of. And you need to incorporate that into your prioritization scheme. Prioritization scheme being, that scheme that drives your backlog, how you sort your backlog so you’re working on the most important — note now I’m not saying the most severe, I’m saying the most important — vulnerabilities first, so that you’re addressing the highest-risk items first.

Dave Bittner:

It’s a really interesting insight. I’m curious, what part do you believe that automation plays in this? I’m thinking of things like artificial intelligence, being able to filter that fire hose of information before it gets to your analysts.

Andrew Kling:

Traditional web searches, searching for keywords, there’s a place for that. It’s part of the, if you want to call it, a stack of technologies that you could use for threat intelligence. But at some point, when you’re a person in my position, I have multiple product lines that I’m responsible for. There are many, many, many technologies that are used across the 600 or 800 engineers that we have in our organization, this part of the Schneider Electric organization. How do I keep track of all that? How do we, even a team of security engineers, think about that? And it’s too much. So you have to start thinking about, can you use an AI to parse the language? Can you use an AI to parse the language and start to pull out metadata and start to pull out themes in the discussion to help narrow your awareness, your search, toward threat intelligence that makes the most sense for us?

Now I’m not condoning that we start trolling the dark web and looking in all those dangerous places around there. There are organizations that do that. I do think that we should be partnered with those kinds of organizations. I talk about with my own organization, we talk about yes, the public-facing threat intelligence that is sort of the first layer, and those AI tools are going to provide a great resource to help narrow that fire hose of data down into some information that we can process.

But I think that we also have to be looking to organizations that bring us a behind-the-scenes type of threat intelligence, whether this is researchers that we’re connecting with and listening to and what they’re investigating. And in that case, researchers, very interestingly, not just researchers looking for vulnerabilities in your product and looking for different ways to attack products and systems that we produce, but researchers who are also looking the other direction about how to detect attacks and progress, how to detect vulnerabilities. Not just finding the vulnerabilities, but what are the techniques that they’re using? What does a mathematical model look like, of a stable DCS system running? And if you see some sort of change in that mathematical model, can you identify a cyberattack and process? These are the kind of researchers that we also try to engage with so that we can sort of see what’s coming from their direction and knowing that the threat actors are also thinking in these terms. How do they come into a system more stealthily? So it’s a back and forth in this threat intelligence.

And then finally, when I think about threat intelligence, I think any large organization like a Schneider Electric should have government contacts. Not just the US government, but governments around the world, and we should be working with them, sometimes at a classified level, not always at a classified level. And this is something that we take very seriously as well.

All of the threat intelligence allows us to build a picture to understand what the adversaries are, who they are, the tools and techniques that they’re using, and what we need to do to address them.

Dave Bittner:

I’m curious, from a personal point of view, that must be, to a certain degree, sobering that the target that you all have on your back as an industry is at the nation-state level. And we’re talking about critical infrastructure and national security.

Andrew Kling:

When I have informal talks with friends or colleagues, I do acknowledge that the headlines really are full of more financially-driven personal cyberattacks. The WannaCrys don’t distinguish anybody, but the WannaCrys of the world are out there trying to go through and get some sort of ransom from somebody.

But when I lead that conversation forward, then, I say the target that’s painted on our back is so much bigger because it’s more than just money. In this case, sometimes it’s human safety that’s in jeopardy, and we have to take this very seriously. And I can say is that we do take it very seriously. It didn’t take the recognition of it becoming human safety that’s involved to get us there, but it does underscore that importance of what we’re doing. You can’t take your eye off of the ball here. You have to constantly be vigilant, and threat intelligence is an important part of paying attention. Something can crop up and it can go from zero to 100 in the blink of an eye. Look at the WannaCry situation, or it started even earlier when EternalBlue from the tool dump hit the street. It went from that to WannaCry so fast. And we knew what the vulnerability was that EternalBlue was taking advantage of and we were working to react to it, yet we still were not faster than WannaCry getting to the street.

Dave Bittner:

Is there anything that you wish that regular folks, literally in their day-to-day lives, who see these stories in the press about the possibilities of electric grids going down and so forth … I guess what I’m getting at is, what’s an appropriate level of concern for people to dial in?

We’ve got folks like you who are out there working and fighting the good fight to make sure that we are safe and that these systems are reliable, and it seems to me like they certainly are, certainly here in the United States, but what’s the appropriate level of concern for folks like us to dial in?

Andrew Kling:

That’s a good question. The one thing that I would ask the average person to understand, just to think about even just for a few minutes, is what the critical infrastructure of their life is. We all like healthcare. We all enjoy clean water. We all understand that food is important to us and that delivery of the food through our supply chain is important. People should just think about that and think about, if an adversary was sufficiently motivated, would they attack our critical infrastructure? What are we doing to protect that critical infrastructure?

And if the average person just thinks about it, they realize this is bigger than buying a piece of antivirus software. It’s bigger than safe email practices. This is something that’s very important. And they need to be asking their leaders, whether that’s leaders in the government, local or federal, whether it is leaders in those companies like the Schneiders of the world that are building products that run, that power the critical infrastructure, they need to be asking their leaders, what are they doing about protecting their critical infrastructure?

Dave Bittner:

Thanks to Andrew Kling from Schneider Electric for joining us.

Don’t forget to sign up for the Recorded Future Cyber Daily email, where every day you’ll receive the top results for trending technical indicators that are crossing the web, cyber news, targeted industries, threat actors, exploited vulnerabilities, malware, suspicious IP addresses, and much more. You can find that at recordedfuture.com/intel.

We hope you’ve enjoyed the show and that you’ll subscribe and help spread the word among your colleagues and online. The Recorded Future podcast team includes Coordinating Producer Amanda McKeon, Executive Producer Greg Barrett. The show is produced by Pratt Street Media, with Editor John Petrik, Executive Producer Peter Kilpe, and I’m Dave Bittner.

Thanks for listening.