Why Does the U.S. Lag Behind China in Vulnerability Reporting?

October 30, 2017 • Amanda McKeon

The U.S. National Vulnerability Database, or NVD, is, in part, a collection of security-related reports. Software vulnerabilities are assigned CVE numbers, which stands for common vulnerabilities and exposures, which help track the issues and provide a common reference for referring to a specific flaw. China has a database of their own, the Chinese National Vulnerability Database, or CNNVD.

Our guest today is Dr. Bill Ladd, chief data scientist at Recorded Future. His team noticed that publicly known vulnerabilities were showing up more quickly in China’s database than in the U.S., quite often taking days instead of weeks. This not only has the potential to put U.S. defenders at a disadvantage, it could also give black hats the upper hand.

In this episode we’ll learn why the NVD lags behind the CNNVD, why it matters, and what could be done to correct it.

This podcast was produced in partnership with the CyberWire and Pratt Street Media, LLC.

For those of you who’d prefer to read, here’s the transcript:

This is Recorded Future, inside threat intelligence for cybersecurity.

Dave Bittner:

Hello everyone, I’m Dave Bittner from the CyberWire. Thanks for joining us for episode 29 of the Recorded Future podcast.

The U.S. National Vulnerability Database, or NVD, is, in part, a collection of security-related reports. Software vulnerabilities are assigned CVE numbers, which stands for common vulnerabilities and exposures, which help track the issues and provide a common reference for referring to a specific flaw. China has a database of their own, the Chinese National Vulnerability Database, or CNNVD.

Our guest today is Bill Ladd, chief data scientist at Recorded Future. His team noticed that publicly known vulnerabilities were showing up more quickly in China’s database than in the U.S., quite often taking days instead of weeks. This not only had the potential to put U.S. defenders at a disadvantage, it could also give black hats the upper hand. Stay with us.

Bill Ladd:

NVD is the National Vulnerability Database, a U.S. effort that tracks vulnerability information, software, severity, etc. Many companies rely on the CVSS scores. For example, they provide an understanding of the severity of vulnerabilities, and they have feeds that go into various commercial products that take advantage of the information they have to help companies protect themselves. We noticed, probably several months back, as we were looking at our data, that there seemed to be a pretty big delay sometimes, between when information about vulnerability was first announced, and when it was included in the NVD, the U.S. National Vulnerability Database. We did a piece of original research. We looked at just what does that system look like.

You know, what we discovered is that there were literally thousands of sources on the web that would have some amount of information about a CVE, explicitly tagged with a CVE identifier, before they were available in the National Vulnerability Database. So, we definitely saw pretty significant lags at a time. You know, typically at least a week, and often weeks or months before information was available in NVD, after it was publicly known on the web and publicly disclosed. We took a closer look at that and discovered that different vendors had different performance in terms of how quickly their vulnerabilities appeared in NVD. And we believe that’s, you know, different vendors basically take the submission process with different levels of seriousness.

Adobe, for example, when they release a vulnerability on their website and disclosed it, it’s typically in NVD very quickly, which means they’re submitting it to NVD very quickly, and other vendors are slower. So, that’s essentially the scenario that we started with, which was, there’s a lot of information out there on the web about vulnerabilities before it’s in our comprehensive national database of vulnerabilities, NVD.

Dave Bittner:

Is there any particular reason why some companies would be slower than others? Is there a practical reason, or is it simply a matter of them setting priorities?

Bill Ladd:

I think it’s really about priorities. It doesn’t seem like there’s any particular rhyme or reason to which companies behave in which way. I think it’s really … again, what is it that they’re putting their efforts into?

Dave Bittner:

Now, if I’m a defender, does that mean that I will likely have learned about a vulnerability elsewhere before seeing it on the NVD?

Bill Ladd:

Absolutely. Certainly, if you’re looking. I think that’s one of the issues, is that different organizations have different levels of resources that they can apply to this problem. We certainly see some researchers who track a collection of these broader sites, or have four or five Twitter handles that they thought probably were people that kind of announced big vulnerabilities. And we have others — other organizations that are more passively waiting for their vulnerability management tools to incorporate the information. So, the information’s clearly out there. There are certainly defenders that make use of it, and then there’s certainly defenders that don’t know about it. We think that’s actually kind of the key part of the problem.

Dave Bittner:

Just sort of as background, I mean, what is the NVD intended to be? What is its intended purpose?

Bill Ladd:

Before there was NVD, there was no common nomenclature or description of vulnerabilities, so they started as a way to provide a consistent nomenclature — these are the CVE numbers, you know, that I’ve been mentioning.

And when everybody has a common name for a vulnerability, then you know it’s very clear. You know, for example, if you’re patching your software. But what vulnerabilities are being fixed and what vulnerabilities are outstanding? The identifiers are key, but it’s also meant to be the comprehensive resource for vulnerability information. You know, managed and maintained by the U.S. government. Their website is actually pretty clear that that’s what they’re attempting to be, certainly, on serious vulnerabilities that have been well known for a while. There’s a lot of information on NVD about the vulnerabilities, you know, links to patches, links to references, and so on.

Dave Bittner:

You have new research that you’ve done, sort of comparing and contrasting how China is reporting vulnerabilities versus the U.S.

Bill Ladd:

What we did is, we saw how NVD was performing, so we thought we would benchmark it against another similarly operating effort, which was the Chinese National Vulnerability Database, or CNNVD. And really, we didn’t have any idea, when we went into it, what it was going to be in terms of how much overlap there was going to be, or what the timeliness was going to be. But the Chinese National Vulnerability Database also very conveniently uses CVE numbers, as well as their own internal IDs. So, it was relatively straightforward to look at their performance on the set of CVEs versus the U.S. It’s also pretty clear that they cover, primarily, the same set of vulnerabilities. The overlap between the two systems is quite high. 90 to 95 percent of vulnerabilities were covered in both systems, and what we discovered is that, by and large, things were reported much more quickly in the Chinese National Vulnerability Database than they were in the U.S. You know, the Chinese database doesn’t have a systemic advantage. They don’t issue the CVEs. They’re not the source of record for the identifiers. They’re not the source of record for where people make submissions. So, there are certainly cases where NVD got it first because the vendor was aggressive in submitting it.

When NVD got it first, CNNVD was likely just a day or two behind. But there were many thousands of vulnerabilities that, again, the Chinese would be a day or two behind the initial public disclosure. We would have that weeks and months delay that we took that we saw with NVD and their research.

Dave Bittner:

Now, just for clarity, would a CVE have the same number in the Chinese Database as in the U.S. Database?

Bill Ladd:

Exactly. The Chinese are basically taking advantage of our CVE identifiers. We find people in the dark web use the same CVE identifiers. They’re helpful in describing what software you’re talking about, what vulnerability, so they absolutely use the same CVEs. They’re not involved in the granting of the CVEs like the U.S. system does, but they’re able to gather that information across the web fairly aggressively.

Dave Bittner:

So, what do you suppose causes this difference between how quickly things are reported there and here in the U.S.?

Bill Ladd:

So, in the U.S., the way that it works is that vendors, when they discover a vulnerability request, issue themselves a CVE number, which is then reserved for that vulnerability. Then they do internal research on it. They might create patches. At some point, they are ready to disclose it publicly, or perhaps the research that they’re working with publishes their report using that CVE number that’s been reserved for a vulnerability. Some point, then, what they do is they submit that CVE information package to the MITRE CVE dictionary. MITRE is a contract firm that manages, essentially, issuing of these identifiers. When a identifier is registered into the CVE dictionary, it automatically gets updated into the National Vulnerability Database. Then, shortly thereafter, the folks at NVD provide a severity score so that organizations can see how risky it is. So, you’ve got a very defined process. Vendor requests a number, vendor discloses the vulnerability for that ID, they submit that to the CVE dictionary, and it’s automatically included in NVD.

What that breaks down is that the vendors are slow. They don’t necessarily push that information to the CVE dictionary, and so you’ve got this process where you’ve got these two different players, MITRE and NVD, each following a process to get this information moving through. The Chinese, on the other hand, are doing, presumably, something more comparable to what we’re doing, which is scanning the internet for mentions of vulnerabilities regardless of where that first mention is, if it’s on NVD, or if it’s on a Microsoft site, or if it’s on a security blogger’s research article. If you gather that information when it’s available then you can add it to your database. So, they’ve clearly created an operation that allows them to gather that information proactively from around the web, as opposed to waiting for the submissions to come in for vendors who may or may not be following a timely process.

Dave Bittner:

And is there any downside in the way they do it? Do they have a lower degree of accuracy for example than we would have?

Bill Ladd:

I think that’s fair. I think there’s certainty if you’re moving faster, you are going to have a little bit less accuracy. But I think that’s okay. I think it’s really a question of basically being clear about where you are in the process. If you’re taking the time to get everything perfectly right, by the time you’re perfectly right, some of the most critical stories will have passed you by. And so, there’s absolutely an accuracy price that you pay for that. But I think the timing, the timeliness, is incredibly important, particularly for the most severe vulnerabilities. So, ideally, what you would do is you would tag the first information as preliminary until you have time to go through a review process. But at least it’s available there for researchers who are exposed to the vulnerabilities. I think that’s a much better approach than simply having no information about the vulnerability while you were waiting to go through that process.

Dave Bittner:

Now, if I’m a researcher, could I simply access the Chinese database? Is there anything keeping me from doing that?

Bill Ladd:

No, you could. It’s a little tricky. You’d want to do some translation. The sites a little flaky.

Dave Bittner:

Oh. It’s in Chinese.

Bill Ladd:

It is in Chinese.

Dave Bittner:

Right.

Bill Ladd:

There’s a little bit of English text in there, and we also see it’s not super stable. It kind of goes up and down, and I don’t know why that is. You certainly could use it, and it’s certainly got more timely information than the U.S. one does. What it’s not, is integrated into the downstream processes, like the vulnerability tools, like the U.S. one is. So, in terms of looking at it as a place to find information, absolutely. In terms of automation, which is where you really want to go, it’s probably less suitable for helping you with an automated framework. For me, the mission would be, if you’re gonna be the vulnerability database, your mission is to be as comprehensive as possible. Understanding that when you first get information, it’s still preliminary, but that you want to have everything that’s available so that you can provide that resource. And I think the Chinese have focused on that mission. And to that end, they clearly have an effort, which goes out and gathers information from across the web.

I think, in the U.S., we’ve ended up with these two different organizations, MITRE and NVD, that each owns a piece of the puzzle and neither perhaps feels complete ownership of the mission, which is to be as comprehensive as possible. They’ve clearly focused on accuracy and no one appears to have prioritized the, “Well, it’s out on the web, should we make sure it’s in our system?” As opposed to waiting for the vendors to submit that information.

Dave Bittner:

One of the things you pointed out in your research is, this has the practical effect that black hat folks could look at the Chinese database and perhaps have a head start on their U.S. counterparts.

Bill Ladd:

I think, absolutely. One of the things we noticed in the original research is that … You know, the day that a major vulnerability is announced, work starts on it on the dark web. Proof-of-concept codes starts to be generated. That happens immediately. And why does it happen immediately? It happens immediately because the black hat community is more actively searching. Again, they’re doing that proactive searching that CNNVD is doing. And if you’ve got someone like CNNVD that’s doing that proactive searching for you, it’s a much easier place to start than trying to monitor the 200 sites that might break CVE knowledge, initially.

Vulnerability publication has always been a little bit of a mixed bag. Do you publish the vulnerability and allow exploits to be written, or do you hide the vulnerability and people don’t patch it, but maybe it’s a zero-day at that point. The point to publicizing the vulnerabilities is to encourage and expedite the company’s ability to patch themselves and mitigate the exposure they have from the vulnerabilities. In any way that you’re doing that, you’re giving an unfair advantage to the black hats. It’s not helping you the way that you would like the system to work. So, if the black hats are more aggressively monitoring, they get access to information sooner. Whether they’re doing the proactive searching themselves or they’re taking advantage of the proactive searching that the Chinese National Database is doing, either way, they’re getting a headstart on the people that are relying on the U.S. NVD.

Dave Bittner:

So, you have a couple of case studies in the report. Can you take us through those?

Bill Ladd:

Sure. And before I do that, let me just point out that neither of these case studies did I look at the difference before I decided to — the difference between NVD and CNNVD — before I decided to cover them. The first case study is we looked at Dirty Cow. We’ve looked at that in the original research several months ago, just as an example where NVD was extremely late. Dirty Cow was a vulnerability that had been in Linux code for a decade. It was a pretty significant vulnerability that could be highly used in exploit kits, and in fact, probably was used in exploit kits as zero-days for years before it was disclosed, because the day that they announced it, the reason that it was discovered was because they had seen it being exploited in the wild. The initial announcement was translated into Russian and dropped into a Russian criminal forum immediately. We saw proof-of-concept code and paste bin within days. It literally took NVD three weeks to get this vulnerability covered in the system.

So, if you’re relying on a centralized system, you know, it was three weeks after this very serious vulnerability was disclosed, even a vulnerability that was actively being exploited in the wild as a zero-day before the disclosure. They don’t come any more serious than this, and it took three weeks. So, we published that example months ago, and I said we did this in NVD. Let’s see how CNNVD did in this. And it was like, oh, they had it within two days, which is pretty typical for them. A day or two within the initial public disclosure, a whole 20 days before NVD. If you’re looking at those sites, they’re covering it.

Another example we looked at was the vulnerability that was used in the Equifax breach. And again, the question was, “Let’s check that one because everybody’s going to be thinking about it when they read about vulnerabilities. It’s the vulnerability in the press right now.” NVD was pretty quick on that one. It took a mere four days to go from disclosure to inclusion at NVD. But when we took a closer look at it, not only did CNNVD have it the same day that it was announced, we found proof-of-concept code on a Chinese hacker site the day it was announced. So, it’s clear that there’s so much going on in terms of taking advantage of these vulnerabilities — these critical vulnerabilities — when they’re first announced that proof of concepts are being built and deployed. Defenders need to actively be protecting themselves, and if you rely on our National Vulnerability Database, there’s nothing in there.

Dave Bittner:

Do you have any sense for what kind of resources the Chinese are dedicating to this project? I mean, is it something that as a nation state that they take seriously and fund, or could it just be a handful of highly motivated people?

Bill Ladd:

It could be both. Right.

Dave Bittner:

Sure.

Bill Ladd:

I mean, it feels like what they’ve done is they’ve taken a manpower solution, and they figured, whether it’s their individual scanning these websites or if they’ve written code to go against them. I think it’s clear there’s a concerted effort to gather this information. I don’t know, but I would believe it’s, like I said, I believe that it’s definitely managed at the nation-state level, that they’ve decided that this is information that they want to gather.

Dave Bittner:

So, do you think it’s a matter of people reporting, following up on their CVEs, and reporting them to NVD more quickly? I don’t want to go as far as to say making that compulsory, but perhaps some sort of a PR push of saying, “Hey folks, it’s important that you stay on top of this.”

Bill Ladd:

I mean, I think that would definitely be a good thing. But I think at the end of the day that’s never going to be good enough. There’s always going to be information that gets out before the process. Whatever the process is, it has time to catch up. I really think that, at some level, you need a proactive approach where you’re actually searching for that type of content and actively bringing it into your system, as opposed to waiting for publication. Publication is great, but companies should do it better and there should be a better program there. But at the end of the day, it’s never going to be perfect. So, rather than assume that you can fix that process, I think it’s really important to figure out how you’re going to gather this information from across the web, as soon as it’s available.

Dave Bittner:

Our thanks to Bill Ladd for once again joining us.

You can read his complete report “The Dragon Is Winning: U.S. Lags Behind Chinese Vulnerability Reporting” online at recordedfuture.com/blog.

Don’t forget to sign up for the Recorded Future Cyber Daily email, where everyday you’ll receive the top results for trending technical indicators that are crossing the web, cyber news, targeted industries, threat actors, exploited vulnerabilities, malware, suspicious IP addresses, and much more. You can find that at recordedfuture.com/intel.

We’ve hope you’ve enjoyed the show and that you’ll subscribe and help spread the word among your colleagues and online. The Recorded Future podcast team includes Coordinating Producer Amanda McKeon, Executive Producer Greg Barrette. The show is produced by Pratt Street Media, with Editor John Petrik, Executive Producer Peter Kilpe, and I’m Dave Bittner.

Thanks for listening.