An Internet Born In a Threat-Free Environment

Posted: 22nd March 2021
An Internet Born In a Threat-Free Environment

Our guest this week is a true internet pioneer. Dr. Paul Vixie describes himself as a “long-time defender of the internet.” He’s an author or co-author of several RFC documents and open source software systems including BIND and Cron, a serial entrepreneur, now CEO and co-founder of his fifth startup company, Farsight Security, and an inductee into the Internet Hall of Fame. 

He joins us with insights on how we are suffering the ramifications of early internet design choices, what that means for global networking going forward, and why he believes it’s best to not rely on outsourcing your DNS.

This podcast was produced in partnership with the CyberWire.

For those of you who'd prefer to read, here's the transcript:

This is Recorded Future, inside threat intelligence for cybersecurity.

Dave Bittner Hello everyone and welcome to episode 201, of the Recorded Future Podcast. I'm Dave Bittner from the CyberWire.

Our guest this week is a true internet pioneer. Dr. Paul Vixie describes himself as a “long-time defender of the internet.” He’s an author or co-author of several RFC documents and open source software systems including BIND and Cron, a serial entrepreneur, now CEO and co-founder of his fifth startup company, Farsight Security, and an inductee into the Internet Hall of Fame. He joins us with insights on how we are suffering the ramifications of early internet design choices, what that means for global networking going forward, and why he believes it’s best to not rely on outsourcing your DNS. Stay with us.

Paul Vixie Mid '90s, I started the first anti-spam company and unwittingly invented the distributed reputation business. So if you've ever heard of something called the RBL, the real-time blackhole list, that was our work, which we did not patent, so now everyone uses it. And pretty much no email you will receive from now until the end of your life, will not have been protected by RBL in some form, so as relevance junky, that was pretty cool.

We later sold that company to Trend Micro because it turned out, that if you stop people from spamming, which was not really illegal, and you do it in partnership with some number of other companies, then it looks like conspiracy, in restraint of trade. So there were a lot of lawsuits and eventually, in order to pay for the lawsuits, I had to sell the company itself. So my partner and I in that venture, decided, "Let's see who wants it, for how much money?" We sold it, we paid off the lawyers and he became their CTO. So, that was actually not a bad outcome, but it was damn foolishness, which I regret.

Before that, I had started a company called Internet Systems Consortium, which was a non-profit, dedicated to DNS software. And so there was a time, when our software, which we originally inherited, part of the Berkeley Unix project, had an almost 100% market share for providing DNS services and consuming DNS services. Eventually, it diversified and so now, there's quite a few companies and some nonprofits, and it's a lot of different DNS stuff. But at the time, when we were trying to figure out how to commercialize and privatized the internet, the DNS turned out to be vital and we started that company.

My co-founder in that venture was a fella named Rick Adams, who had been the original founder of UUNet, So definitely one of my mentors. That company still exists, but is a nonprofit. So even though I did end up working there full-time for around 10 years, I always intended and now have, returned to private industry. We were an internet connectivity provider to AltaVista and other companies like AltaVista, who had a need for pretty high end, pretty complex services. And the cloud didn't exist and so we had a tiny little ISP with 12 people, and we made sure that some of the larger, early sites on the internet were reachable 100% of the time, and so that was a lot of fun.

So I was the co-founder and CTO of a startup called VieU, in that same late '90s era, when we were trying to commercialize and privatized everything. And it looked to us like there was an opportunity for content distribution networks, which of course, are everywhere now. So I think that idea came to a lot of people at the same time. But whereas today, what you're seeing is, content delivery network as a service, from let's say, Akamai or Cloudflare. What we were doing was, content delivery network, as an appliance, in other words, at that time, you could still pay for some rack space and some conductivity somewhere and put your equipment there.

And then our idea was, that you would put our appliance there as the front-end of the whole thing, and that it would help do global load balancing. And it did work, but obviously, the market went a different way. So that startup represents one of my failures, of which you must have some, so I'm glad I had some, but I did not like it at the time.

Dave Bittner Right.

Paul Vixie And then finally this one, Farsight. In 2012, I realized that we had hit on a winning strategy, where there's a lot of relevance and uptake and traction for the Farsight way of doing things. And we have a small, stable, maybe a dozen commercial customers that were actually buying services from, this was my non-profit, I incubated this idea there. And I realized, "I can't do this. I can't make this bigger if I don't have access to capital and also access to employees, who would want stock options." So in 2013, we incorporated Farsight Security as a separate entity, raised money, used that money to buy these assets from my old company, then the nonprofit, and we started operation as Farsight, on July 1st, 2013. So that has been my all consuming activity, in the almost eight years, since then.

Dave Bittner And what does Farsight Security focused on?

Paul Vixie So the problem statement is, that the world is a lot less secure now, than it was before the internet. So when we commercialized and privatized this, we didn't really have a rock solid technical foundation to do it on. The internet was designed by scientists for other scientists, it didn't really have commercial capability. We just used it that way, especially, the web used it that way. So it was very clear that we weren't going to suddenly stop and start over and go back to the drawing board.

Well, what kind of a global internet would actually support what we're trying to do here? No, it doesn't work like that. You take what you have and you make it to what you need. And there's been a lot of trauma from this. Think about the Snowden disclosures in 2013, when he talked about, let a lot of the world know how the world works, which is, "Gee, did you know what the National Security Agency is doing, in terms of monitoring internet traffic?" So that's seen as a threat by people who don't want to be monitored. It doesn't matter if they're breaking the law or not, they think as a matter of principle, that they ought to be able to act privately.

Then you have the whole internet of things, got all these tiny little devices that have microphones and cameras in them and they're connected to our home networks and they are spying on us and our smart TVs are spying on us. So those are a couple of non nefarious, in other words, they're not intended to do harm, but nevertheless threats, to what the internet also represents, which is, "Can we please communicate fluidly with each other and reliably?" But the trouble is that if you do, you will be subject to various types of surveillance. We're not in the anti-surveillance business, but that does inform our worldview, which is that, we're not going to make that problem worse.

So with that backdrop, what we have is an observational company. We sell the ability to find out what happened, even if you weren't there at the time or you weren't nearby. That's why we called it Farsight. So this is not reputation. We're not in the business of saying, "Gee, this domain name is bad. This IP address is good." Whatever. There are plenty of companies, including many of our customers, who do that. What we do is to say, "This is what it did. It's not good. It's not bad. It's just, this is the observation. This is our confidence interval. You should take this into account, in whatever it is you are doing."

So, by sticking to the observational side of things, we've made it possible for enterprises, ISPs, universities, really anybody who has a network full of devices and people, whether they're students, employees, customers, whatever, just connecting all of this stuff to the internet, means that almost anyone in the world, can tinker with your devices and try to break into them and so forth. If you look at the headlines, you'll see this happens every day. So finding out a little bit of who those people are, a little bit of what else those attackers have done, a little bit of how you can predict what they will do next or recognize them if you see them again. These are all things that require observational services. So we are the big kahuna in this. It's only a 40 person company, but we are nevertheless, the leader in the type of security that we provide.

Dave Bittner Well, let's talk about DNS itself. I mean, I know something that you advocate is, running your own local DNS server rather than counting on a cloud service to do that. Can you explain the thought process behind that recommendation?

Paul Vixie Sure. And first off, it has to do with first mover advantage or status quo. In other words, if you want to change the way things work, you need an excuse. You need to be able to say, "I want to change the way things work, and this is why." So it used to be that this service was provided fairly near the edge. Whereas, most companies were offering it to their own employees and customers, universities offered it to their students, ISPs offered it to their customers. And then in the early 2000s, there was a business plan minted, there was an opportunity to collect a lot of data, sell a lot of services and it relied on trying to get DNS away from the edge and put it in the core of the internet and obviously, that was successful.

You look at Google with, And IBM with and Cloudflare with There's 200 more addresses that would be four numbers repeated with dots in them, each one making the claim that they're somehow better than the other 199. It turns out, that almost anything you want to do on the internet begins with a DNS transaction, milliseconds in this case, matter. So if you're doing a lookup from a DNS service that is maybe 15, 20 milliseconds away, so that might be several hundred miles, than that delay is going to insert itself. After every mouse click, you make in a browser, you won't have an opportunity to get that information really, really quickly, as you would have if you were running a recursive server inside your own network.

There's a huge technical advantage and a performance advantage, to running the service yourself. Secondly, there's a security problem with putting it in the middle, actually there's two. And so what happens, if you put something in the middle of the network, is that your traffic to that service is then passing through a whole bunch of third-parties, middle men. And if they want to look at that traffic and try to monetize it, turns out, there's usually not a law that would prohibit that. GDPR does prohibit it, but that's only for Europe. Generally, it's okay to monitor the traffic going through your network, for whatever purpose you deem necessary. That means, there's been a huge amount of DNS related surveillance.

People hate that. They say, "Well, we don't want our traffic to be visible to people, that might not have our best interests in their heart. So let's encrypt it." In other words, "Let's add complexity, key management the session negotiation. Let's add more delay, more software, more lines of code, more state, more opportunities to have bugs, so that we can encrypt this information that's very sensitive, before we send it to the center of the internet."

Now, what they could also have said is, "Sending it to the center of the internet was never a justifiable approach. If I don't want my information to be surveilled, what I should do is, keep it in my house. So I should keep it in my office, in my enterprise, in my university, because then in order for a third-party to surveil, they would have to break into my building and tap my network in some way." So in other words, we did something foolish, which was, to move this function from the edge where it belongs, where it always was, into the middle of the network, where it became subject to all kinds of new attacks. And then we meticulously added complexity to that solution in order that we could somehow avoid the inevitable costs of outsourcing something that should never have been outsourced in the first place.

But there's one other problem, and it has to do with those content delivery network, I told you about. Traditional, if you look something up from your local server, which I would like everybody to run inside their own network. If you look something up, if it knows the answer, you get a very fast copy of that answer, it's great. If it doesn't know the answer, then it's going to have to go find it, which means, let's imagine you're going to a website that's hosted by Akamai or Cloudflare.

You're going to have to go talk to the name servers for that domain, which are probably operated by Akamai or Cloudflare, in this example, and you're going to have to say, "Please tell me what I need to know because there's somebody tugging on my sleeve back home who wants this information." And that's the thing that is sensitive, right? Because you're indicating that you are interested in a particular thing.

Dave Bittner As one of the pioneers of the internet and you are indeed, in the Internet Hall of Fame. When you look back on those early days, I mean, and you compare that to where we are, in those early days, could you have imagined the things that we see today? Was it possible to have the vision, to see where things are these days?

Paul Vixie Well, some people certainly did. I mentioned that my co-founder for the Internet Systems Consortium was Rick Adams and he also is the founder of UUNet, which was the first commercial ISP. So obviously, he knew there was something going to change and there are any number of, you'd call them bloggers now, but basically, thought leaders in that time, who were talking about what a worldwide data network could do for us, how could it change the nature of society, the nature of the economy? I was not interested in any of that, I was simply trying to get it done.

So if I had really stressed myself and said, "All right, what could we do to make this better?" Then, there's some things I would have done differently, right? There was some boneheaded decisions and recommendations from me, that I would like to get those back. But generally speaking, all I knew was, that the sky was the limit and that we had an extent alternative, which was the OSI protocol suite, sometimes called ISO. This was something that the International Telegraphy Union, ITU, was pushing rather heavily. There was a whole system that the world's telephone companies wanted to put in place to make a global data network. But of course, it was optimized for what they cared about, which was billing. They were planning to make a lot of money from this and they made a lot of choices that were in favor of billing and not in favor of scaling.

So there was a fight or a race, who was going to come up with a set of internet protocols or wasn't internet, protocols for a global data network that was going to somehow be the best one and then win, and get out into the market faster and larger and thus, cause the other one to be irrelevant. So I was very much on the internet side and I knew people who were on the other side and we used to argue bitterly, like Star Wars versus Star Trek, which one's better. Okay, fight.

Dave Bittner Right.

Paul Vixie So there's a lot that could have been different if the internet people hadn't won the war. So my only concern was winning that war and I was not thinking about what would then happen, that is bad. You think about electronic mail and I used to work very seriously in that field. I'm a co-author of a book on a utility called, Sendmail, which was the first internet mail, open-source thing, also came from Berkeley Unix. So there was a time, that I was hip deep in that industry, that's why I ended up starting the world's first anti-spam company, is that I was in the email industry and I didn't like all of the stuff that was being transmitted, that no one wanted to receive.

But if we'd thought about that, if we'd realized, if you build a network for scientists, they trust each other, they behave well, for the most part. If some student somewhere misbehaves, probably going to get kicked out of school or something like that. So if you build a network like that, it's going to be very fragile. And if you then make it available to all of humanity, for anybody who can pay whatever it is, $20 a month for a DSL connection or a modem connection back in the day. If you open it up to all of humanity, then you're going to get different behavior.

It won't be a small number of trustworthy scientists. It'll be everybody and once you get everybody, you're getting some criminals, you're getting some opportunists, you're getting some people who behave extractively. And the protocols aren't ready for this, and the system is ripe for abuse, because it is so fragile, because it was like a boy in a bubble. It was born into an environment that had no threats. That's where we find ourselves and if I had had an inkling of how bad things were going to get, I would have pushed for changes back when it was all very small and you just needed to get agreement from a few dozen people, and you could then change the way the internet worked. It's way too late for that.

So we've got structural defects that are built into the architecture of the internet itself, that make it almost impossible to do safely. So, even though there's a huge amount of work going on, trying to encrypt everything, that turns out to be in many cases, solving the wrong problem, that's because the right problem, can't be solved.

Dave Bittner Our thanks to Paul Vixie from Farsigth Security for joining us. Don't forget to sign up for the Recorded Future, Cyber Daily email, where every day you'll receive the top results for trending technical indicators that are crossing the web, cyber news, targeted industries, threat actors, exploited vulnerabilities, malware, suspicious IP addresses, and much more. You can find that at recordedfuture.com/intel. We hope you've enjoyed the show and that you'll subscribe and help spread the word among your colleagues and online. The Recorded Future Podcast production team includes, Coordinating Producer, Caitlin Mattingly. The show is produced by the CyberWire with Executive Editor, Peter Kilpe, and I'm Dave Bittner. Thanks for listening.