Tech-Ed-2010 Session Summaries 7 More on IPV6

Not so long ago I reviewed Mark Minasi’s Tech-Ed 2008 IPV6 presentation. Frankly, it’s hard to imagine anything better, in terms of style and suppleness. Nonetheless, he does not deliver it every year, and others have come forward more recently to explain this technology. This year, Swiss maven Marc Michault presented a survey of IPV6 which covered much of the same ground, but with added depth on some aspects, making this a worthy session to summarize and review.

I. Packet Overview and IPV6 Addresses

Good overview of the packet construction. The header for the packet has been redesigned, and now weighs in at 40 bytes, mainly source and destination addresses; this simplifies the work of routing. The addresses are written in hex, rather than the standard 10-notation of IPV4. They also are rich in leading zeroes, all of which can be unwritten. Thus,

FD00:0000:0000:0021:0001:0000:0000:5143 becomes

FD00:0:0:21:0:0:5143, which can be further condensed to

FD00::21:1:0:0:5143. Note that this distillation of :0:0: to :: can only be done once.

Subnet Masks are also quite different, and come in at 16 bits (network ID is 48, and interface ID (local name)) is 64. This allows some 65,000 subnets. For most purposes this is enough.

Addresses are also different in terms of their number, in that a given node, or NIC, can have several IPV6 addresses.

Address Type #1. The Link Local, which always starts with FE80, is self-generated, just like current APIPA addresses. This is used, among other things, for router discovery. Now even though the number of available identifiers is truly vast, some have fretted that they might not be enough, and have proposed a way to in effect recycle them. This involves adding to the address a percentage sign (%), and then appending the name of the link/subnet on which this address resides. You know, just in case.

Address Type #2. Global Unicast Addresses are routable and can be used for global communications. The first three bits are set to give us values of “2” or “3”. Here, three bits are used to identify the type of address (currently just “2” or “3” is used) and then 45 bits provide the network ID, and are designed for global routing.

Address Type #3. Unique Local IPV6 Unicast Addresses are labeled as FD. These are not used for global routing, and are indeed for intra-organizational routing. Formerly known as link-local.

Address Type #4. Multicast addresses only work for link-local operations; broadcasting in IPV6 cannot be done. For this purpose, names start with FF. Here are some examples of the most common multicast addresses:

FF01::1 Interface-Local All Nodes

FF02::1 Link-Local All Nodes

FF01::2 Interface-Local All Routers

FF02::2 Link-Local All Routers

FF05::2 Site Local All Routers

Solicited Node, Link-Layer Multicast Address. This part was less successful, and after several re-listenings it remains unclear to me.

2. Auto-Configuration

Two types: stateless and stateful. The former is autopilot; the latter requires some upstream preparation and manual setting. Stateless involves link-local neighbor discovery, and router advertisement. Stateful continues to rely on DHCP for settings which cannot be provided by routers (time servers, DNS servers, etc.).

He makes points similar to those in Minasi’s presentation, including the problems of using MAC addresses in IPV6 addresses. Instead, machines self-generate addresses listed as “tentative’” until DAD has confirmed no duplicate exists. Then it is confirmed as “valid”.

Once the machine has identified itself and the other nodes on its segment, it next seeks to see beyond that segment. It looks for routers, with discovery messages sent in multicast; of course only routers respond. The router response gives the machine a prefix for that router’s network, and the machine appends this to its own segment, thus creating a globally routable address.

That’s the easy, stateless style of auto-configuration. Needless to say, the “stateful” style takes more effort. The addresses can be done manually. This is seldom desirable. However DHCP is also an option, allowing many, many settings to be provided, such as domain suffix, time servers, etc. For most corporate networks, this will remain in use for the foreseeable future. It’s only natural the some day soon, routers will be marketed that will provide these DHCP functions.

Michault does not mention this, but it is worth noting that at this point the machine may have four addresses: the initial self-generated one, the one used within its segment, and the one(s) provided by its router.

III. Name Resolution

Based on the nature of name acquisition, one would think that this should be straightforward. One would be wrong. Michault found three points to make:

1. Local subnet, with Link-Local Multicast Name Resolution (LLMNR). As of mid-2010, the details for DNS-related multicasts are still being worked out. In this case, hosts can contact the multicast address and the message will be forwarded to each machine on the segment, who will in turn respond. This also replaces the browser service.

2. Internet, with Peer Name Resolution Protocol (PNRP). LLMNR is okay for identifying and contacting neighbors. More is needed for large scales of communication. This is also used in non-IPV6 networks. This hashes the host’s machine name and defines a proximity based on this. Now, each node (computer) maintains hashes of neighborhood machine names. These are then reviewed. Successively closer and closer nodes are contacted, eventually (it is hoped) yielding a match.

3. Internet, with DNSV6. Few major differences. Though since addresses are much longer, the A record is replaced by an AAAA record. In addition, since reverse notation is in hex, the PTRs can become quite long. Other than this, there are few changes.

Michault summarized by discussing some current technologies of note which run on IPV6. DirectAccess was the foremost example here.

4. Transition Technologies

Michault finally reviewed some important transition technologies for making this work.

Tunneling. Like other types of tunneling, this approach takes IPV6b packets, breaks them up, and gives them IPV4v headers. Normally this is done for encryption, however in this case there is no added security.

ISATAP, or Inter-Site Automatic Tunnel Addressing Protocol. This provides support for IPV6 applications running on an IPV4 network. This is done by adding in effect an IPV6 prefix to an IPV4 address. For private networks, this prefix would be ::0:5EFE; for a public network this would be ::200:5EFE. Michault gave a good explanation of how this works for transporting IPV6 traffic through an IPV4 network, and also for how this would work in a mixed network of IPV4 and IPV6 segments.

6to4. This is intended for IPV6 organizations that must continue to interact with IPV4 ones. It works by viewing the IPV4 network as the 2002:WWXX:YYZZ:: range and then having a gateway turn the IPV4 address into a hex value, which receives a prefix from the gateway. This prefix-plus-hash is then represented to the routers in the organization.

Teredo. “a pain and a bit of a bitch”, to quote Michault. This allows IPV6 hosts to communicate through IPV4 NATs. This transitional technology should soon fade. In this system, hosts get Teredo addresses which include the assigned Teredo address range (2001::/32), the public IPV4 address of their Teredo server, and the obscured public IPV4 address and port for Teredo traffic on their NAT. Communication runs through a complicated process, including NAT1 to NAT2, then NAT2 to Teredo Server 2, and then Teredo Server 2 to Target Host2, followed by Target Host2 to NAT2 to NAT1 to Target Host1. Michault referred the curious to whitepapers and Joseph Davies’ second edition of Understanding IPV6. Perhaps the most reassuring lesson here is that it’s a transitional technology, not intended for permanent presence.

Conclusion: Few are willing to quickly and fully embrace IPV6, whatever its benefits. Yet its strengths are compelling, and most importantly, the world will soon run out of IPV4s. This will be the IP backbone for the rest of our lives, and it behooves us to understand it. Michault’s presentation was valiant, especially for a second language presenter. The pacing, to me, seemed weak. The arcane was presented at times with too little explanation and sometimes, frankly, seemed unnecessary. Explanations were not always strong, though the speaker was articulate, intelligent, and focused. That being said, the explanations sometimes could have been clearer, and the scope for this talk was perhaps too broad. In retrospect, I’d have recommended a shorter list of topics, covered in more depth.

Post-Script: In an age where we have enough IPV6 addresses for every hair on your head to be uniquely identified, it’s inevitable that some day our coffee-makers, credit cards, and pets will have their own unique addresses. Additionally, now that addresses have six letters as well as ten digits, it’s inevitable that people will look for lucky numbers or ways to include messages in their addresses. Rude suggestions to the FBI, DOD, CIA, for example, could be built into addresses. More reflection on these likely futures would be worthwhile.

Tech-Ed-2008- Session Summaries 6 Mark Minasi on IPV6

So often, when troubleshooting computer problems at home, I’ve typed ipconfig /all, and skipped all the weird hex, going straight for the good old IPV4 stuff. But they keep saying we’ll run out of IPV4 addresses around the same time the Mayan calendar winds down, and we’ll have not one but two global cataclysms. But unlike the Mayans, we have been able to see beyond the apocalypse: IPV6 will wrest order from this chaos.

That’s the promise. Mark Minasi has roared in where angels fear to tread, taken on this tough, complicated topic, read more RFCs than I can shake a stick at, and put together a kick-ass presentation on IPV6 to make it all clear. A truly exceptional insightful Tech-Ed session, one of the best.

Mark begins by asking Why Learn Now? and explains some facts and benefits.

- Fact: Chinese has 1.2 billion users and is already constructing an IVP6-based network. He points out that this raises the daunting prospect that future RFCs will be written in Mandarin.

- Fact: We are running out of IPV4 addresses, and you can even get an IPhone app that shows the countdown..

- Benefit: Better, easier automatic configurations, which will supersede DHCP for this purpose.

- Benefit: No broadcasts

- Benefit: Little need for hardware upgrades.

In his presentation, Mark covered the differences with IPV4, unpacked the serpentine names, explained name _types_, and showed how IPV6 works in practice.

How Does IPV4 Compare With IPV6? Currently, in IPV4, I can talk to you if we are in the same subject by shouting on Ethernet. Otherwise, I have my local router forward the message. So the question is should I shout or should I route. How to make this decision?

However, with IPV6, many things are different in word and in fact. There are almost never subnets; instead we say “link”. “Link” means connection to the network. And for IPV6, no DHCP is needed; the machine addresses itself. The next confusing thing is how to read addresses. After all, IPV4 addresses are 32 bits, and IPV6 ones are 128 bits (!). This allows for a truly astronomical number of addresses. Also, it uses hex digits. Colons are placed between each block of four simply “to improve readability”, according to an RFC. Even so, this is a chore to scan. So, there are various schemes for abbreviating them. There are several rules which allow this:

- Trim leading zeroes. :0831: becomes :831: and :0003: becomes :3:.

- A series of 0s can be collapsed to just :: So, for example, FD00:0:0:0:B13A:831:0:6789 becomes just FD00::B13A:831:0:6789.

- How is 127.0.0.1 expressed in IPV6? Very simply: ::1.

Network names are handled differently in IPV6. In IPV4, we would express a network as 210.2.3.0/24. This meant that the ISP controlled the left 24 bits (out of 32, remember), and you controlled the last eight. Things will soon be vastly different.

Names for Networks. In IPV6, about the only carryover is the use of the slash to indicate the number of bits reserved for the network. However in IPV6, this is referred to not as the “network number” but as the “prefix”, and since there are only two used (48 and 64) you hardly even need to articulate them. Old style subnet masks are not used. As someone who has learned and taught the art of subnet mask calculation, I can say that this will not be greatly missed. Mark gives a great example of how this works: “21FB:540A::/48” includes this information implicitly: the left 48 bits are not under the owner’s control, which means the owner has an 80-bit address to work with, and all the IP addresses will start with 21FB:540A:000:. This example is especially relevant because most assigned IPV6 addresses will reserve 40 or 64 bits. Going back to the example of having an 80-bit address space, this means that you are given this many addresses: 2 to the 80^th power. This is enough for most purposes, even for my book and music collections combined.

Mark also explained clearly how to get from a network name such as 21FB:540A::/48 to knowing what addresses will work for our network. It goes like this:

1. The /48 in 21FB:540A::/48 means that the ISP controls 48 bits and we control 80.

2. The 48 bits we do not have breaks down into 12 hex digits.

3. But when we look at 21FB:540A we see that only 8 are used. So four are missing. And since they are unspecified, we know they are zero.

4. Therefore all the addresses on this network start with 21FB:540A:000:

One benefit of using a straight 48-bit reservation for network purposes is that designing routers will be easier with a single consistent, almost universal metric for this. So, here’s how this will play out. Network authorities get 48 bits. The host address gets 64 bits. The remaining 12 bits go to subnet IDs (more later). This usage of “subnet” is what we used to refer to as a site. In this system, we can assign up to 65,000 sites (!), which is enough for most purposes. Then, since we still have 64 bits available for naming machines at each site, we can have 18 quintillion machines. And keep in mind that everyone gets a /48 network. The local Girl Scout team gets one. McDonalds gets one. The US Army gets one. So, the current IPV6 system allows up to 137 billion organizations (can be expanded even more to 250 trillion, if need be). Smaller organizations can opt for just one subnet of 18 quintillion addresses if they so choose. It’s pretty clear that IPV6 was architected with a long-term frame of mind.

Types of Addresses. So far we have reviewed some of the major features of how IPV6 works. Now, let’s take a look at the major types of addresses, since this is the aspect we will see most frequently, and of course which turns up most often on exams.

IP addresses are handled quite differently in this new system. The good news is that our hardware is already able to handle this. There is no bad news. With IPV4, normally a NIC has but a single IP address. However with IPV6, multiples are the norm. In fact, normally your machine will assign itself a bogus temporary address just to find routers, and then it gets a real, routable address.

Now, IPV6 addresses can be divided into three main types. Mark lays out their differences.

Unicast: Addresses aimed at a single connection on a single machine. And there are four sub-types of these:

Global Unicast: These are routable across the Internet.

Link-local Unicast: Only works within a link/subnet. Packets from these are dropped by the first router they encounter.

Site-local Unicast: Can be routed within your organization, but not outside to the Internet.

Unique local Unicast: like private or 192.168.x.x addresses.

Multicast: Used for one-to-many connections. They provide some of the features of broadcasts in IPV4 (but not all). And they also are used to find your local router and other members of your link/subnet. This is in essence a discovery service.

Anycast: Used to find routers mostly. Can also be shared.

Broadcasts are not included within IPV6; multicasts are intended to handle this. With IPV6, machines join a multicast group and are remembered by major hardware such as routers and switches. The addresses used for multicasting have a distinctive feature and start with FF0 or FF1. There is also a single “magic” address for multicasting; send a message to this and it is forwarded to all link/subnet members. Some other multicast addresses are tasked with specific functions. Minasi gives the example of FF02::2; messages sent to this address will go to every router on your link/subnet. So, when a machine comes on, it makes up an address to send a message to this address to see if anyone else is there. Once it received a response, it then fabricates its own proper address for that link/subnet.

Global Unicast addresses: These are globally routable. Their top three bits are 001, and the remaining one can be 0 or 1. Since 0010 equals 2 and 0011 equals 3, this means that global Unicast addresses always start with 2 or 3. Internet-connected routers only forward packets from these types of addresses; all other are discarded.

Link-local Unicast address. Only used for communication within a link/subnet; routers discard them. Note that their top 64 bits are FE80:000:000:000, or FE80::/64. Formerly, the rest of this address was the hardware’s MAC address, but this was too prone to abuse, and now is simply random. Comparable to APIPA addresses.

Site-local Unicast address. They cannot be routed across to the Internet, but they _can_ be routed within a “site”, and are meant to be so associated. Their top 64 bits are felicitously named FEC0:0:0:FFFF. However, the effort to program for this was daunting, and it is being shelved. Microsoft added to this by defining a “well-known” address clients can use to look for DNS servers, and posited three such: FEC0:0:0:FFFF::1, 2, and 3. Regrettably, IPV6 is evolving away from this.

Unique Local Unicast Addresses. Starts with FD00. This seems to have been established mainly as a transitional device for organizations used to 10.x.x.x addresses, and who do not want to give this up.

So, in summary, we see:

FE80: Unicast link-local.

FEC0: Probably DNS server (going away)

FD00: A private address.

FF0 or FF1: A multicast address.

2 or 3: a globally routable Unicast address.

What if an address starts with 0, 1, 4, etc., all the way up to E? Currently undefined.

That does it for the top 48 or 64 bits.

But what about the bottom 64? MAC addresses, already unique, might seem a reasonable source. Use them, or run them through an algorithm to get a 64-bit value might seem an easy way to proceed. However it is estimated that MAC addresses will run out in less than a hundred years. The only issue with this is that any address based on MAC addresses would be individually identifiable anywhere. There are few occasions when this would be desirable, except to Australia or Chinese governments. So IPV6 now auto-generates the 64-bit address value instead.

So currently, address acquisition goes like this. First device self-generates an FE80 address, and looks for routers. The routers provide the top 64 bits of the address. Then the bottom 64 bits are randomly generated. And that’s it! Now, in IPV6 DHCP still exists, but with different functions. It might provide other referential addresses (like time-servers, for example), or it might provide a static address, or a DNS suffix. Etc.

Getting back to the random generation point. It’s always possible that in a sufficiently large organization, like the federal government, two identical random addresses might somehow be generated. An IPV6 function called DAD, or Duplicate Address Detection) addresses this, by querying if anyone else is using a given address. Since the chance of addresses coinciding is so remote, the address goes into service while this DAD check is taking place.

Globally routable addresses are obtained the same way: from routers. Here, two addresses are given. One for two-way communication, and the other for random web surfing.

Mark ended with some brief comments on DNS and name resolution in IPV6. There’s no NetBIOS or WINS. A new name record, AAAA, represents the existence and availability of an IPV6 address.

Once again, Minasi has taken the most complicated, mind-numbing technical topic, and made it so easy even a board of directors could understand it. For a brief, clear outline of how IPV6 works and its salient points, this presentation, still available at Tech-Ed Online, is the place to start. The only thing I would add to this presentation is that with so many addresses being allocated for each individual, it’s inevitable that people will start constructing their own cute IPV6 names, and some will start designing ways to choose “lucky” IPV6 addresses based on one’s astrological sign.

In 2010, last month in fact, Marc Michault returned to this subject with a presentation that covers much the same ground with some new insights. I’ll cover this next.

Tech-Ed 2010 Session Summaries 5: Benjamin Armstrong on Hyper-V and Dynamic Memory

In previous years, Jeff Woolsey has been one of the leading Microsoft speakers on Hyper-V; this year he was absent, but his place was filled by Benjamin Armstrong, feature manager for the Hyper-V team. He was clearly pumped about the new release and its major new feature, dynamic memory, as came across in this clear, focused, productive presentation.

Why care about this? Part of this comes from the fact that we seldom know exactly how much RAM an app will need, since this depends so much on factors such as number of usage, hours of use, types of use, etc. And for VMs this is even more so. Armstrong gave some good examples of “best practices” for this. And since memory can be the biggest single budget item in a server quote, it merits attention and thoughtful calculation. Given this centrality of RAM, for Armstrong and his team, one pivotal concern was optimizing its use. And with this in mind, they gave extended thought to:

- Scale of variation in memory utilization for workloads

- How well the systems were sized initially

- Something that would work for both servers and desktops

- Minimizing system overload

- Satisfying users

How Does it Work? What Does it Do? They are able to add memory in realtime. One difficulty they discovered with this was that on server boot, the BIOS reports memory slots as empty or full, and one early attempt led to changing the BIOS to report the presence of 4096 memory slots on the motherboard. An early build, needless to say. Early efforts also led to weird bugs that had no parallels in the real world. He gave the example of “After adding 60 sticks of memory in half an hour, the 61^st was not correctly enumerated.” With removing memory, other issues emerged, with the result that for this first release they are using a tactic of ballooning, rather than the exact reverse of adding.

Naturally, VMware and its memory overcommit tactic entered this discussion. Here Armstrong made the point that the reward of such approaches depended on how well you sussed out your memory approach at the outset. So, someone with a penchant for four-gig RAM VMs would see vast returns on this, whereas a tighter-fisted admin would not.

The system requirements for using dynamic memory are predictable- SP1 for W2K8S R2 SP1. It will host Server 2003, 2008, and 2008 R2, 32 and 64-bit versions (where applicable). In addition, Vista and Win7, 32 and 64-bit, Enterprise and Ultimate Editions will work with this.

Next he talked about the Dynamic Memory Architecture. Key points here were the memory balancer, operating in user mode, which balances the needs of various VMs for memory. He illustrated how this would work, with the memory manager able to take memory from one VM and commit it to another. The main settings it is guided by are the startup memory and the max memory. The startup memory represents the minimum needed to run; 512 megs is the default for this. The max value is an astonishingly generous 64 gigs!

Availability and Priority are also discussed. Priority is assigned a value, 1 to 10,000, with the default being 5000. Memory buffer was briefly covered.

Armstrong next presented a demo to show dynamic memory in action. This was a mixed success. Soon enough it was followed by a discussion of changes to Root Reserve. This refers to memory always reserved for the parent partition but sometimes this reserve is jeopardized by “rampaging VMs”. The new Hyper-V manages this more smoothly, albeit at the cost of some memory not being available for VMs.

NUMA Management was also changed. Armstrong mercifully explained what NUMA systems are- an architecture for memory management. Without it, the cores on a motherboard can generate contention as they reach for memory and board resources. In this, the board is split up into logical groups, or “NUMA Nodes”. Each has its own bus and memory. There is also a backchannel which allows NUMA-ignorant software to still work. However, the back channel’s speed varies with systems, with the result that performance can range from adequate to awful.

Currently, Hyper-V tries to get all memory from a single NUMA node; when it cannot, it then spans nodes. We can specify which NUMA nodes are used, but this is only available through the WMI, not the GUI for VM management. This of course reduces the spanning, and the attendant chance of hits to performance. The concern is that with machines not dynamically adjusting their memory, they might wind up spanning NUMA nodes, which would have dire consequences for performance. Therefore, SP1 will allow the disabling of node spanning. This can be done from the GUI. The result is that the computer seems to have more than one computer within it as the NUMA nodes are separated.

Memory Techniques/Competition. Here Armstrong explained the philosophical differences between MS and VMWare on this subject. The first difference pertains to knowing system performance. In the course of architecting memory overcommit, VMWare saw uncertainty in the reliability of counters from guest system memory use, and chose instead to construct a black-box model. Microsoft obviously had no such difficulty, and could build on a strong understanding of the guest OS inner workings. The second thing is that VMWare started with swapping for memory management, and ever since has been working to make this work better. MS started differently by setting min and max, and letting the platform decide.

This is one reason for the differences in dynamic memory versus overcommit. He makes a distinction between oversubscription and over-commitment. Oversubscription, as Armstrong describes it, is selling more tickets for an airline flight than you have seats; overcommitment is when everyone shows up.

Why use the dynamic add/remove model? For VMWare, transparent page sharing is the "golden-haired child" of their memory management. They base this on external inspection of the VM's memory management. They hash the memory pages, and then compare them, looking for common bits, and consolidating from there. Microsoft studied this and decided to go down the add/remove memory route because the hash creation and review process is quite processor-intensive. The hash process for VMWare can be quite slow, and can take hours to complete. So it is not dynamic or responsive. Furthermore, recently Intel and AMD system architectures have introduced (much) large pages- 2 megs instead of 4K. This was engineered to help with hardware-related virtualization memory management, among other reasons. Obviously the tremendously complicates the effort of hash comparison. And it is no surprise that the most recent ESX turns this transparent page-sharing off since it offers few benefits.

Second-Level Paging, or paging at the virtualization layer. There can be many problems with this, as Jeff Woolsey has already elaborated. The problem is that when virtualization is done at this layer, the problems come from the fact that when you do this you do not have the context of the guest OS, which knows what everything is doing. External inspection cannot do this. VMware's technique for identifying memory pages to share is random selection, which Armstrong says is not a bad choice because it gets you 75% of the way to optimal, and has no CPU overhead. The biggest advantage of this approach is that it always works.

Armstrong referred briefly to other techniques: Virtual Box talked about memory fusion, corresponding to MS's guest-directed page-sharing. VMWare has spoken of memory compression; this evoked grins at MS because it's such an old idea. The key message here is that this story is not over, and efforts continue to find new tools and techniques. Hyper-V is still a work in progress, with hopes and plans for the future. And obviously it helps to be in the same building as the Windows kernel team.

Overall, clear and informative. One of the things I appreciated the most was the differentiation not merely of technics and tactics, but the juxtaposition of different ways of looking at memory and its management. Worth seeking out and watching.

Tech-Ed 2010 Session Summaries 4: Andy Malone: Cybercrime- The Gathering Storm

Andy Malone has presented on security before this year, but this was my first chance to hear him speak. This was a very strong presentation, and I’m going to look up his other material most definitely. This session covered the tools, tactics, and backgrounds of contemporary cybercrime. It’s come a long way, and Malone helped us understand the threat it presents, and how to counter it. The summary below only covers the highlights of this deep dive.

Current targets of cybercrime: Top Phishing targets- mostly financial of course, at 54%. Payment services came in next at 26%. In addition, there as a curious category called “other”, with 9%; here Andy spent some time. These represent valuable but small and soft targets: dentists, doctors, hotels, etc. They are soft targets, but they offer nearly the same juicy fruit that banks do. They are also much less prepared.

Current avenues of cybercrime: In addition, third-party applications remain king of the hill when it comes to vulnerabilities: Adobe, Flash, pdf-readers, etc. Against this background, Andy identified the current cybercrime trends as being unpatched client-side software, vulnerable internet-facing web sites, zero-day vulnerabilities, and greatest of all, insider threats.

Current Contributing Trends: social engineering, social networking sites, file-sharing sites, pdf and flash files, and individual vulnerability to outsider temptation. The latter seems clearly to be the most serious, in which people within an organization succumb to temptation to just “copy one little file”, or get “just this one little thing”.

Just who does this?: So far this is all fairly standard stuff. But afterwards Malone started dishing the dirt. This started with an exploration of just _who_ undertakes cybercrime now. Not simply random hackers, but companies interested in corporate espionage, Russian criminal business groups, foreign government/military interests, con artists, etc. The widespread use of commonly-available hacker tools from standalone script kiddies to national/corporate spies. The recent break-in experienced by Google at the hands of Chinese agents is the best example of the latter.

The most likely career paths for these groups include coders and programmers, of course, to prepare new malwares but also distributors to provide and sell stolen data, tech experts to maintain and secure the organizations IT assets, hackers to identify and exploit vulnerabilities, fraudsters to conduct phishing and social engineering, hosted system providers, cashiers to manage accounts and money, money mules/smurfs, tellers to launder money, and of course organizational leaders, who might not even be technically savvy.

Malone didn’t just show powerpoints though for this; he displayed a Mandarin “Hacker Intrusion Activity Model Chart” from a hacking school to show how meticulously some groups in some countries have systematized this. Fortunately he also provided a translation.

Besides this obvious organization, there is a deep toolkit of resources and tactics; most are readily available, such as keystroke loggers, remote webcam activators, virus construction toolkits, etc. Malone gave the example of a disk he bought for five dollars in Moscow with over a hundred such tools.

Example: Here Malone gave an example of a small but potent group which sold credit card numbers (his example showed one US hotel chain being a source for these) and was allegedly involved in money laundering to the tune of $200,000,000. However, they did not stop their services here and also offered genuine passports and visas, including American ones. These and other attacks were all dependent upon insider assistance, either recruited or planted. The growing lack of commitment to organizations (and vice versa) only exacerbates this. And it is quite unlikely that the mutual lack of commitment seen with modern workers and employers will change any time soon.

Monitoring Resources. One solution is monitoring software of various types, though of course these are all two-edged swords. Malone gave the example of a keystroke logger which also logged web page visits, searches, chat sessions, etc. One of the most impressive apps he demonstrated was Maltego (he showed the full edition). With this tool, (among other functions) you type in a company’s phone number, adjust the desired granularity, and any site with any reference to this phone number in any way is displayed. These sites can themselves in turn be interrogated. Social networking sites are included in this.

Another powerful resource was Foca (carefully pronounced). With this you can specify all the files of a certain type (like pdf, for example) at a selected site. Then every single one from every single site is displayed, including sub-sites. Then, these files may be selected, and they can be downloaded. But that’s not all. Then, all the metadata from these docs can be extracted, including creators’ usernames, software used, cited email addresses, and even network information, such as every single IP address of every machine contaacted. As Malone says, it is “eye-popping”.

He also demoed Memorize, a website downloader. NetSparker, community edition, a vulnerability scanner, pulls down web sites and scans them for vulnerabilities. The full edition shows passwords as well.

Effects: This concluded his all-too-brief survey of mayhem. Put to systematic, coordinated, malicious ends, the mind boggles. The Georgian cyber attacks are a great example: as the Russian army was invading in August 2008, other Russian entities were launching devastating, strategic attacks on Georgia’s information infrastructure, using botnets and civilian volunteers with improvised software.

The Human Factor: This tight, reasonably technical section was followed a looser, catch-all sort of section about miscellaneous security lapses and funny mistakes, followed by a focus on inside man recruitment.

Malone posits three types of people: engaged/motivated (he mentioned Steve Balmer), not engaged (mentally checked out, typically government bureaucracies), the actively disengaged (negative, undermining, accessible). This third type often has great accesses and great potential for corruption.

How are these people recruited? They go to Black Hat conferences, DefCon, demonstrate their technical skills, advertise services by IRC or similar means. Such industrial espionage has a low cost of entry, a high rate of return, small probability off detection, even lower chance of prosecution, and even even lower chance of meaningful punishment.

Responses: Malone next discussed how to respond to cases of suspected espionage for organizations; this was fairly standard. This was followed by a review of basic forensic skills which should be available in every organization. Then he surveyed some useful, accessible tools for admins and forensics, such as simple ways to capture main configuration information for a computer, very similar to MS “Cofee”. Passware aids the recovery of passwords from files through dictionary, brute force attacks and resetting the local admin password (!), and other resources. An MS product, Security Compliance Manager, gives recommendations on proposed changes to make (depending on your requirements), and even does them for you- this includes settings such as local policy, etc.

Overall, extremely informative in terms of strategic insights (the nature of current threats) and tactical ones (apps mentioned above). Of all the batten-down-the-hatches security presentations this year, this was clearly the best one I attended.

Tech-Ed 2010 Session Summaries 3: Mark Minasi on DNSSEC

Tech-Ed 2010 Session Summaries 3: Mark Minasi on DNSSEC: Why you care, What you can do, How Windows can help you.

Mark Minasi was on a tear at Tech Ed this year with several great sessions, including this impressively researched and engagingly delivered one.

The problem underlying this session is that the current nature of DNS makes it quite possible for others to take over DNS services and masquerade as (for example) banking domains. This is slowly changing, but between 2010 and 2014, an entirely new form of attack will likely take place, in response to the closing of the window of opportunity for this. More on this at the end.

Background of the Problem: DNS “spoofing” (alternately, “cache poisoning”) has been possible for many years, however recent research and developments in security have made such hijacking much easier. In such an attack, an address is taken over, such as blogspot.com, and traffic to it is redirected to another location. The implications for commerce need little elaboration. As we will see, there are solutions, but every link in the DNS name resolution chain must be part of the solution for this to work.

DNSEC and zone signing are a solution. Now, signing your own zone for most of us is not critical. However, we _do_ need a DNS infrastructure that allows you to validate the DNS information from your bank. And now Windows supports this on a server and client basis.

So how does this work?

1. When you seek an address, you request goes to your ISP’s DNS servers.

2. Your ISP’s DNS servers ask for the IP address to be used, indicate the port to be used for response, and designate a transaction ID (TXID) to be used in that response.

3. This is sent to your ISP’s DNS server, which then sends this address to you.

4. Then your ISP’s DNS server stops listening.

And 99.9% of the time, this is what happens. But there is a risk that goes like this.

1. When you seek an address, you request goes to your ISP’s DNS servers.

2. Your ISP’s DNS servers ask for the IP address to be used, indicate the port to be used for response, and designate a transaction ID (TXID) to be used in that response.

3. A deceptive ID is sent to the designated port with the designated TXID. This is then sent to you.

4. This is then adopted and sent to you. The ISP DNS stops listening. You are screwed.

The only form of authentication used for this is the TXID and the selected port number. But they were really meant to be traffic codes, not authentication tools, and hence are quite weak.

Brute force FTW: So, then, trial and error guarantees a certain number of successes. Moreover, port designation has been non-random and therefore predictable. For another thing, traffic between DNS servers for many years was always on port 53, even across different operating systems. On top of that, until 1997, TXIDs were sequential, not random. This made successful attacks fairly simple. In the past few years Windows has improved the assignment of TXIDs, the assignment of port numbers.

The danger is that a well-organized attack could take control of several large ISPs’ DNS servers for an hour or so, which would be long enough to harvest vast numbers of usernames and passwords for major financial institutions. A bot army of (according to Minasi) 50+ million machines could continually try to poison the caches on the DNS servers of major ISPs; eventually some success would be certain.

Background on RR (Resource Records). With DNSSEC, we “sign” a zone of ours. This is done by adding new types of resource records to the ones we already know, such as MX, NS, A, etc. In this system, every resource record gets a new record known as an RRSIG. Then we create and take a private/public key pair and encrypt this RRSIG with the private key. Then, to verify this key, visitors/clients use the public key to decode the RRSIG. This public key, for convenience’ sake, is itself stored in another new type of DNS record which is long, ugly, and contains an identifier for the public key used. Another new type of record contains this.

So, how does this work? First we establish that a putative zone is internally consistent. Here’s how.

1. Get the A (address) record for the target, a very standard task. (In fact, this all also applies to other types of name records too.)

2. Run this record through a hashing algorithm. This yields a hashed version of the A record.

3. Then we go back to the target, whose RRSIG record also contained a hashed A record for this server.

4. Grab that RRSIG and the DNSKEY, which contains the decryption key for the RRSIG.

5. The DNSKEY is run against the RRSIG. This should yield a hash which is identical to the hash in step 2. This proves the server is internally consistent, but that’s all. A fake could be internally consistent, after all

Process-wise, this is smart, but slow. And obviously our zones are going to be a _lot_ bigger as a result of this.

The answer then is the DNSKEY. We can verify this against the parent zone, the grandparent zone, etc. Here’s how:

1. Get DNSKEY from target and create a hash from it.

2. This is then verified against the top-level domain has for this key. If it is consistent, then the DNSKEY is reliable, and matching hashes from it can be trusted.

The top-level domain, if we do not trust it, can itself be verified against the root domain.

One problem that can come up is that if we do not have a record matching a query, a response is sent nonetheless. This means that an adversary could send numerous requests for invalid server names, and learn about or organization by studying the responses. One additional feature on DNSSEC, NSEC3, remedies this by sending hashes back, not hostnames. Strangely, this is not supported on Windows Server 2008 R2.

How to Implement? Given these immense benefits for thwarting domain takeovers and cache poisoning, how do we make this work? Well, many of us don’t need it. No one’s going to poison our cache unless you’re worth the effort. So DNSSEC is of special interest to banks, financial organizations, Dead.net, etc. Beyond this though, all the servers between you and your target must be DNSSEC-aware for this to work. Oh, and your PC’s DNS client has to be DNSSEC-aware too.

One issue with this, as you can see, is that all computers between you and your target have to be DNSSEC-aware. But not all are. The trend is for companies increasingly to sign their zones, though. And to do this, you need Windows Server 2008 R2 for your DNS servers, and the zones needing to be signed are the Internet-facing ones.

Workarounds: Currently, VeriSign is scheduled to implement DNSSEC this year, and the .com top-level domain should be onboard by 2011. Fortunately, while we wait for the rest of the world to make this work, we can use workarounds. Some top-level domains have been signed, such as .org, .se (Sweden, of all countries!), and these can be used as bases for signing as an interim step; they are known as “islands of trust”. Take a hash of their DNSkeys, and use this on your server. This is quite similar to the way we use root certificates. Just as we trust VeriSign, we trust .org or .se.

Icann created a listing of such anchors in February, 2009; this will be taken down when the other top-level domains implement DNSSEC themselves. These were not baked into W2K8S R2, but can be added. The continuing problem is that XP and Vista cannot work with this system. However Windows 7 and W2K8S R2 can. With then, DNSCMD does the signing if asked. This works with static zones. These two steps avoid excessive, bandwidth-sagging traffic.

In addition, the Windows 7/ W2K8S R2 versions works by offline signing, in which the zone is hashed, and then uploaded. One significant change is that the zone signing keys should be changed on a monthly basis. However, when this is done, then the key-changing client needs to update its DS key (an entry in the zone file, remember) of its far upstream DNS partners. This seems like a phenomenal amount of traffic. And it would be. Ideally, we should have some way to support signing, but without the huge traffic surge.

Two Keys Are Better Than One: The solution to this is to have two keys rather than one. One key, the ZSK or Zone-Signing Key, does get changed every month, and which handles most of the zone, but does not put a DS record upstream. This can be changed as often as desired without incurring traffic or service charges. On the other hand, the second type of key, the KSK or Key-Signing Key, and put _this_ in the parent zone. This presents the reliability of an upstreamed key (i.e., one with a DS record in a parent), together with the convenience of one which can be quickly and easily changed for local purposes.

Creating Them: Mark quickly covers the process of creating the keys, noting the use of DNSCMD for this, and giving examples of the commands:

1. Get a text copy of the zone: (from AD-I): dnscmd /zoneexport zonename exportfilename.

2. Create and back up the KSK and the ZSK: For KSK: dnscmd.exe /offlinesign /genkey /alg rsasha1 /length 2048 /flags KSK /some yourcompany.com /sscert /friendlyname KSK-yourcomany.com. Then, for ZSK, dnscmd.exe /offlinesign /genkey /alg rsasha1 /length 2048 /some yourcompany.com /sscert /friendlyname ZSK-yourcomany.com.

3. Sign a zone: dnscmd /offlinesign /signzone.

Near the end Mark also covered trust anchors. He reiterated that the need for these will fade within two years as more top-level domains come onboard with DNSSEC. That being said, for the current need, he showed how to find them, how to read their nomenclature, and how to use them.

Client-Side: The client-side aspect of this received little discussion until the very end, mainly because there was little to say. Only Windows 7 has a DNSSEC-aware DNS client, which gives the _option_ of requiring signatures, through the Name Resolution Policy Table. Obviously the reason this is optional is because most sites still do not use signing. The Table can be enacted through a rule in GPEDIT (in Windows Settings>Name Resolution Policy), and then restarting the DNS client.

At Tech-Ed, Mark discussed the short-term implications of this. The current vulnerability to hijacking and poisoning will end within two or three years. The current vulnerability is exploited by thousands or millions of bots. The natural implication, for criminal organizations is that their window of opportunity for exploiting this is closing. As a result, it’s likely that some serious and sustained attacks will take place in the next very few years.