Crossbow's golden arrows

Crossbow's golden arrows

Summary: Sun's Project Crossbow, now in the OpenSolaris code base, separates physical network interfaces from the OS code accessing that hardware - meaning superficially that it virtualizes networks, more substantively that OpenSolaris on AMD can now compete with Cisco/IOS at 10 cents on the Cisco dollar - and more deeply that people who care about network security now have an exciting new option.

SHARE:
TOPICS: Networking, Oracle
18

Sun's Project Crossbow has now been released to production versions of OpenSolaris for x86 - and will, RSN, appear in supported releases for SPARC.

Nominally what Crossbow does is fully virtualize network interfaces - interposing a virtualization layer between the hardware and the OS to allow mapping of many virtual network interfaces, each essentially indistinguishable in operation from physical hardware, to one real card or port - here's the 411 direct from the Crossbow development site:

Key Features Integrated in Nevada build 105 (Dec 4th, 2008) and available in the next release of OpenSolaris:

  • Performance & latency improvements

    • Dynamic Polling and H/W Classification
    • HW and S/W fanouts to multiple cores
    • Parallelizing the stack all the way from HW to application

  • Virtualization

    • Virtual Wire(TM) - Ability of create Network in a Box
    • NIC Virtualization - HW and S/W based VNICs
    • Etherstubs (Virtual Switches)
    • Service Virtualization - Flows
    • IP Instances for Zones

  • Resource partitioning

    • Bandwidth partitioning for NICs/VNICs/Flows
    • CPU resource and priority assignment on per datalink (NIC/VNIC/Aggr) bassis
    • Class of Service support based on Diffserv tags (DSCP)

  • Flows

    • Based on IP addresses, IP Subnets, Transport and ports
    • Bandwidth control and priority for Flows

  • Analytics/Observability

    • Real Time usage for flows and datalinks
    • Usage history for flows and datalinks
    • Fine grained, per link statistics like packets received via

  • intr/poll, chain lengths, Tx block/wakeup count etc. (Currently tracked by the kernel on per datalink/flow bassis and available from 'mdb' macros such as 'mac_flow' and 'mac_srs').

On its face Crossbow addresses a number of typical sysadmin issues with the use of network interfaces in zones or containers - particularly the problem that the lack of absolute isolation from the physical hardware meant that NIC access had to be co-ordinated outside the zone management function.

That's no longer an issue: you can now (or will soon be able to) consider network interfaces as integral to either containers or zones and move them around with the same cheerful joie de vive we've all previously been applying to files, users, and rights.

To be honest my personal reaction to all this is along the lines of "oh, Whoopidee do!" because the whole business of sticking multiple NIC cards in a machine and tying them to applications is a Wintel/x86 thing with no role in a well run SPARC/Solaris shop where normal Unix device sharing works perfectly well.

However... Crossbow is important -and thus well worth your time to learn about- in two distinct ways:

  1. Crossbow offers, for x86 users, an important reason for choosing OpenSolaris over a BSD, Linux, or even Windows alternative; and,
  2. Crossbow opens an entire new applications market in network routing and packet management for Sun.

A high volume, high reliability, switch router from someone like Cisco can easily run into the thirty thousand and up range - and everything it does can now be done on a $6K AMD box running OpenSolaris against a couple of multi-port gigabit ethernet cards.

When Sun bought Cobalt back in 2000, part of the dream was to build on the company's expertise to make and sell a true network connectivity appliance but, between technology limitations and the destructive response from middle management, it didn't happen - then.

Today the stars look aligned to make this work: there's more open source expertise, the technology is vastly better, and any volume achievements will not depend on Sun's regular sales channels.

The latter will, I think, prove to be important for Sun first and the industry second because it reduces data center costs and clutter quite considerably while extending the typical Unix sysadmin's span of control into network management.

That's cool - and even cooler? There's actually something there for guys like me too: Crossbow fits the packet handling changes made to the N1 during the N2 transition - thus making use of the on-board cryptology processors in that machine easy for people who want to use Crossbow to handle routing on secure networks simply by creating dedicated zones on Coolthreads servers.

And that's a big deal - because it wipes out two very dangerous sources of vulnerability (IOS and the Cisco guy), improves performance, and eliminates a couple of cost sources into the bargain. Progress, at least as far as I'm concerned, doesn't get better than this - and that's the bottom line.

Topics: Networking, Oracle

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

18 comments
Log in or register to join the discussion
  • Ok so all this network stuff is really cool...

    ..but like Sun's CPU stratergy it seems as if it is competing with itself. Will this tempt away Linux customers. Ummm.. NO! Will it cause Cisco to loose market share. Umm.. NO!

    Perhaps it will make existing Solaris users stick with Solaris. I have yet to find one of my company's many customers say that they moved back to or decided to stay with Solaris because of ZFS or DTrace. The biggest reason for using Solaris is because of existing Applications of some sort that have not been ported or people are too scared to mess with.
    junknstuff@...
  • IBM's had this for years

    With IBM's AIX VIO stuff, you can virtualize NICs and hard drive partitions. Ford utilized the HD stuff a lot, whereas I didn't see much NIC virtualizing. Probably since they are Cisco's best customer . . .

    Because of the Balkanization and Gerrymandering of UNIX, Ford had the network department put in charge of DNS and DHCP. They outsourced it to a group that placed Sun ultrasparc workstations in Ford buildings. Over the years those servers were NOT upgraded, so they really became a SPOF. Never mind that the network group had cutting edge Cisco equipment that was MORE than capable of doing this.

    When sysadmins needed to fix issues around this, they had to deal with a less-than-responsive vendor. It was a true ClusterF. The network group was unwilling to give up the "control" over this function - thus Balkanization. Since the management didn't get involved in this power struggle, they basically sided with the network group over the sysadmins - thus Gerrymandering.

    I'm not sure what they do today (that was in 2007), but I wouldn't be surprised that they still rely on >10 year old technology.
    Roger Ramjet
    • VIO server

      is part of the LPAR system. It provides a dedicated (hidden) lpar which uniquely controls the devices (ip, scsi, etc) and then shares them out to other lpars.

      Crossbow doesn't need the lpar layer and so doesn't inherit the costs and inefficiencies of doing this.

      You get some similar effects (particularly for traditional sysadmins) but the costs and hw implications are very different as are the limits on what you can do - much closer for vio server than crossbow.

      From my own experience: try running sybase backup server via the usual client server routine but with both sides running via vio - it's terribly inefficient and terribly failure prone - we had to close of essentially all other access while it ran. Crossbow has no such cross effects.
      murph_z
      • VIO/Crossbow/VMWare

        [i]is part of the LPAR system. It provides a dedicated (hidden) lpar which uniquely controls the devices (ip, scsi, etc) and then shares them out to other LPARs[/i]

        I'm with you so far. A VIO server is analogous to a device virtulization layer. It controls the physical devices and also presents virtual devices to other LPAR in the frame.

        [i]Crossbow doesn't need the lpar layer and so doesn't inherit the costs and inefficiencies of doing this.[/i]

        It still needs a 'layer' though. Software that uses CPU, Memory overhead. No difference if implemented on a SystemP or a Sun box. SAME concept. Your description of what Crossbow does is EXACTLY what a SEA does in the IBM VIO world. Its also nearly identical to VMWare's network implementation.

        [i]From my own experience: try running sybase backup server via the usual client server routine but with both sides running via vio - it's terribly inefficient and terribly failure prone - we had to close of essentially all other access while it ran. Crossbow has no such cross effects.[/i]

        Why would Crossbow be special? Its a basic concept, you are fanning several virtual adapters into a single physical. Of course you will have contention if one of those virtual adapters can saturate the entire physical adapter. And, as always it really comes down to architecture. My guess is your VIO solution in your example was not architected properly. If indeed it was, some things are just not good candidates for virtulization -- perhaps the IOP/Network requirements of that particular situation would necessitate physical hardware.

        I would prefer taking a snapshot of the Sybase database on the SAN and backing it up from a completely different dedicated server, but you don't believe in SANs either ;)
        civikminded
      • AIX already have this

        AIX has three ways for networking the LPARS.

        LPAR with dedicated NIC(top performance)
        LPAR with shared IO through an IO-server(flexibility)

        And IVE(Integrated Virtual Ethernet) which seems a lot like Crossbow. Virtualisation on the NIC, no need for IO-server, connects directly to the bus, offload TCP-processing. Available now.
        unix-admin
  • RE: Crossbow's golden arrows

    Borrowing your tendency to long quotes:

    [b][i]On its face Crossbow addresses a number of typical sysadmin issues with the use of network interfaces in zones or containers - particularly the problem that the lack of absolute isolation from the physical hardware meant that NIC access had to be co-ordinated outside the zone management function.

    That?s no longer an issue: you can now (or will soon be able to) consider network interfaces as integral to either containers or zones and move them around with the same cheerful joie de vive we?ve all previously been applying to files, users, and rights.

    To be honest my personal reaction to all this is along the lines of ?oh, Whoopidee do!? because the whole business of sticking multiple NIC cards in a machine and tying them to applications is a Wintel/x86 thing with no role in a well run SPARC/Solaris shop where normal Unix device sharing works perfectly well.[/i][/b]

    Taking all stated facts at face value and focusing on [b]only[/b] your logic, we see that...

    1. Until Crossbow, Solaris shared the same limitations that everyone else did. Otherwise, Crossbow would be neither innovative nor newsworthy.

    2. Other Unixes share those limitations.

    3. In the past it has been addressed in other ways.

    In what way then, are those limitations a "Wintel/86 thing"? If, as you indicate, everyone including Solaris has had the problem up to now and have had to address it in other ways, then it's obviously an "EVERYBODY thing". Or is Windows on AMD exempt? Lintel?

    I happen to like Unix as much as the next guy, but I'm going to have to agree with a growing number of respondents that it's getting to the point where your posts have long gone beyond simple lazy imprecision and devolved into tiresome hatespeech. Can you not post ANYTHING truly positive without a dig? I'd suggest that you're parodying Mike Cox, reporting on your Sun rep's advice, but that can't be... Mike is funny.

    Here's a challenge. Post something about Unix. [b]Only[/b] Unix. Use the words "Wintel" or "Microsoft" and you lose. It's a simple thing... should be easy.
    dave.leigh@...
    • Multiple NICs

      Using many NICS in one server started as an accomodation to the "fall through" packet processing model in Windows NT. In that environment switching streams carried big performance penalities, so people put in many cards.

      This problem has never existed in Unix - any Unix, whether for x86 or anything else. - vbut, of course, people who know how to do networking in NT, think they know how to do it elsewhere and so put many cards in Unix machines. It's dumb - but understandable as an artifact of adaptation to poor software design in NT.
      murph_z
      • Then what is the problem that Crossbow solves?

        [b][i]This problem has never existed in Unix - any Unix, whether for x86 or anything else.[/i][/b]

        [i]Some[/i] problem had to exist, or else Crossbow wouldn't be news. I think you've muddied the picture for people who don't regularly maintain Unix servers.

        Here you're implying that switching streams [i]doesn't[/i] and [i]never has[/i] carried a penalty for any Unix, and that using multiple NICs is just a preference. So terribly unclear why [i]"Crossbow offers, for x86 users, an important reason for choosing OpenSolaris over a BSD, [or] Linux... alternative."[/i] Important reasons are usually pretty clearly defined... they jump out at you. Why should this one jump out at me?

        I'm being obtuse, I know. But I've seen enough glazed eyes on managers to know that this doesn't provide them with a clear choice. The challenge that any tech advocate has in 'selling' a technology to management: how do you unglaze the checkbook holder's eyes and explain why he should buy this instead of new carpet?

        In this case, the typical PHB might respond with, "It sounds to me like you're asking me to invest in a bunch of infrastructure changes and training that are far more expensive than the cost of a couple of NICs. Money's tight. What do I get in return?"
        dave.leigh@...
        • What I think I know . . .

          On AIX machines, you need to assign buses (with cards) to each LPAR. If you have a single 4 port NIC card, you could only assign it to one LPAR. If you wanted to have more than 1 LPAR (of course!) you need to match each LPAR to a (different) bus that has a NIC card on it. So that nice "virtual" LPAR you have (usually) requires a NIC card, a SCSI card and a HBA card (or two).

          IBM also allows for a VIO - which is a LPAR that connects to NIC and SCSI cards - but then it presents a virtual interface to other LPARS - so that they don't have to have physical cards. Murph explained how cludgy this is and how Crossbow does it better (wasn't that the same name that Sun used for its mice?).

          With Crossbow (and VIO) you only need a single physical NIC - and all the VMs can use it.
          Roger Ramjet
        • Problems addressed

          As you probably noticed, I'm conflicted on this issue - however:

          1 - crossbow addresses (incorrect) buyer beliefs about the need for cards and virtualization - that's important to Sun Sales.

          2 - crossbow opens new markets for Sun - competing with cisco et al. That's the one that's likely, I think, to be important in the market. 6K for an AMD/Opensolaris appliance vs 32K for a cisco? kind of a no brainer.

          3 - crossbow lets you eliminate some vulnerabilities from secure networks. That's the one that's valuable to me.
          murph_z
          • Uhm, okay...

            I believe one addresses incorrect buyer beliefs with education, not by selling them stuff they incorrectly believe they need. If word ever got out that the perceived need for the software was based on fallacious assumptions it could be devastating for Sun sales. I doubt this is the case, though... there are probably some very [i]correct[/i] buyer beliefs about the need for cards and virtualization, or Sun wouldn't have invested the effort.

            Still, I'm not sure that this would divert someone into a Sun purchase unless they were headed that way to begin with. This looks like one of those things that if you need it, you already know it; and if you're struggling to imagine an application for it, you don't need it.

            --

            [i]?You do not really understand something unless you can explain it to your grandmother.?[b] ? Albert Einstein.[/b]

            'I guess I don't understand this yet.' [b]? Richard Feynman[/b]
            [/i]
            dave.leigh@...
          • Ah, if only that were true

            but it's not - and you'r emissing the points about routing applicances and security.
            murph_z
          • Really?

            I [i]do[/i] get the points about routing appliances and security, which is why I'm not asking any questions about them. What I'm interested in is the benefits of [i]this particular[/i] method of virtualizing the NIC on the server as opposed to all the others we've heard of. Let's face it, simply virtualizing the NIC is old hat... every LPAR, every virtual machine does it. But I suspect there's something that's truly exciting here makes this very different that shoots right over the head of anybody holding the purse strings.

            What I've been asking you to do is simply articulate what that might be. And I'm bending over backward to get you to do it. But if you don't wanna, fine. FWIW, here's the most interesting throwaway statement you've made on the entire topic:

            [b][i]That?s no longer an issue: you can now (or will soon be able to) consider network interfaces as integral to either containers or zones and move them around with the same cheerful joie de vive we?ve all previously been applying to files, users, and rights.[/i][/b]

            That, combined with what Erik said about multiple NICs providing backup, puts in my mind that besides simply sharing physical resources (which we already do), potential benefits of Crossbow include increased portability, DR, and redundancy for my apps. Or is that off-base?
            dave.leigh@...
        • Crossbow allows NIC Virtualization

          The problem crossbow solves is that of NIC virtualization and allowing QoS of NICs when provisioned to multiple virtual hosts within a physical host.

          For those who are familiar with VMWare, this is the equivalent of Virtual NICs
          Unix_Magic
      • Huh?

        I thought you put multiple NICs in one server so that if one gets fried you aren't dead in the water.
        Erik Engbrecht
        • That's another post hoc adaptation

          Once you take the interface off the board and adopt the nic approach, you inherit its failure rate.

          But, of course, you didn't need to do this in the first place...
          murph_z
  • Back to Sun marketing

    At least you didn't run down MS, but this is getting rather lame.

    This was a *nix column once wasn't it ?
    tonymcs@...
  • RE: Crossbow's golden arrows

    I kid you not, Crossbow is very unique and
    Paul got it right on! As for some of the
    questions raised, I put some pointers on
    my blog at http://blogs.sun.com/sunay/entry/crossbow_enables_an_open_networking
    When we started looking at this problem
    (way back in 2001), all the Unices (including Solaris before Crossbow) couldn't even keep a packet on one CPU or run a packet till completion or
    even know whose packet was being processed
    for most part. So it was not possible for
    them to even do anything close to doing virtualization or scaling the way we
    wanted to do i.e. perfectly. It was a lot
    of hard sweat and blood over a long period to ensure that we can figure out the packet owner even before we start processing the packet (a pretty hard problem on receive side). Anyway, instead of writing a dissertation here, check out
    some more details on Crossbow page at
    http://opensolaris.org/os/project/crossbow
    or at my blog (http://blogs.sun.com/sunay). And we really appreciate all opinion and time from you all. Thanks.
    sunay99