Have a sneer percolating in your system but not enough time/energy to make a whole post about it? Go forth and be mid - welcome to the Stubsack, your first port of call for learning fresh Awful you’ll near-instantly regret.

Any awful.systems sub may be subsneered in this subthread, techtakes or no.

If your sneer seems higher quality than you thought, feel free to cut’n’paste it into its own post — there’s no quota for posting and the bar really isn’t that high.

The post Xitter web has spawned so many “esoteric” right wing freaks, but there’s no appropriate sneer-space for them. I’m talking redscare-ish, reality challenged “culture critics” who write about everything but understand nothing. I’m talking about reply-guys who make the same 6 tweets about the same 3 subjects. They’re inescapable at this point, yet I don’t see them mocked (as much as they should be)

Like, there was one dude a while back who insisted that women couldn’t be surgeons because they didn’t believe in the moon or in stars? I think each and every one of these guys is uniquely fucked up and if I can’t escape them, I would love to sneer at them.

(Credit and/or blame to David Gerard for starting this.)

  • corbin@awful.systems
    link
    fedilink
    English
    arrow-up
    12
    ·
    14 hours ago

    An employee of cybersecurity darling Trail of Bits has published a record of their sheer incompetence in virtual-machine design and security analysis framed as chatbot critihype.

    opinions about QEMU and Linux

    They say:

    If it wasn’t clear before, I will state it plainly: you can no longer assume a mere VM will contain a sufficiently advanced AI agent. To use a 2010s term of art, you should treat such agents as an advanced persistent threat.

    My friend in Flying Spaghetti Monster, you did not actually secure the VM! They go on to explain how they did not secure the VM:

    For those curious, libslirp is a library that enables VMs to have networking, which you almost always want. I did not even know what libslirp was, or that the version I was running had both known and fixed-but-unmarked vulnerabilities.

    QEMU does not have bridged networking enabled by default, so the VM can’t transparently access the host or reach the Internet; it’s something that the user must explicitly request. I know this because I have had the experience of spending a weekend with QEMU networking. Moreover, libslirp corresponds to the -net user backend, the default, which is known to be slow, insecure, and missing features like IPv6. The standard approach for QEMU is to either wire up a TUN/TAP interface or to use passt. I suspect that the author uses some sort of convenience scripts that they didn’t write themselves. I’m not quite cynical enough to guess Vagrant, but it wouldn’t be the first shop I’ve heard of that couldn’t wean themselves off it.

    First it tried identifying what was accessible via the network on the host; it found a CUPS server (with a known CVE that had not made it to oldstable packages), but was not able to complete exploitation due to AppArmor. It then detected I run my host kernel with mitigations=off and attempted to use hardware bugs to get a read oracle of host memory (the primitive was too unreliable).

    Linux has hardware-bug mitigations enabled by default and the author disabled them for speed. It was secure by default and the author made it insecure.

    An off-the-shelf VM is not enough to contain a modern, cyber-capable AI agent. There is simply too much attack surface.

    They deliberately misconfigured the off-the-shelf tool to make it look bad. Why would somebody want to make Free Software look bad? Hmm…

    What can we do? A start is using a virtualization technology that was purposely built with a minimal attack surface and a focus on security, like Firecracker. I had the AI agent run against Firecracker. It was able to hardlock the machine due to more Linux kernel flaws (all patched in upstream), but could not successfully escape.

    You mean AWS Firecracker, the AWS tool developed by AWS? Was this whole thing an AWS ad? It feels like this article was like a combination of negging and sponsored content.

    • froztbyte@awful.systems
      link
      fedilink
      English
      arrow-up
      6
      ·
      edit-2
      13 hours ago

      I’d seen that link and didn’t even go looking, but wow this is way worse that I would’ve expected

      it’s been a while since I last looked (I usually just deploy whatever network I need) but iirc even the typical desktop config distributions of qemu-kvm are a macvtap or link+nat setup

      It then detected I run my host kernel with mitigations=off

      I wonder if this person even thought to run a nessus/openvas/whatever scan against the same machine

      also, quite seriously, what fucking clowning is this to have this be some “I slapped together a test on my desktop” rather than having a lab bench? is it too hard and costly for trail of bits employees to afford a 1u they can pxeboot for experiments? wild

      The standard approach for QEMU is to either wire up a TUN/TAP interface or to use passt.

      (e: adding)

      The standard approach for QEMU is to either wire up a TUN/TAP interface or to use passt.

      other typical modes are bridged (e.g. having various vlans available as bridges and then managing vm exposure to vlan by interface attachment), direct interface assignment (hw offload to the guest)

      passt is also commonly used for podman, and it’s moooostly userspace[0] IP and no real good l2 options

      1. I had an absolute rager[2] of a time figuring out how to make n-many podman containers[3] spin up into an assortment of vlans for realtime multi-vlan testing
      2. of the graaaargh kind
      3. don’t do containers, kids. they’re bad for your health

      (I’m only half joking. it’s stunning how badly container-management software and tooling all suck)

      • corbin@awful.systems
        link
        fedilink
        English
        arrow-up
        4
        ·
        12 hours ago

        Yeah, networking is such a hassle.

        lab bench?

        I think that this is really insightful, as it’s actually part of the magic trick. Like, normally I’d start by wondering about air-gapping, but there can’t be an air gap in the network because the entire trick relies on ChatGPT tokens flowing into the machine under test and into some system shell (in some VM, yadda yadda) and back out again, so of course this is going to be an inherently insecure setup.

        • froztbyte@awful.systems
          link
          fedilink
          English
          arrow-up
          3
          ·
          edit-2
          1 hour ago

          the networking part is fine (I do a looooot of networking), but the software quality and choices (esp with containers) is just extremely meh

          re the gapping, eh, there are ways. just off the top of my head: rig up the serial console, screen+camera and usb keyboard signalling (which you can data diode on transmission later via usbnet). and that’s before you go for esoteric side channels and utter silliness

          but “I turned off spectre and friends mitigation” is just some extremely funny shit

          • froztbyte@awful.systems
            link
            fedilink
            English
            arrow-up
            2
            ·
            9 hours ago

            idly, I do wonder how well an ocr-only clodjeepeetea would work. every time I look in on all these “agent session” tools they appear to be a very mech turk thing of a phenomenal bit of hidden state, and a tiny little bit of visible distraction. feels like ocr-only might break a bunch of the usual tricks that are used to build on for the shiny demo