Show HN: Redbean – Single-file distributable web server

Cute idea. Sounds like magic.

• edoceo · 2049 days ago

  Built with actual magic: https://storage.googleapis.com/justine/ape.html From the same person I think

  • indigodaddy · 2049 days ago

    Author mentions that the APE’s can be booted from the BIOS. Would this imply you could perhaps implement a lambda-like cloud service that could spawn/start the APE on demand? Edit, I’d guess you’d want to/have to, also have a tcp/ip stack in the APE as well (but perhaps just going for containerization of the APE is the better play and not have to worry about any of that..)

No relation to http://redbeanphp.com/ I guess :D

• rebelpixel · 2049 days ago

  I think that's still worth mentioning considering they're roughly in the same sphere of interest, to avoid the confusion.

Zero/near-zero dependencies, minimal configuration, portable, and trivial to run. Awesome. Even if Redbean here doesn't end up taking over the world, I hope that more ecosystems and toolchains sit up and take notice. We need more tools that look like this.

I really like your view of the world, that programs should be portable, tiny, and just work. The slamming of so much functionality into a zip file is inspiring. Would it be possible to do something similar with Free Pascal, allowing the recreation of something like Turbo Pascal, except really, really portable .com output?

• jart · 2049 days ago

  Author here. Absolutely. I used to love Turbo Pascal and Delphi when I was younger. If Free Pascal uses GNU LD.BFD or LLVM LLD when it links programs, then all you'd need to do is is configure it to use cosmopolitan.a when linking system call functions like read(), write(), etc. See https://github.com/jart/cosmopolitan Another option is if Free Pascal wants to write all the system call support from scratch, then doing that now is going to be a whole lot easier since the Cosmopolitan codebase does a really good job documenting all the magic numbers you'll need. See files like https://github.com/jart/cosmopolitan/blob/master/libc/sysv/s... and https://github.com/jart/cosmopolitan/blob/master/libc/sysv/c... I've been working on a tiny C11 compiler called chibicc which has most GNU extensions and I managed to get it working as an actually portable executable with an integrated assembler: https://github.com/jart/cosmopolitan/blob/master/third_party... I also got Antirez's KILO text editor working as an APE binary. https://github.com/jart/cosmopolitan/blob/master/examples/ki... If we can build a linker too then we can get a freestanding single file toolchain + ide that's able to be a modern version of Turbo C.

  • akavel · 2049 days ago

    I think the Zig community/BDFL might appreciate adding an APE target "os" to their buildchain (LLVM-based); that could be a match made in heaven!

    • jart · 2049 days ago

      Author here. I've been chatting with the Zig BDFL on Twitter recently about contributing APE support and he's been super supportive so far. It's going to be a nontrivial undertaking but would certainly be rewarding for everyone if we can make it happen.

      • akavel · 2048 days ago

        Oh wow!!!!!!!

      • InitEnabler · 2048 days ago

        Very cool! Maybe I should give Zig a try too. Also hats off to you @jart! Is this a one time thing or are you thinking of something bigger going with this project?

      • dom96 · 2047 days ago

        If you're interested, I'd expect adding support for this to the Nim programming language would be more trivial since Nim compiles to C. I had a quick look into doing this on your original thread[1], but got stuck pretty early on and just didn't have the time to get back into it. If that's something you'd be willing to pursue I'd love to help any way I can. Feel free to DM me on Twitter[2] any time. 1 - https://news.ycombinator.com/item?id=25558643 2 - https://twitter.com/d0m96

  • ronyeh · 2049 days ago

    Since I can't upvote your Show HN twice, I'll upvote this comment. I learned something interesting tonight from your websites. Thanks! chibicc is a really fun idea! If I was in college, I would take a compiler course from you. Taken from your README: "chibicc is developed as the reference implementation for a book I'm currently writing about the C compiler and the low-level programming. The book covers the vast topic with an incremental approach; in the first chapter, readers will implement a "compiler" that accepts just a single number as a "language", which will then gain one feature at a time in each section of the book until the language that the compiler accepts matches what the C11 spec specifies. I took this incremental approach from the paper by Abdulaziz Ghuloum."

    • xearl · 2048 days ago

      chibicc is actually written by Rui Ueyama. Github: https://github.com/rui314/chibicc Previous HN: https://news.ycombinator.com/item?id=24676851

      • ronyeh · 2047 days ago

        I see. My mistake. So jart is trying to turn it into a APE? I learn something new with every comment. So I would edit my comment to say: I'd love to take a compiler course from Rui, and a systems course from jart.

    • ronyeh · 2047 days ago

      Edit: as pointed out by xearl, I accidentally copied the README from github user https://github.com/rui314 and attributed it incorrectly to jart. That's what I get for browsing HN on my phone late at night.

  • mhd · 2049 days ago

    Speaking of other languages, given that the APE is a zip container, it shouldn't be too hard to have a "starkit" like setup where the binary part is basically node/python/perl/tcl and the main script and all required libraries are in the executable.

  • whizzter · 2049 days ago

    I love what you've produced here. I noticed that pthreads seems to be fail-stubbed for the time being, i guess it's a todo with tricky semantics? As for holding off on UI for the time being is probably the right thing to do, keeping with the minimal style planning on going for raw-x11,etc. However, as much as I personally love plain framebuffers (late 90s democoder myself), it'd be totally non-accessible for "plain" applications and totally lacking performance compared to anything accelerated when it comes to rendering. Graphics cards are just so many times faster that it's not really even the same kind of usage (only sad part is Apple punting on OpenGL driver updates even if Intel on the low end has become so much better). That said, having looked for a minimal UI framework and sadly they seem to be far inbetween these days (libUI is fairly compact on win/osX but required GTK on *nix but they've lost a bit of steam since 2018) Best of luck on it all (hopefully I can snatch out some time to contribute)

    • jart · 2049 days ago

      So far the only platform where I've managed to get threads to work has been Windows. I came pretty close to getting clone() on Linux working a few days ago. Threads are so hard because no API was ever defined for them in original UNIX and BSD, so therefore each modern system today does them in a completely different way. Another blocker is that not all operating systems let us modify segment registers which is something that the Linux compiler assumes is available for TLS. So it'd be a miracle if it ended up being possible to make threads work portably. fork() on the other hand was easy to get working on all platforms. I love fork() and vfork(). As for GUIs I'm still open to merging raw x11 support. One tricky issue is that we'd need to define a new API that veneers framebuffers, and so far Cosmopolitan has mostly refrained from defining novel interfaces, since the focus has been twiddling with low level magic numbers in order to make the textbook interfaces we already have work everywhere!

      • whizzter · 2049 days ago

        Ah segregs is gonna be a pita indeed if it's all hard-wired to the compiler. Considered making some way to add "portable" modules yet(generic APE code with plat-specific impls) to avoid adding stuff to the core? I know the default calling-conv is slightly different but maybe by smth like defining virtual ports? (ie "send" structs since alignment rules should be mostly the same?)

      • fwsgonzo · 2048 days ago

        Threads are a PITA everywhere. On Linux the SYSCALL stuff complicates things because of all the register moves, the FUTEX stuff, and potential races with how threads are exited.

  • anthk · 2048 days ago

    Now you can port Slashem to more platforms than Nethack 3.4.3 itself :D

  • dicroce · 2047 days ago

    Hey, question... Is an LD_PRELOAD version of cosmopolitan possible?

Caddy is a single file web server as well :) https://caddyserver.com/ Admittedly they solve different usecases, where Redbean seems to want to serve a whole static site as a single file, which isn't currently possible with Caddy, (but you can run `caddy file-server` to serve the current directory as a static site), but it may become possible in the future with Golang's new embed package https://golang.org/pkg/embed/

• mholt · 2049 days ago

  That's actually a pretty interesting idea. I wonder if we could make a lite version of Caddy bundled with your website embedded...no need to mess with file systems or permissions or devices and folders to run a static site. Might need some massaging to become a truly pleasant workflow though.

  • francislavoie · 2049 days ago

    Yeah - I could see it being part of xcaddy tooling to package up the static assets in the binary :)

  • IceWreck · 2049 days ago

    The go std library offers the embed package. You can just embed your files and use the std lib's webserver and generate a single file binary. No need for Caddy. All this would be roughly 15 lines of code.

    • mholt · 2048 days ago

      People use Caddy for automatic HTTPS, which the std lib does not give you.

      • anderspitman · 2048 days ago

        Obviously you don't need to know this but others might not be aware that certmagic is a great middle ground between Caddy and rolling your own ACME.

• kapitalx · 2049 days ago

  Caddy is cool! Redbean is magic :p Thanks for sharing.

• jart · 2049 days ago

  Caddy is cool but it has a separate 33mb executable for each operating system. Redbean is a 128kb executable that runs on all platforms so it has fewer moving parts and is 1584x tinier. All the convenience of being able to "just add your assets to the zip executable" wouldn't have been feasible if we needed to repeat that process n times for each operating system.

  • francislavoie · 2049 days ago

    Yes of course - like I said, they solve different usecases. Caddy has way, way more features overall to make up for the bigger binary.

• goldenkey · 2048 days ago

  Is that smile at the end of your first sentence meant to be condescending? You are affiliated with Caddy and ought to mention that when you promote your own software. https://news.ycombinator.com/item?id=16687686 This is not a competition. Caddy is not even in the same realm as Redbean. Redbean is a multi-platform binary that doubles as a ZIP. It's roughly 1500 times smaller than Caddy too. > different usecases Right. :)

  • francislavoie · 2048 days ago

    > Is that smile at the end of your first sentence meant to be condescending? What? Of course not! You're reading way too much into that. It's simply enthusiasm. I read about Redbean and it being a single file webserver and the first thing to come to mind is Caddy, as "prior art". Like I said, different usecases; Caddy is a general purpose webserver which has automatic certificate management, H2/H3 support, reverse proxy, file server, a strong plugin system, the list goes on. Of course it'll be bigger in size. Redbean is super cool and does have the advantage in file size and portability, clearly, but it is essentially a single-purpose tool. And there's nothing wrong with that.

The short description is a bit too short; this is mind blowing! Redbean is a portable, single file executable webserver which also acts as a zip-file for the static content it hosts, and runs on Linux + Mac + Windows + FreeBSD + OpenBSD + NetBSD + BIOS without any recompilation... You can manage the static content using standard pkzip tooling! Now that's what I call thinking out of the box! Edited: some rewording and breathing space

• tutfbhuf · 2049 days ago

  Is it that this binary does some tricks to run on Linux, Mac, Windows, FreeBSD, OpenBSD, NetBSD, BIOS or does it really work on all operating systems including something like Plan 9 or even TempleOS? And does it run on different architectures like ARMv6?

  • mhd · 2049 days ago

    I think it qemu's itself if run on another platform.

  • twic · 2048 days ago

    Native execution on x86_64 only. "Linux + Mac + Windows + FreeBSD + OpenBSD + NetBSD + BIOS", where "BIOS" means that it will boot a machine and run on bare metal. On non x86_64 systems, the driver script will try to run the binary using QEMU (which has to be installed and on the path): https://github.com/jart/cosmopolitan/blob/40291c9db386d8a952... If i have understood how the portability works, it would be pretty straightforward to add support for another OS which uses the same ABI as the unices and has a compatible system call table - just a matter of adding a column to consts.sh (system call numbers in the nr section, constants from system headers in their appropriate sections) and adding any OS-specific system call rituals to systemfive.S. Note that all system calls will go through the syscall interface. If your OS has some other way to make system calls, like Linux's vDSO or io_uring, that won't be used. If your OS had a different system call table, like Windows, then you would need to write more bespoke code for it, like all the Windows wrappers: https://github.com/jart/cosmopolitan/tree/0e85b136ae1beb0d2b... You would then need to go through the Cosmopolitan libc and add implementations of its functions on top of your wrappers.

.apk files and .ipa files are secretly zip files too. With the addition of a few manifest files and other crufty things embedded in the zip, I bet you could add Android and iOS to the list of supported platforms too.

• akavel · 2049 days ago

  I've recently succeeded in building working .apk files byte-by-byte completely from scratch so I could maybe help some. Open an issue on https://github.com/akavel/hellomello if interested in some talking/advice and/or watch my talk on NimConf 2020 for an overview (I don't have a link handy as I'm on mobile).

• nonameiguess · 2048 days ago

  Jars, rpms, whls, Microsoft Office, probably a bunch of other stuff I don’t even know about are actually zip files.

  • citruscomputing · 2048 days ago

    .epubs too!

I just spent 30 minutes reading about this. I'm so shocked that I'm logging in to comment after 5 years of lurking. Justine has built a c library that allows you compile a binary once and have it run it on any os or baremetal. The SAME binary. Quite frankly, that sentence doesn't even make sense to me. Check out https://storage.googleapis.com/justine/cosmopolitan/index.ht... As far as I'm concerned, this is literal magic. Look at the magic numbers: https://github.com/jart/cosmopolitan/blob/37a4c70c3634862d8d... I could go on, but there's no binary portability comparison with any other language. And she has made some pretty neat optimizations. Back in the day, I saw some pretty neat stuff with the ELF format, but this takes the cake. Wow. Edit: I'm editing because this is just so bloody absurd. https://storage.googleapis.com/justine/printimage.html $ ./printimage.com someimage.jpg Like wow. And also video. https://storage.googleapis.com/justine/printvideo.html I'm struggling to put my shock into words. I've been around. There's engineering. There's academia. But this falls into straight-up wizardry.

• malux85 · 2049 days ago

  I agree, I got to the page on actually portable executable, and had to read it a few times to make sure I was grokking it properly. This is just sheer cleverness, THIS is the stuff that should be on hacker news!

  • hprx · 2049 days ago

    For real. It's hard to grok cause there are no words to describe it. When people say that something is cross-platform, they usually mean a) that the software will build on multiple platforms b) there is some sort of vm which runs the executable (jvm, beam, wasm) This is the SAME binary. Running everywhere. Could be super-useful as an archival format for mission critical code. sqlite comes to mind. Maybe it's late at night, but I'm struggling to find a suitable word for the project that is better than portable or cross-platform. Literally, I can't find a better word than the project name - " actually portable executable." @author - if you're reading this please set up something for a few bucks a month on your github sponsors. I don't think I have any use for the library, but this is so outrageous it deserves something more than imaginary internet points.

    • jart · 2049 days ago

      Author here. Thanks! I've added a new sponsor tier per your request. https://github.com/sponsors/jart

      • ActorNightly · 2049 days ago

        Do you have a direct paypall? Im not a fan of subscription services but Id like to support this.

        • jart · 2049 days ago

          I do indeed have PayPal: jtunney@gmail.com Thank you for your support!

  • akalsz · 2049 days ago

    > THIS is the stuff that should be on hacker news! If interested, see previous discussions of APE: https://news.ycombinator.com/item?id=24256883 https://news.ycombinator.com/item?id=25556286

    • alisonatwork · 2049 days ago

      Thanks for the links. The discussion on the first one is interesting, it wasn't clear to me from the original doc that the UNIX version needs to overwrite itself to get to the ELF header. That's a bit unfortunate, because it means you can't share the program again after you ran it the first time on BSD or Linux. Still very clever, though. I imagine there could be a "repackage for distribution" switch added to put the MZ header back again.

      • jart · 2049 days ago

        I did the research and what we've accomplished here, while imperfect, is the best of all worlds solution. I like to think of it as an installer that takes a few microseconds, because it only needs to change 64 bytes. I've been considering adding a CLI flag where you can say ./foo.com --restore and it'll put the original 64 byte header back in place. Perhaps one day we can change the Linux, BSD, and XNU kernels so they can recognize the APE executable format. But until that happens we've got a great shell script hack that's now required by POSIX which is exceedingly fast and works on pretty much all systems stretching back decades.

        • geocar · 2047 days ago

          Why didn't you copy to a tmpfs and overwrite that version instead? Seems more likely to have installed programs on read-only media and a little tmpfs, instead of a writable binary and no tmpfs. Do you have any information on some of the other strategies you researched?

• ghoomketu · 2049 days ago

  I really don't understand internals ibut I've often downloaded cygwin exe file for commands like grep, tail, etc.. Are you telling me that is no longer required?

  • hprx · 2049 days ago

    On linux, I downloaded the printvideo binary and played a friggin crab video on my terminal after just a chmod. I don't have any other OS to confirm, but it sure looks like it if someone compiles/links coreutils against this.

• skrebbel · 2049 days ago

  Everything she does has this level of jaw-dropping amazingness. Between her and Fabrice Bellard i don't know many people who consistently get my chin to hit the table. I hope that Cosmopolitan becomes mainstream, so much more software could have that It Just Works quality.

  • sneak · 2049 days ago

    "Jaw-dropping" is definitely a term I'd use to describe some of her more infamous tweets and blog posts, too.

    • dang · 2048 days ago

      Please don't. Resisting extrinsic provocation is key to thread quality.

      • sneak · 2048 days ago

        Understood (with my apologies, I mistakenly assumed it was on topic). FWIW, I like and agree with the things I was referencing, I didn't mean it in any way as a jab or provocation. I shall endeavor to do a better job staying on the topic of the article in question.

        • waheoo · 2048 days ago

          It was on topic, your mistake was to write a comment that comes off snarky and political of you fail to read it with positive intent.

  • ksec · 2048 days ago

    Mike Pall for LuaJIT

• alisonatwork · 2049 days ago

  Agreed, this is the best programming-related thing I've seen on the internet in a long time. It reminds me of when I was a teenager, excited to become a computer programmer. Then I grew up and joined the real world workforce and it was all far less exciting than I imagined. The coolest thing about this Actually Portable Executable is that once there's a compiler and linker built with it, I can play around with writing C on Windows without having to faff about with WSL or MinGW or learning what makes MSVC different from the C I learned in university and coded in my first job before I became a Java/JavaScript/Python/etc programmer. When I discovered Go I thought I had discovered the better C that just worked everywhere, but the idea of having plain old C that also just works everywhere is very appealing.

  • Abishek_Muthian · 2048 days ago

    I like Go for the same reason, but I have an irrational fear that someday I might be in a life or death situation and should program my way out using a bare bones computer without Internet; So I keep tabs on C occasionally. αcτµαlly pδrταblε εxεcµταblε has now piqued my interest, So I might actually indulge in C programming for fun again after all. Thanks Justine.

  • yw3410 · 2047 days ago

    I think it's really cool project; but could you explain why it's important that a binary works everywhere for you? I don't really get the enthusiasm for this sort of thing. Maybe I have a blind spot or something or I've just been in Unix-land for too long.

    • alisonatwork · 2046 days ago

      I think the UNIX thing is the blind spot. People who run Linux or BSD don't seem so bothered to recompile for their own platform, safe in the knowledge that it will almost certainly just work. But on Windows, trying to get open source software written in C to compile has been a pain for decades. There have been some attempts to make it work (notably Cygwin, and now MSYS2), but it can still be challenging, especially with programs that were never designed to be compiled for non-UNIX (or non-Linux!) targets. Having a C library that is designed from the ground up to target multiple operating systems, and an executable format that means the first compile is the only compile, that takes away a lot of toil that programmers normally have when trying to build their own abstractions around each platform's C library. I guess from UNIX land you could see the benefit as removing the need for autoconf or platform ifdefs.

• lenkite · 2049 days ago

  Will this work if your code uses the C++ standard library ? Or is this just for pure C ?

  • ZiiS · 2049 days ago

    The are a few options but all require some effort https://github.com/jart/cosmopolitan/issues/27

• Proven · 2049 days ago

  That malware for OS X that does nothing (so far) from the news earlier this week also has one multi-arch binary for all modern Apple notebook architectures (AMD64 and ARM64)

  • stqism · 2048 days ago

    In MacOS this is really just a feature of using Mach-O’s multi architecture abilities, they call this universal binaries.

• TeMPOraL · 2048 days ago

  Agreed. This single sentence at the end of αcτµαlly pδrταblε εxεcµταblε page: > I believe the best chance we have of doing that [writing software that stands the test of time with minimal toil], is by gluing together the binary interfaces that've already achieved a decades-long consensus, and ignoring the APIs. ...it's a kind of thought I don't think I could ever come up with. Mind blown.

  • JesseMeyer · 2048 days ago

    You go to war with the army you have.

• lambda_obrien · 2048 days ago

  Strong agree; someone needs to give her like a million dollars a year salary and unlimited resources to explore whatever she wants for life, just to see what she comes up with.

  • mobilemidget · 2048 days ago

    one millliiiooonnn dollars.... Hey Dr Evil (Austin powers reference), I think one billion is okay this year and age :) but yea I agree

• matheusmoreira · 2048 days ago

  Wow, that cosmopolitan C library is absolutely awesome. The ABI specialization is extremely interesting. Never seen anything like that before. I thought compilers did that. I don't understand why they couldn't properly optimize memcpy. I wonder how it handles system calls. Let me check the source... https://github.com/jart/cosmopolitan/blob/master/tool/build/... https://github.com/jart/cosmopolitan/blob/master/tool/build/... > void OpSyscall(struct Machine *, uint32_t); Pretty interesting... Need some time to figure out how it works. The system call entry points: https://github.com/jart/cosmopolitan/blob/master/libc/sysv/s... https://github.com/jart/cosmopolitan/blob/master/libc/sysv/s... It uses a jump slot technique which is interesting: https://github.com/jart/cosmopolitan/blob/master/libc/sysv/m... System call numbers: https://github.com/jart/cosmopolitan/blob/master/libc/sysv/c... https://github.com/jart/cosmopolitan/blob/master/libc/sysv/s... Looks like it packs the system call numbers of all operating systems into a single constant. For some reason there are some dedicated implementations of Linux system calls using inline assembly rather than the entry point: https://github.com/jart/cosmopolitan/tree/master/libc/linux More related files: https://github.com/jart/cosmopolitan/blob/master/libc/calls/... https://github.com/jart/cosmopolitan/blob/master/libc/calls/... Also funny how it calls Linux "GNU/Systemd"!

  • Mister_X · 2046 days ago

    "...it calls Linux "GNU/Systemd"" Yeah, it would have taken less bits to use GNU/POS instead, because it ain't Linux anymore. Poettering Operating System.

• user3939382 · 2048 days ago

  It's funny because the high-level introductions to C I would read as a kid before I really knew much about computing would almost always start out by explaining that one of the biggest strengths of C was its portability, and how it allowed you to write the same code that worked on every platform. Of course I came to learn that's true only in a sense that has no connection to practical reality, this looks like an exception.

• niDistinct · 2048 days ago

  Impressive.

• rustypython · 2047 days ago

  Could you please explain to someone from the web domain with very little experience with compiled C programs why this is significantly better than, say, distributing a python program? Python runs on all the platforms mentioned. Or alternatively requiring compilation from source?

  • tvprod1203 · 2045 days ago

    The python script/program ultimately depends on a Python interpreter which currently has to be distributed as a different binary for every platform. So in the end, the difference is purely where the platform-differentiated compilation happens for VM & interpreted languages like Java & Python. This would instead allow you to ship a single Python executable that simply works everywhere. Whether with a bundled-in script/program, or just the interpreter.

• yumraj · 2047 days ago

  While this is absolutely amazing, and I'm still trying to see if I'm even able to grok the how.. But, the paranoid in me is now curious/afraid if this approach can/will lead to multi-OS malware?

  • tvprod1203 · 2045 days ago

    That sort of malware already exists. This does simplify making it by a lot, because now it can much more easily ensure its portability without external communication.

Presumably X86_64 only?

• ZiiS · 2049 days ago

  Not sure the is a working example. But they talk of embeding qemu to create cross archetecture executables. But the Window's and actual compiled code would still be x86-64.

• sscarduzio · 2049 days ago

  She says somewhere in her website that arm/apple silicon support is really easy to add, with no significant size or performance penalty, the same way she added qemu support.

OK I see we are going crazy, me too. Thanks @jart. I have been thinking about portable web apps with embedded SQLite for some time. I do not have the technical chops at C level to pull this off. I am really inspired by this project. I hope Redbean leads a way to distribute self-hosted apps in today's era of the cloud.

• BugWatch · 2048 days ago

  I still use the classic version of Tiddlywiki that is a single self-contained self-editing html file, and I "app-ify" it on Windows by changing the extension to *.hta. But having the similar solution with the SQLite, would be a perfect combination.

The APE shit is really cool. The webserver is novel but I am just wondering, how would it compare to something like Apache/ISS in terms on security/config/etc, or is this mostly just to move small sites around?

Is it a "fat" binary?

• mekkkkkk · 2048 days ago

  AFAIK a fat binary is just a binary with all dependencies included. It would still be compiled for a specific platform. The magic part about this is that it's a single file compiled once that can be run on most platforms. And it also adheres to the format of a zip file, so you can add, modify and remove assets as you please inside the actual file, post compilation.

  • orhmeh09 · 2048 days ago

    No, fat binaries are not binaries with all dependencies included. Fat binaries are executable files that contain equivalent compiled code for multiple architectures. Some also facilitate running binaries across different operating systems. More: https://en.wikipedia.org/wiki/Fat_binary The idea also reminds me of (executable) self-extracting ZIP archives that were common once upon a time.

    • mekkkkkk · 2047 days ago

      Thanks! Seems like I've misunderstood the concept a bit. My apologies.

I am floored. Not only is this just breathtaking sorcery, I also needed this exact thing for a project. I don’t understand probably 70% of the description of this software, but I understand what it does, and I’m more excited to carefully study and use this than I have been about anything in years.

Everybody is hyped up here but can someone explain to me what are the implications for the future of programming? Will I be able to write apps in a single language and have then run truly native on any OS or what else will we be able to do?

• alisonatwork · 2049 days ago

  This is something like a "fat binary" with some magic in the header that allows it to run out of the box on Windows and several different UNIXes, a bit like those sneaky shell scripts that run on several platforms[0]. It still needs to include some native code for each of the platforms inside the compiled binary, so it's perhaps better to see it as a slick packaging/distribution solution for C programmers than a total game changer for developers everywhere. The more interesting thing I think is the mapping of C library functionality between Linux, BSD and Windows. There have been other attempts to do this, but they never seem to get much traction. I think it's because C programmers tend to get wedded to their OS of choice and then invest time in improving that platform's C libraries. The problem with trying to remain portable with everything is that you tend to be stuck working with the lowest common denominator functionality, which can be frustrating when doing OS-specific stuff would open up so many more options. All that said, the cleverness of this project is inspirational in a way that might encourage other programmers to also focus more on building against simple functionality that is truly portable. It shows that you don't need all the bells and whistles to build cool stuff. That feels aligned with early UNIX philosophy and (more broadly) the hacker ethic, which is why it's neat to see, even if never gets adopted as a mainstream thing. [0] https://github.com/llamasoft/polyshell

  • r0b05 · 2049 days ago

    Appreciate the response. So it's a bigger win for C devs specifically. I am a C# dev so I am trying to understand what opportunities this project unlocks.

    • goldenkey · 2048 days ago

      Why you have pigeonholed yourself? "X Dev" is so cliche. You can compile C# to x86 machine code - this project has less to do with C and more to do with PE format and syscall tables. It means that it's language agnostic but not standard library agnostic.

      • gjvc · 2044 days ago

        Completely agree. The program is written in C# -- you aren't! :-)

Nice job, Justine. This level of excitement reminds me of the early days of Linux. I think it has the potential to go far.

• jart · 2049 days ago

  Author here. Thanks! I was deeply inspired when I read the source code to early versions of Linux and UNIX. The dream of that simple elegant power is something I'm hoping to restore for a new generation. All I'm doing to achieve that is simply standing on the shoulders of the giants who came before me.

  • fwsgonzo · 2048 days ago

    Have you considered trying to make system calls inline to avoid the call overhead? Or already thought about it? I've been banging my head against the wall of trying to make a micro KVM guest and the glibc startup code uses inline SYSCALL everywhere, I assume to make things faster. It does seem to call CPUID a lot too, but I guess it's not as expensive as having to make full-blown system calls.

    • jart · 2048 days ago

      Cosmopolitan defines linkable symbols for all the __NR_syscall constants so you can absolutely use inlining if you like to live dangerously. For example: https://github.com/jart/cosmopolitan/blob/91f4167a45f811fde6... There really isn't any performance advantage to doing this, because the SYSCALL instruction itself has something like a 2000+ cycle cost due to all the copying that needs to happen on the kernel side along with things like spectre mitigations which have made is slower. So the 8 cycle cost of having a normal read() function wrapper is the wrong thing to be focusing on. Cosmopolitan aims to address the performance cost of SYSCALL by making it possible to run your binary on bare metal, where your program becomes a kernel, and therefore needn't pay any context switching costs at all.

      • fwsgonzo · 2048 days ago

        And it's even more overhead when exiting a VM, then?

  • AtlasBarfed · 2048 days ago

    I can't believe I'm ASKING for a blog post. Can you blog a bit of the career/cognitive/skill evolutionary history behind this? Did the cosmo stuff come out of this or something else? Learned this in X level school, found out about Y, read about Y, learned this, got this job doing this, learned this from Z, got experience of AA... etc. Of course, assuming this exists, and you exist. And blogs "exist".

    • jart · 2048 days ago

      Author here. When I first discovered the idea for Actually Portable Executable I thought I was going mad like Terry Davis. I couldn't even believe that it was possible until I actually did it. It cost me my job but I couldn't be happier with the result. This project is going to make problems disappear for so many developers. It's the sort of tool people can get excited about. I'm so happy that my prior career success and top-notch training have granted me the privilege of bringing something this cool to the world. Enjoy!

      • cxr · 2048 days ago

        In all seriousness: whether it's exciting or not, this is something that should be getting federal funding. It really shouldn't be difficult for large, government-backed organizations to justify their interest. I don't have a lead for you (otherwise, I'd have invoked it by now to work on my own project[1], which has something in common through the use of polyglot files to achieve a similar effect[2], while mostly otherwise taking a different approach), but since you have a high profile, casting a wide net to attract the attention of someone able to secure the funding is something to consider. 1. < https://wiki.triplescripts.org/wiki/The_why_and_how_of_tripl... > 2. E.g., < https://www.youtube.com/watch?v=TUpd70Mu0Ek >

Am I the only one having issues trying to get this working? The webserver starts just fine, but once I add the index.html with zip as in the example, it stops working. This is on Mac 10.15, CentOS 8 and Ubuntu 18.04 LTS. Centos: [centos@test ~]$ ./redbean.com -vv error: Uncaught SIGSEGV on test.novalocal ./redbean.com EINVAL/err=22/errno:2/GetLastError:0 Linux test.novalocal 4.18.0-240.10.1.el8_3.x86_64 #1 SMP Mon Jan 18 17:05:51 UTC 2021 On the mac: ~ my-Maccie$./redbean.com -vv Killed: 9 On Ubuntu: ubuntu@localhost:~$ ./redbean.com -vv error: Uncaught SIGSEGV on localhost ./redbean.com EINVAL/err=22/errno:2/GetLastError:0 Linux localhost 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021

• TeddyDD · 2049 days ago

  No, I have the same issue on Linux Mint 20.1

• r0b05 · 2049 days ago

  After adding a new file with WinRAR, I get: EINVAL/err=87/errno:2/GetLastError:203 The parameter is incorrect. Probably the ZIP needs to be saved in a certain format.

  • jart · 2049 days ago

    Author here. The zip shouldn't need to be saved in any special format. If WinRAR is posing problems then please file an issue because I'd like to fix that. That error doesn't seem like something that would indicate executable corruption. I want to learn more.

    • r0b05 · 2049 days ago

      Sure thing, will provide as much info as possible.

    • user-the-name · 2049 days ago

      Have to say though, using zip so you can reuse the compression is quite a masterstroke.

    • adontz · 2049 days ago

      IDK if this is enough https://pastebin.com/8RyPWQFf

    • xPaw · 2049 days ago

      I tried to add a file using Windows file explorer, but it just complains "The Compressed (zipped) Folder is invalid or corrupted." This is on Windows 10 19042 build. 7-Zip doesn't even see the folders for that matter.

• jart · 2049 days ago

  Author here. It works fine on RHEL5 / CentOs5 for me. I can also confirm RHEL7. Please file an issue. What will help in particular is if you can give me the faulting RIP address. That should be in the crash report or in your dmesg log.

• tromp · 2049 days ago

  Same here on MacOS Catalina 10.15.7. The original invocation runs fine $ bash -c './redbean.com -vv' W2021-02-26T01:50:12.665277:tool/net/redbean.c:1144:redbean:3302] setsockopt(server, IPPROTO_TCP, TCP_FASTOPEN, &yes, sizeof(yes)) → EINVAL/err=22/errno:22/GetLastError:0 W--------------------000015:tool/net/redbean.c:1145:redbean:3302] setsockopt(server, IPPROTO_TCP, TCP_QUICKACK, &yes, sizeof(yes)) → ENOPROTOOPT/err=42/errno:42/GetLastError:0 V--------------------000022:tool/net/redbean.c:1166:redbean:3302] 0.0.0.0:8080 listen Visiting http://127.0.0.1:8080/ in my web browser shows a nice redbean page with lobster graphic at http://127.0.0.1:8080/tool/net/redbean.html but then following usage instructions in another shell: $ echo '<b>hello</b>' >index.html $ zip redbean.com index.html adding: index.html (stored 0%) $ ./redbean.com -vv Killed: 9 I blame my zip $ zip -v Copyright (c) 1990-2008 Info-ZIP - Type 'zip "-L"' for software license. This is Zip 3.0 (July 5th 2008), by Info-ZIP. Currently maintained by E. Gordon. Addition of index.html shrinks(!) redbean.com from size 204800 to size 204348, with changes starting from char 172938. That clearly corrupts the portable executable format.

• oneeyedpigeon · 2049 days ago

  Yup, I get "Killed: 9" on macOS (Big Sur) too. I actually had some kind of problem (I think) with the original invocation: $ bash -c './redbean.com -vv' W2021-02-26T02:06:29.162953:tool/net/redbean.c:1144:redbean:83621] setsockopt(server, IPPROTO_TCP, TCP_FASTOPEN, &yes, sizeof(yes)) → EINVAL/err=22/errno:22/GetLastError:0 W--------------------000026:tool/net/redbean.c:1145:redbean:83621] setsockopt(server, IPPROTO_TCP, TCP_QUICKACK, &yes, sizeof(yes)) → ENOPROTOOPT/err=42/errno:42/GetLastError:0 error:./redbean.com: check failed: 0xffffffffffffffff != 0xffffffffffffffff (48) 6fffffffff70 0000004034cf UNKNOWN 6fffffffff80 000000403225 UNKNOWN 6fffffffffc0 0000004027f3 UNKNOWN 6fffffffffe0 000000402a21 UNKNOWN 7ffeefbff870 00000040227b UNKNOWN

• zimpenfish · 2049 days ago

  IIRC, from vague memories of things flying past, Big Sur won't let a modified binary run once it's been checked by Gatekeeper. Which means the first run gets checked and notarised but when you add the `index.html`, the on-disk binary has changed and Gatekeeper won't allow it to run - I guess it's to prevent malicious code modifications, etc.

  • jart · 2049 days ago

    Oh wow that's good to know. Maybe you have to do something like `cat foo.com >foo2.com` after you've modified it with the zip tool.

  • safeaim · 2048 days ago

    Yeah, but this isn't Big Sur, it's Catalina. But maybe it's the same protection there as well? The binary didn't work on Ubuntu 18.04 nor CentOS8 either, but I'll report a issue later on today.

  • anderspitman · 2048 days ago

    Thank goodness AAPL is here to protect us from doing cool stuff.

  • oscargrouch · 2048 days ago

    Its good from a security perspective, but also its a nightmare to know that the mothership knows everything you run on your machine. This will also prevent distribution from outside of their appstore giving it will prevent updates. So as anything Apple, the world security here is being used as code for personal machines remotely controlled by the mothership. You trade privacy and freedoms for a little convenience.

    • zimpenfish · 2047 days ago

      > This will also prevent distribution from outside of their appstore giving it will prevent updates. I think that if an app is correctly codesigned, Gatekeeper has no issue with it. Also if it's not correctly codesigned but you've ticked "allow unsigned binaries", you can still run them. > the mothership knows everything you run on your machine I can't remember the details from the last time this came up but IIRC it only sends a hash and possibly even then only the first time you launch it. Either way, they're not going to block out-of-appstore distribution except if you're a known malicious actor.

  • easton · 2048 days ago

    It looks like you can get around this by adding Terminal.app as a "developer tool" (i.e. it can create processes without hitting Gatekeeper), maybe that works? (once I did that and relaunched terminal, I can run cosmopolitan binaries). https://docwhat.org/upgrading-to-catalina

  • Wowfunhappy · 2047 days ago

    What happens if you switch off Gatekeeper? `sudo spctl --master-disable`

• lemniscare · 2048 days ago

  Yeah another killed: 9 on Mac BigSur. Could you export the tool in a way that we don't have to zip new files into it? Instead we would have to gcc the whole thing once and be done with it.

• johnx123-up · 2048 days ago

  Someone already reported the same https://github.com/jart/cosmopolitan/issues/56

• brabel · 2048 days ago

  On Mac Big Sur it doesn't seem to work either: ▶ zsh -c './redbean.com -vv' W2021-02-26T12:12:24.019387:tool/net/redbean.c:1144:redbean:56779] setsockopt(server, IPPROTO_TCP, TCP_FASTOPEN, &yes, sizeof(yes)) → EINVAL/err=22/errno:22/GetLastError:0 W--------------------000028:tool/net/redbean.c:1145:redbean:56779] setsockopt(server, IPPROTO_TCP, TCP_QUICKACK, &yes, sizeof(yes)) → ENOPROTOOPT/err=42/errno:42/GetLastError:0 V--------------------000023:tool/net/redbean.c:1166:redbean:56779] 0.0.0.0:8080 listen ^CV2021-02-26T12:12:27.012972:tool/net/redbean.c:1183:redbean:56779] 0.0.0.0:8080 terminated EDIT: I am an idiot: it's working just fine... the messages looked like an error to me, but visiting https://localhost:8080/ does show a nice HTML page!

• awestroke · 2048 days ago

  Disable Gatekeeper: sudo spctl --master-disable

• snarfy · 2047 days ago

  No luck for me on win10. I can open the archive but get errors trying to add to it. 7zip and win explorer zip both fail with either unsupported or corrupted archive errors. Since it's a .com file, after downloading, win 10 also claims 'this dangerous file comes from the internet and is blocked' which you have to check a box to unblock it. I did, but still get the errors trying to add to it.

• fastball · 2046 days ago

  Fixed for me (macOS) in this[1] commit. [1] https://github.com/jart/cosmopolitan/commit/218ef491476f15ab...

this is outstanding! speechless.

Odd, I'm getting $ ./printimage.com image.jpg run-detectors: unable to find an interpreter for ./printimage.com

• jart · 2049 days ago

  Author here. That error happens if you have binfmt_misc enabled. The solution is simple: sudo sh -c "echo ':APE:M::MZqFpD::/bin/sh:' >/proc/sys/fs/binfmt_misc/register" I intend to upstream a patch with the Linux kernel so it can do this automatically. If any kernel devs are reading, please email me!

  • est · 2049 days ago

    Does it work on cheap Xen vps? Because "/proc/sys/fs/binfmt_misc/register" looks absent from mine.

    • jart · 2049 days ago

      That's strange. I'd recommend filing an issue. Any details on how I can reproduce the error will be helpful, even if I have to temporarily rent one of those things. I'm on a mission to make binaries portable.

      • na-- · 2048 days ago

        FWIW, it works on QubesOS, which is Xen based. Though it works only in my Fedora and Debian VMs, but not on my Archlinux-based one. I think that's because I have WINE installed in the Archlinux VMs, since that's what gets started when I run ./redbean.com. But even after I remove WINE, it still doesn't work, I probably have to restart (i.e. remove WINE from the TemplateVM, not the AppVM) for it to work. In any case, incredible work, I am in awe!

  • FeepingCreature · 2048 days ago

    Instead of sudo sh -c to sudo the redirects, consider echo 'abcde' |sudo tee /proc/bla. Less nesting :)

    • npongratz · 2048 days ago

      Do you mean less syntactical nesting? The pipe to sudo tee creates a new child shell, just as sudo sh -c does.

      • FeepingCreature · 2048 days ago

        Yes.

It's a neat coincidence that this is on at the same time as an article about CP/M, because I think there could be some overlap in the programs that would be a good fit for cosmopolitan. Sure, there's server appliances like this, and I hope someone makes a neat services wrapper to abstract at least some platform's intricacies so you can do a "cosmo-service up redbean" on BSD/Linux/Windows. But coming back to CP/M, there you had a lowest common denominator of terminal applications, too, but spread across different architectures. And you could still produce some quite intersting, if a bit business-like applications. These days you probably can even rely on more ANSI colors and maybe even unicode fonts (hopefull as an option, not mandatory). The ZIP characteristics of the APE format make it even easier to distribute a whole application in a rather simple way. Yes, sure, you can do regular Unix-style servers and pipe-it-together CLI tools, but I wouldn't mind more self-contained "business" applications with a lo-fi aesthetic regarding interface and API usage. The PICO-8 of TUIs…

• jart · 2049 days ago

  Author here. We're living in the most exciting time for developing terminal applications. When Microsoft unexpectedly added support for VT100 and XTERM codes to CMD.EXE it totally changed the equation and ANSI became universal for the first time. Blinkenlights is an example of a TUI application I created using Cosmopolitan and it literally works everywhere. https://justine.lol/blinkenlights/index.html You don't need curses. All that's needed is an ioctl() call which flips a bit in termios. Cosmopolitan polyfills that across operating systems. Another cool example of a demo app is this conway's game of life tui gui: https://justine.lol/apelife/index.html

  • rexreed · 2048 days ago

    Does that remove full binary compatibility for those running on older OS from Microsoft with an older cmd.exe?

    • jart · 2048 days ago

      Cosmopolitan supports Windows 7. The WinMain() polyfill is programmed to gracefully fall back if ANSI isn't available. So the binary still loads fine. You just need to run it in something like MinTTY instead if your TUI program is run on an old version of Windows.

Just make sure you don't have another server running on :8080. I had syncthing running and kept getting some error whenever I tried to run redbean. The error didn't make much sense, but eventually I realised that this was the casue of the error. I'm really impressed how portable this is. The only improvement I can see is if it auto-opened your browser to :8080. That way it would be easy to distribute the binary and have people run your application and just interact with it through their browser. No need to ship electron then!

• r0b05 · 2048 days ago

  Let's say we are able to do this. Would we have any security concerns to be addressed?

  • anaganisk · 2048 days ago

    How different would it be from opening a random site in your web browser?

  • t0astbread · 2048 days ago

    There is the possibility of other programs making HTTP requests so you shouldn't assume trust on any request automatically. What you can do is give the browser a cookie once when your app starts which you then check on every request. Oh and listen on 127.0.0.1, not 0.0.0.0. That way your app is not needlessly exposed to the network.

• sva_ · 2048 days ago

  I think you could trivially issue a system() call, you'd just have to use the corresponding command to open the default browser for each particular system. (Linux: xdg-open , MacOs: open , Windows: start )

I can't get it to run on Ubuntu. "run-detectors: unable to find an interpreter for ./redbean.com"

> exes still end up being roughly 100x smaller than Go Hello World > Is the Cosmopolitan Runtime lean and mean like Go's 2mb Hello World executables? :-)

>the sources that went into creating it are nontrivial and only recommended reading if you really love printing lemurs in terminals. this is delightful

Why does the magic numbers list have a XENIX column?

• jart · 2049 days ago

  Author here. The name was chosen mostly out of playfulness to describe Windows since I seem to recall XENIX being the earliest example of Microsoft dipping their toes in the water with UNIX support.

  • robaato · 2048 days ago

    Back in the day ('80s) worked at Logica with colleagues who did Xenix support!

> That performance is thanks to zip and gzip using the same compression format, which enables kernelspace copies Can someone elaborate on the relationship between gzip and kernel space copies?

• jart · 2049 days ago

  Normally when a browser asks for Content-Encoding: gzip, the http server needs to schlep the file into memory, compress it, and only then is it able to send the output to the client. Since redbean reads files from a zip archive, that file is already compressed. So what it does is it uses mmap() to load the zip executable in memory. Therefore instead of read()+write() it can just call writev() on the mapped memory area. That enables us to skip the extra copy. The kernel can just copy the gzipped file contents directly to the network wire without having to pay any userspace context switching penalties.

Awesome! You could even use a zip file as a container for the assets you want to serve! see https://gildas-lormeau.github.io/ (the page is a self-extracted zip file). And here's a demo showing how to update an image: https://www.youtube.com/watch?v=RJWhDmYhnX4

Someone please be kind and explain to me how this works? Don't different executable formats need different headers? Don't ELF need "0x7F ELF" header? Don't Windows executable need "MZ" header? How is it fulfilling both requirements simultaneously?

• Gys · 2049 days ago

  The OP explains https://justine.lol/ape.html

This is awesome, I've wanted a way to package a webapp for distribution locally and I think this + sqlite would be a perfect combination.

Does it work with this level of bare metal https://news.ycombinator.com/item?id=26271356

If you want to learn more how these things work I'd highly suggest going through the PoC||GTFO archive ( https://github.com/angea/pocorgtfo/blob/master/README.md ) and check out entries by Ange Albertini or entries named like "This ZIP is also a PDF".

• donaldihunter · 2048 days ago

  I came here for this. E.g. https://www.alchemistowl.org/pocorgtfo/pocorgtfo19.pdf "This file, pocorgtfo19.pdf, is valid as a PDF document, a ZIP archive, and a HTML page. It is also available as a Windows PE executable, a PNG image and an MP4 video, all of which have the same MD5 as this PDF."

Would GUI programs be possible with this? I don't mean some cross platform framework monstrosity. But if some least common denominator was found to be able to create window, add some input fields and buttons using the native UI elements of the OS? Does the Win32 and Cocoa APIs for example, also have "magic numbers" that are kept stable?

• hnarn · 2048 days ago

  > Would GUI programs be possible with this? This is a cross platform web server, not a cross platform web browser.

  • bni · 2048 days ago

    Sorry, I forgot that this post was about the web server and not about αcτµαlly pδrταblε εxεcµταblε

One small complaint I have is that there is already a popular project called RedBean: https://www.redbeanphp.com/

Out of curiosity, are there any general benchmarks available as to how this stacks up to something like nginx when it comes to static page serving?

The amount of work that went to this is insane. She made a (full ?) implementation of libc: https://github.com/jart/cosmopolitan/tree/master/libc

• dev-tacular · 2048 days ago

  Right? It's amazing to see what some people can accomplish with the same 24 hours a day we all get.

I’m not an expert at C code but it’s really nice to read redbean.c - lots of relatively small functions with great names. Good example to follow.

This is the coolest thing I’ve seen on HN in a long time. Nice work

This is so powerful that it can be used as a "weapon": it's the perfect lib for making malware. I think this is a compliment!

• cirowrc · 2048 days ago

  scary scary scary

• nabla9 · 2048 days ago

  She started her career with a very successful spam tool. She was 14 at the time.

  • syncbehind · 2048 days ago

    I definitely was not coding when I was 14. Dang.

TLDR: This is a packed file format that can look to the OS as being executable (for Windows/Linux/OSX). The code is x86 and so offers native performance on those processors. On other platforms, an x86 emulator is built in so can't offer native execution speeds. While neat, its not the "best of all worlds" due to the lack of native code on anything other than x86/x64. Also, claiming "bare metal" is supported is a stretch as you are limited to having no I/O of any form (as there is no platform code).

• OOPMan · 2048 days ago

  No I/O eh? I'm sure the FP zealots will love it :-)

• jart · 2048 days ago

  I'm just one woman. What Cosmopolitan does so far, it does really well. It was only as recently as a few days ago that I got mmap() and malloc() polyfilled on bare metal. It has serial uart i/o. It's going to have e1000/virtio sockets soon. You can help me will that future into existence. I need people who know o/s dev and can help me write code that does things like correctly set up pic controller.

  • TickleSteve · 2048 days ago

    Sorry, wasn't meant as a criticism, just as an explanation because people were assuming magic... Taking on bare-metal is a significant job much greater than your original task and you may be better targeting something like buildroot instead, after all what is the Linux kernel but a hardware abstraction layer.

    • jart · 2048 days ago

      How hard could stdio/sockets on metal possibly be? So far bare metal has been a walk in the park. I love the fact that PML4T lets me do things like move memory without copying it. I want to be able to have that power without schlepping in the entire Linux world. I believe it should be possible for programs to be able to boot on metal as just programs which are tiny and auditable. Especially considering everything runs on hypervisors these days, which are in some respects the true operating system. Ring 0 is the new Ring 3. Who among you is willing to accept being pushed to the outer bounds of computing? If MINIX gets to run in Ring -3 then it should at least be easy for our programs to run in Ring0.

      • brabel · 2048 days ago

        As someone who knows programming in higher level languages well but has no experience with low level, bare metal, how can I get started with this stuff you're working with? > So far bare metal has been a walk in the park. I love the fact that PML4T lets me do things like move memory without copying it. > I believe it should be possible for programs to be able to boot on metal as just programs which are tiny and auditable. I would love to be able to learn more about this. Maybe even contribute to your project :) perhaps a good way to start would be to address some small tasks you might have laying around?!

        • jart · 2048 days ago

          And I would love to encourage you to do so! If you're working for Cosmopolitan then I'll be working hard to support you. There's a long history of depth to metal. Many people start off learning it by having fun with the old skool ibm pc boot process. Here's a gentle introduction: https://justine.lol/blinkenlights/realmode.html and https://github.com/jart/sectorlisp and http://reddit.com/r/osdev Those concepts are implemented in the Cosmopolitan codebase here: https://github.com/jart/cosmopolitan/blob/master/ape/ape.S That file runs a few hundred lines of old school assembly in order to bring us into the modern era. Towards the end of the file you'll notice it starts calling functions that are written in C which configure memory: https://github.com/jart/cosmopolitan/blob/master/libc/runtim... After it configures memory, the ape.S file is able to call _start(). It also sets a bit in __hostos so that functions like write() know to use the serial port instead of issuing system calls: https://github.com/jart/cosmopolitan/blob/master/libc/calls/... That's the simplest possible example of a driver with spin locks which uses the x86 IN/OUT instructions. If you want to take a dive off the deep end right now with Cosmopolitan on metal -- simulated -- then try the following build commands on your Linux terminal: git clone https://github.com/jart/cosmopolitan cd cosmopolitan make -j8 o//examples/hello2.com qemu-system-x86_64 -m 16 -nographic -fda o//examples/hello2.com sed -i -e 's/USE_SYMBOL_HACK 0/USE_SYMBOL_HACK 1/' ape/ape.S make -j8 o//examples/hello2.com o//tool/build/blinkenlights.com o//tool/build/blinkenlights.com -rt o//examples/hello2.com

      • TickleSteve · 2048 days ago

        Its not the difficulty, but the scale. Working on one bare metal system is relatively easy as long as you're willing to dive into drivers. Getting 1001 I2C drivers or maybe writing a drivers for tens of different types of flash devices is a different matter. Your project will quickly become overrun with drivers for literally tens of thousands of devices... then comes maintenance. Look at the Linux source tree... Good drivers are not trivial either. Simple ones are of course. Basically, its a time-sink and it doesn't become useful for claiming "portability" until you're approaching the scale of something like Linux.

        • jart · 2048 days ago

          You're confusing portability with requirements. I'm aware that a long tail exists. Linux is a big tent for that sort of thing. I think that's great. I also want the freedom to live outside that tent during the times when all I need are the other 85% of use cases, which as I've shown, can be easily achieved by just one person. I don't believe that just because I ship actually portable executables that are able to do e1000/virtio that every hardware manufacturer is going to be kicking down my door eager to get support for their silicon merged. Linux is already doing a great job at that and I think Cosmopolitan is complementary.

Windows 10 (10.0.18363 Build 18363). Just downloaded and executed. No zip modification at all. W2021-02-26T05:00:59.959364:tool/net/redbean.c:1141:redbean-2021-02-25:49336] setsockopt(server, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(yes)) → ENOPROTOOPT/err=10042/errno:10042/GetLastError:0 W--------------------000674:tool/net/redbean.c:1145:redbean-2021-02-25:49336] setsockopt(server, IPPROTO_TCP, TCP_QUICKACK, &yes, sizeof(yes)) → ENOPROTOOPT/err=10042/errno:10042/GetLastError:0 Its amazing if works though, i can put my static site in an exe. That is so cool. EDIT: Runs on without modifying even with this message. EDIT: Cannot modify zip on windows. "File not found or no read permission" using explorer. With 7-zip "Operation is not supported." EDIT: Used WSL to update zip which then results in SIGSEGV

So it's a tiny virtual machine. Intriguing, but not really shocking.

• vanderZwan · 2048 days ago

  What is your definition of a virtual machine and what do you think is happening here?

• joh6nn · 2048 days ago

  "no wireless. less space than a nomad"

• mikewarot · 2048 days ago

  >So it's a tiny virtual machine. Intriguing, but not really shocking. Nope... it's native code, that runs on multiple platforms, all in one small executable. How? Lots of very clever hacking, and trimming all the accumulated dreck out of the normal C runtime, while making it work across all the supported platforms. (Polyfill is the term she uses) This is the most amazing thing I've seen in a decade or so. It is right up there with GIT in terms of the possibility space it opens up.

This is brilliant.

I was literally thinking about how cool this would be if it existed like a few days ago!!! Down to the freaky portable executable! Congrats on reading my mind and making what was supposed to be impossible?

Ok well this doesn’t work on Mac, so much for not needing to cross-compile on every architecture. It’s because of the OSX Gatekeeper but if majority of people need to self-sign or otherwise compile this binary I’m not sure if it is that portable be it if it’s only a Mac thing. This dumps the zip content directly on the wire skipping an extra kernel write. There have been attempts before to run the same binary on multiple architectures by inserting magic headers without having to recompile. There are shell/batch versions of the same thing [0]. The issue is such projects usually never get much traction since every programmer is wedded to their choice of OS and typically relies on APIs specific to each system. There are also “fat binaries” which accomplish the same thing if one truly wanted a cross platform single executable. There isn’t a huge need for them currently and not just because they’re “fat” (something this approach seems to tackle primarily) so I expect for the same reasons there won’t be many needs for this approach either. I see a lot of people making the comparison between this and the OG Linux tools. I absolutely don’t think this qualifies as one, most old Linux tools are irreplaceable or at least were when they first came out and there were no alternatives. Serving a bundled zip file on a web server across many different platforms is not something people are dying to do. You can run a web server like nginx or Apache to index your files or if you really hate those why not a single line command which is supported on almost any platform: python -m http.server 8080 [0] https://github.com/llamasoft/polyshell

• waheoo · 2048 days ago

  Youre really missing the point here. This zip file is an executable webserver, you just add files to it and it will serve them when you run it. If you can't understand what is cool about that maybe just don't speak.

As amazing as this is unfortunately it doesn't work for me and I am on Linux. I get bash: ./redbean.com: No such file or directory after chmod-ing. Maybe I am dumb but yeah.

• anderspitman · 2048 days ago

  Is your system 32 bit by chance?

  • thecodrr · 2048 days ago

    No, not really. It's x64. I really wanted to give this a try.

• johnx123-up · 2048 days ago

  Did you make it executable with chmod +x ?

  • thecodrr · 2048 days ago

    Yes. That's the funny part. It doesn't work despite that. uname -r gives 5.7.7-amd64-desktop.

• zer0gravity · 2048 days ago

  I get the same :( I'm on Archlinux.