Show HN: I've updated my Linux System Call website and API (syscall.sh)

102 points by guitmz •

I first released those in back in 2020 and now I finally added more architectures, Kernel selector, syscall descriptions and a few other niceties. It should work a bit better on mobile devices too but I am no CSS expert.

The API is at https://api.syscall.sh

If syscalls are so similar, for instance, like 'read' and 'readv' or 'pread', 'sendmsg' and 'sendmsgv' or 'fork' and 'vfork'.

Does it still make sense to term or classify them as two distinct syscalls?

It definitely makes sense to have a section "similar syscalls on this architecture" on the details page.

There's a "Same syscall on other architectures" section already, maybe I could add the Similar section next to it. Edit: but this means it needs a way to actually know they are similar apart from only their names

This maybe unpopular idea if you've avoided AI thus far, but it does seem like an LLM would make short work of that task.

I wouldn't call any of them similar, except for 'pread' maybe. Each one has a lot of subtle semantics. Either way, it's syscall.tbl that determines whether they are distinct syscalls, not the author's page

I'm just noting this because it makes the number of Linux syscalls seem very exaggerated at a whole around 400 syscalls.

But, anyways, not like it matters that much but still.

400 distinct syscalls is a very large number of syscalls if you generally presume they're entirely distinct.

I think the number is mostly meaningless even if we ignore whether they're distinct or not. Consider ioctl() - it's a single syscall but if I remember correctly the number of subcommands is in the thousands.

That's right.

Moreover, for a large number of traditional Linux syscalls (including the many variants of read/write/send/recv) there are now better alternatives that are invoked as subcommands of a single syscall: io_uring_enter.

io-uring is great for this.

However, I am bit skeptical as to whether io-uring will ever support all Linux syscalls.

At this point I am convinced that io-uring primarily supports I/O API / syscalls. That would mean syscalls relating to disk and network I/O. While this covers a lot of syscalls it most definitely doesn't cover all.

Though, to give some credit her, it does also cover to syscalls relating to kernel features like futex but yeah, for the most part it doesn't cover much that doesn't have to do with I/O.

Which is obviously not a problem but raises the question of whether it ever will support other kernel features.

Yes, the ioctl has been notorious for sneaking in arbitrary amount of functionality into the kernel space, and serving generally as a kitchen sink: "You can rarely convince Linus to make a new syscall, but making a new ioctl is much easier". That's still one syscall though.

And how do you even compare how the distinct the syscalls are (e.g., pipe() and open() are about 70% similar)? And after that, what do you do with this metrics? How do you sum it up? Again, pipe() and open() are 70% similar — does that mean we have 1.3 syscalls instead of 2?

I would count "pipe" and "open" together as 0 (useful) syscalls, because both have been obsolete for a very long time (even if they occupy 2 slots in the syscall table and those 2 slots will remain occupied forever).

These 2 syscalls have a history typical for Linux syscalls, where frequently new syscall variants make completely obsolete their predecessors, but the old syscalls must still remain supported forever by the newer Linux kernels, to allow running legacy programs.

"pipe" has been obsoleted by "pipe2" (on 2008-10-09), which in turn has been obsoleted by "IORING_OP_PIPE".

"open" has been obsoleted by "openat", which in turn has been obsoleted by "IORING_OP_OPENAT".

Most users do not invoke Linux syscalls directly, but they use the standard C library, which has frequently shielded the end user from the evolution of the Linux syscalls. For instance, you might use the functions "open()" and "pipe()" in your C program, but the C library might invoke instead of those the more modern syscalls "openat" and "pipe2", after adjusting their parameters.

I think you're missing the point I'm trying to make. Both pipe2 and openat has about 70% similar functionality (and so do the IORING_OP_PIPE and IORING_OP_OPENAT operations) — really, the only difference is that pipe2 returns two new fds instead of just one, and that the file opened is not a regular file. So, are those 2 syscalls or 1.3? What about triples or quadruples of syscalls with overlapping functionality, do we go full inclusion-exclusion formula on them? What about that 70% similarity estimation — how do establish that? Do you count the similarity/overlap of the underlying implementations? Only of the user-facing functionality? Both, somehow? Something else entirely?

Or, you know, we can instead not engage in this subjective quagmire and look at the actual, factual number that undeniably exists. It may not be very useful for any practical purpose except for pointless dick-measuring between OSes with radically different APIs and internal architectures — but neither is this trying to measure syscall similarity, really.

The number of Linux syscalls depends on the architecture and on x86-64 there are over 450 Linux syscalls.

Nonetheless, about one third of those syscalls are obsolete (being replaced by newer improved syscalls), even if a large number of programs are still using them, and about one other third of them are rarely used in the environments familiar to most Linux users, like personal computers and small servers.

The obsolete syscalls inflate a lot the number of Linux syscalls, because there are a lot of syscalls that were originally defined with an inadequate set of parameters, then they were fixed by adding parameters or changing the meaning of the existing parameters. There are many syscalls were even 3 or 4 attempts were needed until reaching an acceptable definition for their parameters.

New programs should use only the newest improved syscalls, but the documentation of the Linux syscalls in various places is usually not up-to-date itself, so a lot of people are still using older syscalls, without being aware that they are suboptimal. So the obsolete syscalls will have to be supported forever by the Linux kernels (though in most cases when an obsolete syscall is invoked it just fills the parameters with adjusted values and then it jumps to the current syscall).

I'd love to learn more about the syscalls that took 3 or 4 attempts.

Well, there really is about 400 syscall numbers allocated, and each has its own entrypoint in the kernel. That's an entirely objective criterion as opposed to arguing "well, sbrk and mmap (or recv and read) are kinda similar, maybe we should count them as just one syscall instead of two", because in the end we can might as well say "well, technically there is only one syscall, it just has an additional seventh argument that serves as the 'function denominator', like the ioctl's request parameter, you know".

I'm obviously not going to discuss this argument further.

Trying to counter a logical argument with a technical argument is pretty lame and doesn't make for a healthy conversation.

Uh, sorry, what? Trying to measure similarity between syscalls is a) impossible, b) pointless, so obviously I'd really like to understand why someone would want to do something like this — but I'd rather not speculate about what and why you think and believe, and instead to just straight hear from you.

If you're interested in why I think what I think — there's a sibling comment where I tried to articulate my position. I'd like to hear about yours, if you don't mind.

P.S. You didn't actually offer any argument, logical (?) or technical (?!), in your original comment, and I think you're actually have the meaning of those words reversed: "logical argument" appeals to logical relations between the abstract concepts, while "technical argument" appeals to the underlying technology (the kernel implementation in our case) and tries to deal with particularities of reality. So if anything, my argument was the logical (semantical?) one, not that it really matters.

Yeah, sure, but the obvious fact that is a lot of syscalls are 'family' relatives.

And they wouldn't ever count as distinct syscalls.

> they wouldn't ever count as distinct syscalls.

According to whom, exactly?

If a syscall morphs into several slightly variant syscalls, then there's no way these are distinct syscalls.

If a syscall is designed to counter the flaws have been observed in another syscall, then there's no way this is a distinct syscall.

If a syscall is designed to add feature to an another syscall, then there's no way this is distinct syscall.

And so on and so forth.

If you actually look at Linux kernel code, you will quickly note that a lot of these related syscall even happen to share a large amount of implementation code.

So, yeah

  > 400 distinct syscalls is a very large number of syscalls

Compared to what? It seems windows has 2000+ syscalls all combined (nt+win32). For just ntos there are 490.

https://github.com/hfiref0x/SyscallTables

https://github.com/j00ru/windows-syscalls

And does the number it matter? Why can't it be 1000, 2000, 4000. In my opinion the interface is free to be defined as long as it is documented properly.

Yes because they have a different "syscall number".

Doesn't really matter if it has the same name or similar (or identical) function. This website seeks to describe the syscall ABI and that means that each number should get it's own entry.

And there is no identical syscalls in the list, they all have slightly different behavior.

It would be nice to make a banner or warning for deprecated or surpassed syscalls. For example (as mentioned in other comments) you should probably use pipe2 instead of pipe and similar other examples.

On x86_64 and x32, the syscall number goes in eax, not rax. The upper 32 bits of rax are ignored and can be anything.

Good catch, thanks! Since 5.14 (I think?) the kernel only reads eax (before that it used all of rax and returned ENOSYS if the upper half wasn't zero), so I'm adding a note about it to the x64 and x32 conventions

According to [1], the kernel has flip-flopped a few times on this, but the original behavior was to ignore the upper 32 bits.

[1] https://lore.kernel.org/all/20210518191303.4135296-5-hpa@zyt...

This is very handy, thank you. Currently learning aarch64 assembly

This is really great resource, didn't know about it. But one question not related to the website above - why the syscalls vary per architecture if the linux API is the same?

Or is it I am confusing it with the C standard library/POSIX and assuming that it should be the same everywhere?

The Linux kernel API is the same everywhere, but the syscall ABI (numbers, calling convention, entry paths) is specific to a architecture, while it's the C library that wraps those details into the portable POSIX API we code against

Also newer archs (riscv) don't support certain syscalls (open, fork) they are still functions you can call under your standard musl,glibc but they inernally call openat, clone.

I see, but why were they numbered differently? Why not use the same numbers?

As far as I know, the numbers were assigned independently by each port's developers in the 90s as they added calls in their own order. Then it became a frozen ABI nobody could ever renumber, and couldn't have been identical anyway since arches need different syscall variants for things like 32bit vs 64bit types

I also made a CLI for consulting the syscall API when I originally released the website. I think I can tidy up the code and make it public too. It uses a embedded dataset so it does not have to query the API endpoint all the time

I am not familiar with the internals, so excuse my question, but how does one builds this? Separately, how often can it get stale?

It was stale for many years because I had a script to generate the tables for me that ran manually and then I had to convert the data into SQL so my API could use it (and also convert to html for the frontend). I changed the script to generate the dataset from Kernel sources as JSON instead and the API loads it and makes it available.

Then the frontend is refreshed from the API weekly via a cronjob with some simple Go templating

That website messes with my graphics for some reason.

Would also recommend using commands like:

echo '#include <sys/syscall.h>' | gcc -E -dM -xc - | grep -i "read"

To get the real values on your PC to make sure.

I had LLMs give wrong numbers or constant definitions several times but getting it to generate a command like this and seeing it for myself works every time.

Also chromium docs has a table:

https://www.chromium.org/chromium-os/developer-library/refer...

It would be very useful to just make this website directly link to the syscall number.

Hacker News