ID:1754732
 
Applies to:DM Language
Status: Open

Issue hasn't been assigned a status value.
Lemme be honest: the miserable state of the compiler is an endless source of frustration for developers using the BYOND platform. After all, there's quite a few developers out there (think several dozen from SS13 alone) who have to deal with the problems and limitations of this damn thing.

It's not currently in active development. It hasn't been since Dan split (though Lummox has made some valiant efforts), and its todo list of fixes and enhancements is over a decade long at this point.

And it's getting quite old making feature requests and bug reports that we all know are never getting addressed.

So how about letting us take a whack at it?
I don't think the source for the compiler can be released without opensourcing the whole thing. But that'd proably be the ideal solution. I'm pretty certain I could get a lot more performance out of BYOND by making simple, small and completely non-backwards compatible changes to the compiler and VM.
I am sorry for your frustrations. It isn't true that we haven't touched the compiler; aside from lots of language additions in the last ten years, we've also improved size limits, updated a number of the algorithms, and integrated threading. The compiler is not the most pleasant thing to work with because it was written by Dan from scratch instead of via Lex/yacc (flex/bison, maybe there's something newer now!), which is the standard these days.

As Tobba said, it's really not possible to disentangle the compiler from the server-- for one thing, it seems like the main purpose of improving the compiler would be to update the language and/or optimize the VM, which would require changes to the bytecode.

I have absolutely no doubt that there are people here who could contribute to this effort. However, I am not ready to open-source this. There are a lot of reasons, but it is largely because the software is tied into the hub, which is also the source of our business at the moment. As such, we are going to be intertwined with any sort of open-source effort and we simply don't have the time to put that on the table right now. However, should we distance ourselves from BYOND, or start making enough where we can get more official people involved to lead this sort of effort, it's possible. Certainly if we abandon BYOND completely, I will just release the code into the wild (sans hub, but I suppose people could run their own hub servers if that is something they desire).
As has been requested, suggested and said multiple times, I'm sure there's some on these forums you know that would be willing to use some of their free time to help with things like this, which would get BYOND at least a slightly bigger working force behind it, which could get it back on track. I suppose this has it's problems but I'm sure if given some effort it could work out.
Wouldn't improving upon the existing product here, EG making things faster, adding some things people have been asking for, and fixing the bugs/problems/unfinished features that are lurking around be good for business? As it is right now, your efforts are greatly appreciated but they aren't proceeding all that fast, and money won't flow forever.
Tom wrote:
I am sorry for your frustrations. It isn't true that we haven't touched the compiler; aside from lots of language additions in the last ten years, we've also improved size limits, updated a number of the algorithms, and integrated threading. The compiler is not the most pleasant thing to work with because it was written by Dan from scratch instead of via Lex/yacc (flex/bison, maybe there's something newer now!), which is the standard these days.

Yeah, you guys have done solid work on it. It just feels like the current state is "legacy" rather than "active development", since many important issues are languishing with no apparent progress for years.

The most painful problems are the basic parsing errors and design flaws, which have been left on the backburner while other issues have been addressed and many features have been added. In my eyes, these are the absolute top priorities: I would fix them before even starting to think about working on features for the webclient, or anything else, so it's especially frustrating to see them treated as afterthoughts or legacy support issues. In an open-source project, we would just fix them ourselves, but with a closed one all we can do is wait, complain, and write wacky hacky workarounds.

Tom wrote:
As Tobba said, it's really not possible to disentangle the compiler from the server
There are a lot of reasons, but it is largely because the software is tied into the hub, which is also the source of our business at the moment.

I understand. So if the entire thing is unrealistic to release right now, what about releasing selected sections of the source? I'd hope that the code that actually parses, processes, and compiles DM into the bytecode is at least somewhat separate from the server code. Even just publishing that would be very helpful: it wouldn't allow us to write a functioning compiler ourselves, but we could see the problem areas and suggest possible fixes. It'd also be good to know what the compiler is actually doing rather than having to speculate as to why a certain piece of code is compiling strangely: for example, how and why the bare return keyword is processed incorrectly at the end of a proc (resulting in extra overhead), and what can be done to fix this (may be a very simple patch).

Also, would it be feasible to separate off the pre-processor (eg as a script, or separate program) and release that? It's completely independent from the compiler in principle, and shouldn't have serious financial or security concerns to complicate things.
It needs quite a bit of work, and could be very helpful if updated to even C99 standards, so it'd be good if we could work on that at least.