What we talked about
Takashi Kokubun is a Staff Developer on the Ruby JIT team at Shopify, where he works on ZJIT, the next-generation Just-In-Time compiler for Ruby that shipped with Ruby 4.0. Before going deep on compilers full time, he worked on distributed systems and infrastructure, and along the way earned a Master of Science in Computer Science from Georgia Tech while working full time in the United States.
Show notes
Takashi, welcome to the show. 
Full transcript
Federico Ramallo (00:00) Welcome back to the pre-Vered podcast where we spotlight extraordinary people and remarkable talent reshaping our world. Today we’re joined by Takashi Kokubun. Takashi is a staff developer at Shopify where he works on optimizing ZGIT, Ruby’s JIT compiler. He has a strong focus on performance and compiler optimization.
Before that, he spent years working on infrastructure and distributed systems, and over time made a transition from doing compiler work in his spare time to making it his full-time career. He’s also earned a Master of Science in Computer Science from Georgia Tech while working full-time in the United States, which is a great achievement. Takashi, welcome to the show.
Takashi Kokubun (00:52) Thank you. My name is Takeshi Kokubun. Glad to be here at podcast.
Federico Ramallo (00:58) I am honored to have you here today.
Takashi Kokubun (01:01) Thank you for having me.
Federico Ramallo (01:03) So for people meeting you for the first time, can you tell us a little bit about what do you do today?
Takashi Kokubun (01:09) Sure, I’m on the Ruby infrastructure team at Shopify. It’s a sub team called Ruby JIT team in the Ruby infrastructure team. I work on ZG, that is a JIT compiler, Just-In-Time compiler for Ruby programming language that has been merged to Ruby 4.0. We released it last year in the Christmas.
we developed ZG as the next version of another JIT compiler called YG that has been merged to also this, the Ruby programming language since Ruby 3.1. just to clarify the context, just-in-time compiler is generally for speeding up the programming language in general, especially for dynamic programming languages like Ruby, it’s useful to have a…
just-in-time compilation, that means you can analyze how a program works just-in-time and then use that runtime information to leverage that into optimizing how your code will actually work in the machine code. Just-in-time compiler generally speeds up program by generating a native machine code instead of using the interpreter implementation.
Ruby, C-Ruby, C-Ruby is the reference implementation for the Ruby programming language. And that generally uses the interpreter as the execution engine for the Ruby programs. But now with YG or ZG, you can compile Ruby programs into from interpreted by interpreted implementations into machine code that can be natively executed by the CPU. So yeah, that’s what we do up here at Shopify. Our team has like.
several people in the RubyG team and like 10 or a few more people in the Ruby Info team. And then our group, Ruby and Rails infrastructure team group have like 40-ish people who are working full-time on Ruby or Rails-related open source project. yeah, I’m also a Ruby committer working directly contributing to the C Ruby repository.
So yeah, that’s I think what I’ve been doing these days.
Federico Ramallo (03:21) That’s amazing. That’s amazing. We can talk a little bit more about Just-In-Time compiler because I think that’s an amazing piece of technology. When did you first become interested about software engineering?
Takashi Kokubun (03:35) I think that was when I was a kid, when I was doing games, also had some, so like when I was like five or six years old, my uncle gave me a computer and he was a programmer at the time, so like he had a bunch of computers and…
One of them was somehow given to me while none of my families were using computers at all. Somehow it was mine and then I got access to just using the computers. then over time, at that time, I didn’t have access to the internet. And so I wasn’t able to learn anything out of just using the computer. But when I turned like travel or so, I got access to the internet and then…
using the internet, I met some people online and then they were, some of them were like making games by programming. And then I also wanted to make one because I was into doing games. And so that time I started making some games, but it wasn’t required to do programming there. You just needed to use a framework that allows you to make a game without programming. So.
I wasn’t doing that, but because I needed to publish the website to like publish the games, I hosted the website and then that made me do some like tricks, like doing JavaScript to make some like actions in the website. then like that was the introduction to programming in general. And I was like at the time wasn’t able to do programming at much.
Other than just copy pasting the JavaScript code and also some hosting some PHP programs to do some forms there, but I always respected people who can code like or write programs themselves there were some some in the community and so that sort of made me think like people who can write code are cool and so when I also went then I
When I became a high school student, there was some special class in the school that teaches you how to do some programming. And that was the first time that I actually started doing or learning some programming. There was a course in the program that teaches you how to write C or how to make robots work in embedded programming language, programming in C.
And there’s another class that teaches you how to solve web puzzles or do some competitive programming, like lead code in Java. And so in high school, I learned how to write some C and write some Java. So that was first introduction to programming. then as soon as I graduated from high school, I started studying C very seriously. And as soon as I entered the university,
I applied for a part-time job writing C and there, there was a program for people who don’t know how to do programming at all and I learned how to write C there as well and from there I got like different part-time jobs as a student and then there like I think I started growing interesting software engineering through that experience.
Federico Ramallo (06:52) That’s amazing. So many coding languages, right?
Takashi Kokubun (06:55) Yeah
Federico Ramallo (06:58) What was your experience learning multi, you know, every time you learn a new coding language, what was your experience?
Takashi Kokubun (07:06) So the first experience in C, like when I was making just robots work, that was just maybe just changing a little bit of example code to make different moves. then, so that was the first one. And then the second one was Java. And that was like a little bit more ergonomic than C. You can use more richer types than just older C.
And so, but I guess in the first experience in writing in C, the part-time job, I also needed to use C++ to do some, like make some geo applications for the company internally. So C++, guess, was the main thing I learned, like was kind of like the first language I learned. And then,
After that job, I moved on to joining a small company that makes iPhone applications, like games, iOS application games, mobile games, because I also was interested in making games from back when I was a kid. So I finally was able to join a company where they make a game. so I started…
writing Objective-Z because at the time, Swift wasn’t the thing and you had to write Objective-Z to make games in iPhone 4 iPhone. And Objective-Z was a sort of very awkward or verbal language, would say, like a bracket and a lot of very long names. So I didn’t like the language.
it felt like you’re writing so very long verbals, not very approachable code. And so that experience is bad, but at that same time, that was a front end, like the iPhone client, but I needed to develop a server for making a ranking for the game that I was developing. That was a puzzle game and I needed to sort of make a ranking of how users are.
doing their puzzles better than others. And for that server application, I first chose PHP as a server language because I was also like, I had some experience when I was a kid deploying PHP applications to the server. And I thought PHP is kind of like the de facto or most popular language for deploying the server applications. But when I picked a book about
That was a cake PHP book and the first page said, cake PHP is made after Ruby on Rails. like then I thought, don’t you just use Ruby? And also on Twitter back when it was called Twitter, some friends around me was also writing Ruby. And so when I was a university student, I thought Ruby was kind of cool. Like people writing Ruby.
Federico Ramallo (09:44) Hahaha
Takashi Kokubun (10:03) seemed cool at the time. There was also a very young developer who were still maybe like a junior high school student, but who became a Ruby commuter because he was so good at coding in the C to B tradition. So he was very young, but became a Ruby commuter and also he worked at the company that I joined later. so that company also looked cool. So anyways, there were a lot of…
Ruby, like cool Ruby writing people. And so I got into writing, like that made me very interested in writing Ruby. And so I guess that was the first time when, actually, so yeah, at the company, I actually wrote the server application in Ruby on Rails because of the, because I read that in the KQHB book and the Ruby language experience was much better than writing Objective-C.
It’s less verbose, short, very intuitive syntax. So I actually enjoyed writing the server side, then they actually developing the game in the client side. So that’s when I stopped making games in the programming. So the next company was the Cookpad, young Ruby committer was also there. And also I liked writing Ruby and developing the server application.
Federico Ramallo (10:55) Hehehe.
Takashi Kokubun (11:21) I joined Cookpad to develop various applications and that also made me into hacking on the open source projects because they had a lot of open source project contributors and authors. So that’s the experience that I have been going through.
Federico Ramallo (11:39) Wow, that’s amazing. mean, thank you for the author of that book that mentioned Ruby on Rails. So at what point did compilers and Ruby internals became a serious interest for you?
Takashi Kokubun (11:46) Yeah.
So when I joined Cookpad, there was some other person who were writing faster template engine for the Hamo template engine language. Hamo is just one template engine used by the Ruby community. And at that time, Hamo was the slowest template engine that like, it was like six times slower than Slim or ERB.
Slim is also under template engine language and URB is a default template engine language. And then because Cookpad was using Hamo, they were interested in speeding up and that person was speeding up the Hamo template engine. But because that seemed so cool, I started doing the same thing myself as well. while that person already made a Hamo implementation, I made another…
hammer implementation that’s also fast. And there was a bug in my implementation that needed to happen, but I forgot or didn’t know it need to be that case. like, because of missing that feature, my hammer implementation became slightly or yeah, faster than the other implementation that they were already developing. And so because of that behavior,
because it became as fast as slim because of that. My Hamo implementation got trending in the GitHub trending page, like ranked one in the Ruby language trending page. And that was when I was almost becoming a full-time worker at Cookpad. then, so Temperate Engine is kind of like a compiler.
you compile or first you parse the Ruby program, I mean, sorry, the Haml template language into some intermediate representation you want to work on and to compile that into Ruby code. Template engine generally compile the template language you want to implement into the host language you want to run it on. So like if you want to run Haml on the Ruby,
then you want to compile the template engine, like a Haml language into Ruby. And so it’s a compiler from Haml into Ruby. there you needed to, you can speed it up by having a compiler like pipeline. you, for example, let’s say you have embedded Ruby expression around like HTML attributes and inside the HTML tag.
you can, for example, create a string buffer and append static strings like bracket HTML and then append that into the string buffer. And then you embed the dynamic string embedded in the templates into the string buffer. And that’s kind of like a small compiler where you can pre-compute the combined result. For example, when you have multiple static strings,
adjacent to each other and in that case at the compile time you can combine these string little or strings into single string little and make it faster. So you can do that kind of optimization in the template language implementation and because of that you I got into like interested in optimizing
doing optimizing compilers by doing that kind of like transformation of intermediary representation like static strings, dynamic strings. That was sort of much easier than doing a general programming language representation, but it still was enough to let me get interested in doing that kind of stuff. yeah, so that was…
first into developing a tempered engine that happened to be fast. I later fixed the bug that I found to be the reason why it became faster than the other version. anyways, I successfully made a tempered engine implementation that’s as fast as other implementations like CMO ERB. And so because of that successful experience there, I started also like…
getting not only just interested in doing optimization work in the compilers, but also because I had to learn how the Ruby virtual machine work in order to speed up the template language. You’ll, for example, have to understand virtual machine instructions for concatenating strings. And if you know how Ruby speeds up the string concatenation, you have options to…
change how the Ruby code is generated for a template engine. For example, when you have like string append operator, like larger than larger than, sorry, smaller than smaller than, get that Ruby virtual machine generates a very fast instruction that doesn’t make a frame push for the method call. But if you, for example, I concatenate a string using a .concat, then
that makes a mythical because that’s now specifically optimized by the Ruby virtual machine. So because of that difference, I got kind of more familiar with how the Ruby virtual machine works. then there’s, I then joined, mean, attended the Ruby conference in Japan. At that Ruby conference, like in 2015 or so, there was a talk of…
about making Ruby faster. That was the last keynote at the conference. And that was very inspiring keynote because they, even Phoenix discussed how Ruby could be faster if you compile Ruby into the editor VM IR. Editor VM IR is a compiler infrastructure that is used by, for example, the Clang C compiler or Swift or many other native modern languages like Rust. And
LLVM is a powerful infrastructure that allows you to speed up a programming language if you just turn your programming language into their intermediary presentation. And so they gave the presentation of how Ruby could be faster by just doing that kind of thing. the idea that you can also convert the Ruby methods implemented in C into LLVM IR so you can combine
Ruby translated into A.L.V.M. and C translated into C.L.V.M. and optimize them together. And so that seemed like something we could do to speed up the Ruby language itself significantly. Like when I implemented the faster Hamo implementation, already got, did everything I could do to speed up the Hamo language into the maximum of their potential.
So if you want to make it faster than that, then you have to actually work on speaking of Ruby itself. So that’s actually what I started doing. So the next year I made a LLVM-based JIT compiler, actually that’s called LLVM-RB because LLVM Ruby. so that’s a LLVM-RB project. And I published it and sort of presented it at the next RubyKaigi. And as an agent talk,
And in the same year, there was another talk that also was giving a presentation about the G compiler called MJIT. So MJIT was a G compiler that is developed with using the C compiler as the backend for making native code. So usually,
usual programming languages don’t use C compiler when it’s running for making the machine code, but it’s a unique design. The MgT unique design is when you call out a Ruby method, you just spawn a C compiler process and let it compile Ruby code into C code. then, sorry, the G compiler rewrites this Ruby code into C code and then…
the C code is then fed into the C compiler and C compiler generates the machine code and the MJIT dynamically loads the binary created by the C code and C compiler and loads it at runtime and run it. So that’s how MJIT worked and the author of the Ruby programming language, Matt liked the idea better than integrating LLVM into the C to B implementation. So…
that’s what they adapted in the first JIT compiler in the Ruby 2.6. Yeah, what was the question? Yeah, so yeah, that’s how I guess how I got into optimizing the compilers. so yeah.
Federico Ramallo (20:42) Interesting. I didn’t know that it was because you started with Hamel, ⁓ improving Hamel. I’ve been studying more the Nokogiri gem, how it works. It gets an XML or JSON or whatever, or HTML, and it of split up into multiple nodes, nested nodes. Is that something similar what you do with Hamel and how you interpret the nested elements?
Takashi Kokubun (20:46) Mm-hmm.
kind of, definitely you have to parse the kind of like a node tree, though the difference is for Nokogiri, the input is going to be the HTML, but for Hamo, HTML is the output, so you’re going to parse the Hamo syntax and get the HTML-like structure tree, but I guess at that time, it’s going to be the same as dealing with the Nokogiri nodes, because it’s just a bunch of Ruby objects, so.
Yeah, kind of like the same.
Federico Ramallo (21:34) Right. And did you implement lookahead to kind of optimize for what is going to come in the next instructions?
Takashi Kokubun (21:43) for Hamel.
Federico Ramallo (21:45) for a new HAML implementation.
Takashi Kokubun (21:45) Yeah, yeah,
yes. So current hammer invitation has a lot of optimization that takes a bunch of, like a sequence of their specific, like special instructions, like concatenate this static string or concatenate this dynamic Ruby expression. And when you have like multiple static strings, they combine that.
intermediative position into a single intermediative position that concatenates a single aesthetic string. So that kind of thing happens. And there’s also another interesting optimization where when the embedded expression is a string little that also has a string interpolation like a sharp brace. And then in that case, how mutation can break up the string little into intermediative position of the
static string and the dynamic Ruby expression and the static string so that these static strings can be combined into the preceding static string and the succeeding static string and therefore you can reduce the number of string allocations. that’s kind of, yeah, I think that’s something that uses a look ahead of like, optimizer pipeline.
Federico Ramallo (22:55) Interesting, interesting. So basically, you split up the strings into steady strings, and then you just do the dynamic part.
Takashi Kokubun (23:05) Mm-hmm, yes.
Federico Ramallo (23:09) What has been the benefit of using just-in-time JIT instead of C bindings? Do you have a benefit in performance or is it about that you can’t, or are there other benefits?
Takashi Kokubun (23:28) think it’s a good question. So if you just want the performance, you would write, for example, in different languages, like you can create a C extension even for Ruby to just make it fast. But I think the point is you want to, you are using Ruby because you want to write Ruby and make it more productive, make you more productive by writing Ruby. And so if you just want the performance,
you write Ruby, I mean, sorry, you write C, but what G compiler allows you is just, you can just write Ruby and make it fast by just turning on the G compiler. So you can avoid losing the productivity by switching the language from Ruby to C or anything that supports supported by the extensions. So yeah, I think there’s, we often attempt to make optimizations that
allow Ruby developers to not care about how it’s implemented. Like I previously shared that the string concatenation can be faster if you use the smaller than operator. But if you have a G compiler that supports speeding up the other version, like the .concat, then you don’t need to worry about what is fast in this particular Ruby imitation. And so I want the G compiler to allow people to…
not worry about it and just be more productive by just writing whatever Ruby code they want to write. And so that’s the vision, I think, the speeding up the language itself rather than asking people to write C.
Federico Ramallo (24:56) Right, because with C, can only implement, I’ve seen part of the language implemented. If you use this method, it’s fast. If you use this other method, it’s 10 times slower or whatever. ⁓ So it’s kind of like all in or nothing. So with JIT, you can basically optimize across the board all the methods and then build up from there.
Takashi Kokubun (25:07) Yeah.
Hmm.
Hmm. Yeah.
Federico Ramallo (25:22) Interesting. And what parallels have you found from the Hamel optimization and the JIT compiler? What lessons were you able to translate from one experience to the other?
Takashi Kokubun (25:34) I think the essentially for me, they look like the same thing. Like they are vastly different, in the actual like structure. But, I think the idea is the same. You have a input language, you convert that into whatever intermediary repetition you’re comfortable or transforming. And then you try to convert that into.
Sorry, you first optimize that intermediate repetition you created for that particular implementation. And then after optimizing those intermediate repetition, you convert that into the output language. So in Hamo, you take a Hamo and parse that into the tree of the Hamo nodes. And then that tree is then converted into a temporary engine language.
Temporate engine intermediate repetition. There’s a gem code, Ruby gem code, tempo.gem and tempo.gem provides an intermediate repetition that’s useful for optimizing temperate engines. so we use a tempo IR intermediate repetition as the output of the hammer tree. And then we then let the tempo gem generate the HTML string or sorry, the Ruby code to render the HTML string. And so the output is a Ruby code.
And so, sort of like the input is Haml, the tempo IR, and then optimize the IR, and then the Ruby code. What happens in the G compiler is, for example, in ZJ or YJ, you take the Ruby and Ruby code is then parsed into the Ruby tree, like abstract syntax tree, and then the tree is then internally compiled into their integrated representation called bytecode.
Ruby Virtual Machine has its own original bytecode, Ruby Virtual Machine bytecode. And for ZGIT, the bytecode is input. We translate the bytecode into ZGIT’s own IR, and then it does the optimization on the IR and then output the machine code. So it’s the bytecode is input, then compare it into the IR, optimize the IR, and then output the machine code. So that way is kind of similar essentially to me.
So yeah, think the similarities between those projects.
Federico Ramallo (27:47) Right, right.
I mean, I remember when we used to cache more, which is kind of a workaround when you have slow processes. Caching is better than processing, right? So I’m wondering if at one point optimization, when you increase the speed, then it doesn’t make sense to cache anymore and deal with all the trade-off of caching.
Takashi Kokubun (28:11) So I think it kind of depends on what caching you are going to do. like, let’s say you have a method that does caching, like memorize this instance variable by like bar bar equal. When it’s not defined, you do this thing and then if it’s already been done for the object, you just reuse that result. So that kind of thing, when the body is like IO to the dev.
database then no matter how you optimize the Ruby language, it’s not gonna be faster because it’s just done by under process. So the caching may be still useful there, but when it comes to the Ruby, like speaking of the language itself, I think the compiler could cast some result when the same thing or same code is repeated over and over and if the compiler could figure out the result is going to be the same in those different places then,
compiler can take out those repeated duplicates and just execute it once and view the result in multiple places. So I think that way the compiler could do the caching for you automatically and you can stop worrying about caching yourself. So I guess those two points are why I would race for this discussion.
Federico Ramallo (29:22) Right, right.
How much of a different approach are you doing between development and production? mean, with production, can pre-compile more or go further ahead. we also want on development the benefit of interpreted languages that you don’t need to compile or pre-compile or spend time waiting for the computer to process the code, right? How are you being able to manage those trade-offs?
Takashi Kokubun (29:49) That’s a great question. We are thinking of supporting a feature to pre-compile some of the Ruby programs into machine code before they put it into production because today at Shopify, sometimes it’s been a problem to just deploy the G-compiler in production and just turn it on because compiling also takes some computation resource.
So if you just boot application and start a JIT immediately and run all the methods, I mean, compile all the methods then, sometimes when the application is large enough, the workload spent on the JIT compilation is so large that that can be very slow at boot. And so we kind of tune the parameters to make it less bad in the warm up performance.
And so we’ve been able to sort of deal with that by just doing the operational approach there. But ideally, when something can be computed beforehand, we should be able to do that and then just deploy that to compile code to the production so that we don’t need to deal with the compilation overhead itself. However, there’s a reason why we are doing it that way at the moment. Like if you…
have to pre-compile something and need to deal with the deployment, then it’s gonna be harder for you to deploy Ruby. So we want to make it easy to keep, like keep it easy to deploy Ruby to production. Like at the moment, you just need to turn on a flag and then the decompiler will just work because it’s gonna just…
compile just in time, not something, like not take something that’s compiled beforehand. So you don’t need to worry about pre-compiling something and just copying it to production. But we do want to support that because sometimes in production that matters and it is indeed possible in some cases, like when some Ruby code is very un-arisable and like statically,
decide that if the behavior is very statically decided, then you don’t need to spend time in production, analyze the type types in production, specifically optimize that coding to the runtime behavior. So in that case, like we have like one workload where everything is very known by just parsing the Ruby code. And in that case, we want to ideally support pre-compiling that and just loading the code in the production.
And the other case is where the behavior is very dynamic and you don’t know types until you just run the providers on the production application. So yeah, that’s how we categorize these two things. And in the future, in the future ZG versions, we are thinking of adding such a feature to pre-compile Ruby or caching the past results like in the server when the G compiler generates a code.
we could cache the code when the same assumption holds in the next run. And if we can safely verify that it’s safe to reuse the code passcode, then we could have a cache directory where you can just dump the G code into there and reuse whenever it’s possible to reuse. So that’s how we are thinking about that thing. Like it’s a trade off of making it harder to deploy versus making it.
faster to deploy that into production, especially in the warm up.
Federico Ramallo (33:10) Right, right. Yeah, I mean, that has been my experience with Java apps that are droplets. I don’t remember the name, but you load them up, they take forever to load. It takes a lot of memory to load because it loads everything in memory. And then the first request takes forever and the second one is faster and the third one is faster, And it’s kind of frustrating when you’re, you know, deploying an application and the deployment time takes forever.
Takashi Kokubun (33:27) Mm.
Federico Ramallo (33:36) compared to my experience with Ruby, where the deployment is much, much faster. So I can see a benefit of pre-compilation, but the trade-off is longer deployment time, or more complexity.
Takashi Kokubun (33:49) To be clear though, YJ, which is the current production ready to compiler for Ruby 4.0 is designed to be fast in the completion. So unless you are like super big like Shopify, you wouldn’t like experience the slow warmup because the compression is just very fast compared to regular just any compilers that do more transition than like the Java. like, yeah, but.
we want to add more positions and when that happens that might take some time. So when that actually happens, we might want to do that kind of approach so that we can add more stuff without slowing down the performance.
Federico Ramallo (34:29) Interesting, What are your goals with ZJIT? What do you want to improve compared to YJIT?
Takashi Kokubun (34:38) So the reason why we started ZGIT is because due to the nature of the YG design, which is to do fast compilation, like that is also good at a warm up performance, but that architecture is something like this. So you take a block, like when you have an if statement, you cut that into the blocks of branches. then YG would take one block at a time and compile.
instructions in a single block using the knowledge inside the block local information and just compile one block at a time and move on to our next broken compiler, broken one at a time. So that limits you from making cross-blocking with these optimizations. Like when you have large method that has a lot of blocks, you could ideally optimize using cross-blocking information.
when you have a variable defined in a different block and you use that in a different block, then you want to leverage the type information or what computation is performed on the different block. But if you only compile a one block at a time, then you cannot optimize across different blocks. So ZG on the other hand, takes a whole method and compiles that the multiple blocks in all the blocks in the method into
like at the same time. So you can do cross-block optimization because of that design. And you can also, we are thinking of adding that inlining that allows you to even do the cross-method optimization. So that way we can unblock more optimization opportunities. Like let’s say you have this method that creates a hash object with this string key, but when the key and the hash object is like.
in a different block or methods, then unless you combine these two places and optimize them together, you cannot pre-compute the hash of the string key, for example, to speed up the hash lookup. So these are the kind of organizations that we want to unblock in ZGIT. So this is one thing we want to do in ZGIT, which is unblock more optimization that YGIT.
couldn’t do or things that were hard in YGIT. The other thing we are working on today in ZGIT is so-called library frames authentication idea. that is in C2B, the different simulation that you would be probably using in production for many people. C2B spends a lot of computers, CPU cycles in…
making a call because Ruby is a very dynamic language that requires the language to have more, so many method like thing fields in the metadata of the frame. Whenever you make a method call in Ruby, you write the program counter, stack pointer and frame flags and environment pointer. And those, so many things need to be written whenever you just make one method call, but
with a compiler, G compiler, we want to make it fast. In YG, we experimented with making it faster by this idea called Rider frames, something similar to this Rider frames idea, but we spent like two months experimenting with that and we saw a performance regression in the interpreter because of, to allow the G compiler transition, we needed to do some modification in the interpreter.
And that made a regression in the interpreter. So we kind of gave up the idea. But in ZG, because we have a different architecture in terms of how we hook the G code, we just use a different machine code for making the G2G like meta codes. And so that kind of made it easier for us to land this meta code traditional idea. So the library from this idea is to
At the compile time, we make a pointer, I mean, sorry, the struct allocated on the heap that has the metadata like frame flags and the environment pointer offset or the stack pointer offset or the location of the objects that you need to sort of remember when zc garbage collector or the sum of traditional or something unexpected happens. So those kind of metadata is
pre-computed at the compile time and then we write the pointer to that metadata, just a single pointer on the method push and all other fields that are stored in the metadata are going to be not written into the frame as of the execution. And so it’s going to be from a bunch of writes into the frames to just one frame pointer write. And so,
it’s gonna be faster, like every method call is going to be faster because a bunch of writes are going to be optimizing to just one pointer write. And what allows it to be just one pointer write is just lazy load the frame metadata from this single pointer. So we need to modify the interpreter to read that single pointer and retrieve that out of the just in kind of compiler frames. So.
That’s what we are working on in that ZG today. Once we successfully implement it and if we successfully make it not slower in the interpreter, like if we don’t introduce that regression in the interpreter, we can merge it and make every method go faster. And because we didn’t make this part faster in YJ, if we land this feature in ZJ, it’s going to be much faster than YJ.
That’s what we are trying to make.
Federico Ramallo (40:29) Interesting. Do think that’s going to require more memory to…
Takashi Kokubun (40:33) It’s a good question. I think the frame metadata is going to need some memory. And so it’s going to consume more memory, but it’s not going to consume memory at every method code because at compile time, only one struct is going to be allocated for each call site. And when the single method is called a lot by multiple different places, they’re going to share the same pointer. And so it’s not gonna be crazy.
YJ also uses metadata, allocates metadata and that’s also shared across different calls. But I think as of now, our estimate is that the memory usage by the ZJ is not gonna be much worse than YJ, and probably because we don’t generate a lot of different versions for the same method implementation. We do like optimization.
like versioning. like in YGIT, we do versioning of the same block over time and we compile multiple versions so that we can optimize the block into the specific execution context. Like when this method takes this type, then use this optimized block. And when this takes another type, then use this block. And so we kind of do different optimization techniques. so…
there’s a trade off of speeding this program up versus how much memory we use. And so we are gonna be able to use like, it has many knobs of using this memory versus speeding this up. And so we’re gonna deploy this to production and tune this, make it reasonable memory consumption at least for Shopify. And so because Shopify’s application is very big, I think it’s gonna be good enough for.
many other applications as well. So that’s our strategy at the moment.
Federico Ramallo (42:22) So it’s going to be… the memory is going to increase based on the complexity of the application, but the payoff is going to be much better performance and lower memory consumption of the application because you can reuse the intermediate language for multiple methods.
Takashi Kokubun (42:37) Sure, and also we are probably going to provide parameters that allow you to trade these. For example, we already have the YGExecMem option or YGMemSize and ZGMemSize that allow you to just decrease the size of a memory you use for the compiler. So let’s say you specify just the 128, which is a default.
megabytes, megabytes, YG or ZG will not consume more memory than that for that process. And you can just lower that to, for example, 64 or 32. And that has been working very well for companies like GitHub. And so I think even if you, even if ZG started using more memory for each method, you can still limit it to a certain threshold and that will still allow you to.
straight memory size versus speed and you can find the right parameter that works best for your particular application by just changing that.
Federico Ramallo (43:44) Amazing,
I’m wondering what has been your experience moving from Japan to the United States and what are the advice would you give engineers that are thinking to make the same move?
Takashi Kokubun (43:54) Sure. So one of the reasons why I made this move was back when I was in Japan, I attended some conferences where some people working abroad or working in the United States were there and they said they have very high salary. And they, so in Japan, the software engineers in general are
paid less, much less than the people in the super developers in the United States, especially the people are in the South San Francisco Bay Area or New York. So one, you get paid a lot if you just move to the Silicon Valley and two, you have more like interesting opportunities or openings because they pay a lot there, good talents come to that place and
that the companies who need those talents will also move to that place. And so in Silicon Valley or San Francisco, you get a lot of interesting opportunities or bigger big tech companies there. And big tech also has opportunities like compiler developers because unless you’re so big that you need to speed up the language by themselves, you don’t get to spend money on speeding up the language. And so those big companies,
tend to have compiler questions and because I got the interest in compilers, that was one of the motivations that why I wanted to move to the US and I indeed have the compiler job now and so if I wanted to switch like positions, then there are plenty of opportunities in this. I live in Singapore and Bali and the opportunities in there. So yeah, that’s one.
experience that that has been the case so far. Like after I got interested in moving to the United States for that reason, I met some like
engineers who work in Silicon Valley at the conference. like the second company I joined after Cookpad was the Treasuredata and their headquarter is in the United States, but I joined their Japan branch, the subsidiary in Tokyo. I first joined the Tokyo, like their company, yeah, Tokyo port. And then when I was in Tokyo, I sometimes had opportunities to…
go to Silicon Valley and join events or conferences there. And that inspired me to like, they had some like good, like famous engineers. were, they were who were working there at Silicon Valley. And I was able to meet those people at these sites. And so I thought if we move to the United States or move to Silicon Valley, then there are a lot of such opportunities to meet great talents and people there.
So yeah, that was one motivation. And I was able to join some meetups sometimes to meet people there. We both went to SF Ruby, right? And so that kind of things also happened there too in these days too. yeah, that’s one experience. Other thing is like, guess, so the cultural difference between Japan and the United States are actually there. Like, I guess the…
medical expenses kind of much more expensive in States. Like in Japan, almost everything is free. When you go to dentist, it’s always going to be free, but here it’s always going to cost some money. I’ve been paying a lot to just fix my, yeah, can’t, yeah. So yeah, just teeth in general. another thing I guess is,
food is much cheaper in Japan. It’s kind of like the cheapest thing, but when you eat ramen, for example, in the United States, it’s gonna be triple times more expensive than in Japan. And well, you still pay a lot, so I can afford eating three times more expensive ramen, but it’s still gonna be expensive. But the rent is also…
very crazy, I finally purchased a condo in the United States and I don’t need to pay rents, but that was still very expensive when I was renting. so I guess that was a challenge. And yeah, the medical expenses and rent, very expensive rents and I mean, especially for the Silicon Valley, I guess.
Federico Ramallo (48:16) Takashi, thank you very much for joining us today. I truly appreciate it. Any final remarks before we wrap it up?
Takashi Kokubun (48:23) Well, I guess we are making ZG fast and it’s gonna be the next version is going to be released this year, December 25th, 2026. So stay tuned. We are hoping to make it faster than YG this time. yeah, I’m hoping that it’s gonna be production ready this year. yeah, stay tuned. Thank you.
Federico Ramallo (48:45) Amazing, I’m looking forward to see it. Thank you.
Takashi Kokubun (48:48) Thank you.