Monkey Patch Detection in Ruby
My last post detailed one way that CRuby will eliminate some intermediate array allocations when using methods like Array#hash and Array#max. Part of the technique hinges on detecting when someone monkey patches array. Today, I thought we’d dive a little bit in to how CRuby detects and de-optimizes itself when these “important” methods get monkey patched. Monkey Patching Problem The optimization in the previous post made the assumption that the implementation Array#max was the original definition (as defined in Ruby...
6 days ago
Eliminating Intermediate Array Allocations
Recently I gave a talk at RailsWorld (hopefully they’ll post the video soon), and part of my presentation was about eliminating allocations in tokenizers. I presented a simple function for measuring allocations: def allocations x = GC.stat(:total_allocated_objects) yield GC.stat(:total_allocated_objects) - x end Everything in Ruby is an object, but not all objects actually make allocations. We can use the above function to measure allocations made in a block. Here are some examples of code that never allocate: p allocations {...
23 days ago
Using Serial Ports with Ruby
Lets mess around with serial ports today! I love doing hardware hacking, and dealing with serial ports is a common thing you have to do when working with embedded systems. Of course I want to do everything with Ruby, and I had found Ruby serial port libraries to be either lacking, or too complex, so I decided to write my own. I feel like I’ve not done a good enough job promoting the library, so today we’re going to mess...
8 months ago
Using Serial Ports with Ruby
Lets mess around with serial ports today! I love doing hardware hacking, and dealing with serial ports is a common thing you have to do when working with embedded systems. Of course I want to do everything with Ruby, and I had found Ruby serial port libraries to be either lacking, or too complex, so I decided to write my own. I feel like I’ve not done a good enough job promoting the library, so today we’re going to mess...
8 months ago
Fast Tokenizers with StringScanner
Lately I’ve been messing around with writing a GraphQL parser called TinyGQL. I wanted to see how fast I could make a GraphQL parser without writing any C extensions. I think I did pretty well, but I’ve learned some tricks for speeding up parsers and I want to share them. Today we’re going to specifically look at the lexing part of parsing. Lexing is just breaking down an input string in to a series of tokens. It’s the parser’s job...
about 1 year ago
Fast Tokenizers with StringScanner
Lately I’ve been messing around with writing a GraphQL parser called TinyGQL. I wanted to see how fast I could make a GraphQL parser without writing any C extensions. I think I did pretty well, but I’ve learned some tricks for speeding up parsers and I want to share them. Today we’re going to specifically look at the lexing part of parsing. Lexing is just breaking down an input string in to a series of tokens. It’s the parser’s job...
about 1 year ago
Fast Tokenizers with StringScanner
Lately I’ve been messing around with writing a GraphQL parser called TinyGQL. I wanted to see how fast I could make a GraphQL parser without writing any C extensions. I think I did pretty well, but I’ve learned some tricks for speeding up parsers and I want to share them. Today we’re going to specifically look at the lexing part of parsing. Lexing is just breaking down an input string in to a series of tokens. It’s the parser’s job...
about 1 year ago
Bitmap Matrix and Undirected Graphs in Ruby
I’ve been working my way through Engineering a Compiler. I really enjoy the book, but one part has you build an interference graph for doing register allocation via graph coloring. An interference graph is an undirected graph, and one way you can represent an undirected graph is with a bitmap matrix. A bitmap matrix is just a matrix but the values in the matrix can only be 1 or 0. If every node in your graph maps to an index,...
over 1 year ago
Bitmap Matrix and Undirected Graphs in Ruby
I’ve been working my way through Engineering a Compiler. I really enjoy the book, but one part has you build an interference graph for doing register allocation via graph coloring. An interference graph is an undirected graph, and one way you can represent an undirected graph is with a bitmap matrix. A bitmap matrix is just a matrix but the values in the matrix can only be 1 or 0. If every node in your graph maps to an index,...
over 1 year ago
Vim, tmux, and Fish
I do most of my text editing with MacVim, but when I pair with people I like to use tmate. tmate is just an easy way to connect tmux sessions with a remote person. But this means that I go from coding in a GUI to coding in a terminal. Normally this wouldn’t be a problem, but I had made a Fish alias that would open the MacVim GUI every time I typed vim in the terminal. Of course when...
almost 2 years ago
Vim, tmux, and Fish
I do most of my text editing with MacVim, but when I pair with people I like to use tmate. tmate is just an easy way to connect tmux sessions with a remote person. But this means that I go from coding in a GUI to coding in a terminal. Normally this wouldn’t be a problem, but I had made a Fish alias that would open the MacVim GUI every time I typed vim in the terminal. Of course when...
almost 2 years ago
In Memory of a Giant
The Ruby community has lost a giant. As a programmer, I always feel as if I’m standing on the shoulders of giants. Chris Seaton was one of those giants. I’ve been working at the same company as Chris for the past 2 years. However, I first met him through the open source world many years ago. He was working on a Ruby implementation called TruffleRuby, and got his PhD in Ruby. Can you believe that? A PhD in Ruby? I’d...
almost 2 years ago
In Memory of a Giant
The Ruby community has lost a giant. As a programmer, I always feel as if I’m standing on the shoulders of giants. Chris Seaton was one of those giants. I’ve been working at the same company as Chris for the past 2 years. However, I first met him through the open source world many years ago. He was working on a Ruby implementation called TruffleRuby, and got his PhD in Ruby. Can you believe that? A PhD in Ruby? I’d...
almost 2 years ago
Cross Platform Machine Code
I hate writing if statements. I’ve been working on a couple different assemblers for Ruby. Fisk is a pure Ruby x86 assembler. You can use it to generate bytes that can be executed on x86 machines. AArch64 is a pure Ruby ARM64 assembler. You can use it to generate bytes that can be executed on ARM64 machines. Both of these libraries just generate bytes that can be interpreted by their respective processors. Unfortunately you can’t just generate bytes and expect...
over 2 years ago
Cross Platform Machine Code
I hate writing if statements. I’ve been working on a couple different assemblers for Ruby. Fisk is a pure Ruby x86 assembler. You can use it to generate bytes that can be executed on x86 machines. AArch64 is a pure Ruby ARM64 assembler. You can use it to generate bytes that can be executed on ARM64 machines. Both of these libraries just generate bytes that can be interpreted by their respective processors. Unfortunately you can’t just generate bytes and expect...
over 2 years ago
Homebrew, Rosetta, and Ruby
Hi everyone! I finally upgraded to an M1. It’s really really great, but the main problem is that some projects I work on like TenderJIT and YJIT only really work on x86_64 and these new M1 machines use ARM chips. Fortunately we can run x86_64 software via Rosetta, so we can still do development work on x86 specific software. I’ve seen some solutions for setting up a dev environment that uses Rosetta, but I’d like to share what I did....
almost 3 years ago
Homebrew, Rosetta, and Ruby
Hi everyone! I finally upgraded to an M1. It’s really really great, but the main problem is that some projects I work on like TenderJIT and YJIT only really work on x86_64 and these new M1 machines use ARM chips. Fortunately we can run x86_64 software via Rosetta, so we can still do development work on x86 specific software. I’ve seen some solutions for setting up a dev environment that uses Rosetta, but I’d like to share what I did....
almost 3 years ago
Publishing Gems With Your YubiKey
The recent compromise of ua-parser-js has put the security and trust of published packages at the top of my mind lately. In order to mitigate the risk of any Ruby Gems I manage from being hijacked, I enabled 2FA on my RubyGems.org account. This means that whenever I publish a Ruby Gem, I have to enter a one time passcode. I have to admit, I find this to be a pain. Whenever I do a release of Rails, I have...
almost 3 years ago
Publishing Gems With Your YubiKey
The recent compromise of ua-parser-js has put the security and trust of published packages at the top of my mind lately. In order to mitigate the risk of any Ruby Gems I manage from being hijacked, I enabled 2FA on my RubyGems.org account. This means that whenever I publish a Ruby Gem, I have to enter a one time passcode. I have to admit, I find this to be a pain. Whenever I do a release of Rails, I have...
almost 3 years ago
Debugging an Assertion Error in Ruby
I hope nobody runs in to a problem where they need the information in this post, but in case you do, I hope this post is helpful. (I’m talking to you, future Aaron! lol) I committed a patch to Ruby that caused the tests to start failing. This was the patch: commit 1be84e53d76cff30ae371f0b397336dee934499d Author: Aaron Patterson <tenderlove@ruby-lang.org> Date: Mon Feb 1 10:42:13 2021 -0800 Don't pin `val` passed in to `rb_define_const`. The caller should be responsible for holding a pinned...
over 3 years ago
Debugging an Assertion Error in Ruby
I hope nobody runs in to a problem where they need the information in this post, but in case you do, I hope this post is helpful. (I’m talking to you, future Aaron! lol) I committed a patch to Ruby that caused the tests to start failing. This was the patch: commit 1be84e53d76cff30ae371f0b397336dee934499d Author: Aaron Patterson <tenderlove@ruby-lang.org> Date: Mon Feb 1 10:42:13 2021 -0800 Don't pin `val` passed in to `rb_define_const`. The caller should be responsible for holding a pinned...
over 3 years ago
Counting Write Barrier Unprotected Objects
This is just a quick post mostly as a note to myself (because I forget the jq commands). Ruby objects that are not protected with a write barrier must be examined on every minor GC. That means that any objects in your system that live for a long time and don’t have write barrier protection will cause unnecessary overhead on every minor collection. Heap dumps will tell you which objects have a write barrier. In Rails apps I use a...
about 4 years ago
Counting Write Barrier Unprotected Objects
This is just a quick post mostly as a note to myself (because I forget the jq commands). Ruby objects that are not protected with a write barrier must be examined on every minor GC. That means that any objects in your system that live for a long time and don’t have write barrier protection will cause unnecessary overhead on every minor collection. Heap dumps will tell you which objects have a write barrier. In Rails apps I use a...
about 4 years ago
Guide to String Encoding in Ruby
Encoding issues don’t seem to happen frequently, but that is a blessing and a curse. It’s great not to fix them very frequently, but when you do need to fix them, lack of experience can leave you feeling lost. This post is meant to be a sort of guide about what to do when you encounter different types of encoding errors in Ruby. First we’ll cover what an encoding object is, then we’ll look at common encoding exceptions and how...
almost 5 years ago
Guide to String Encoding in Ruby
Encoding issues don’t seem to happen frequently, but that is a blessing and a curse. It’s great not to fix them very frequently, but when you do need to fix them, lack of experience can leave you feeling lost. This post is meant to be a sort of guide about what to do when you encounter different types of encoding errors in Ruby. First we’ll cover what an encoding object is, then we’ll look at common encoding exceptions and how...
almost 5 years ago
My Career Goals
I was going to tweet about this, but then I thought I’d have to make a bunch of tweets, and writing a blurgh post just seemed easier. Plus I don’t really have any puns in this post, so I can’t tweet it! My Career Goals I think many people aren’t sure what they want to do in their career. When I first started programming, I wasn’t sure what I wanted to do with my career. But after years of experience,...
about 5 years ago
My Career Goals
I was going to tweet about this, but then I thought I’d have to make a bunch of tweets, and writing a blurgh post just seemed easier. Plus I don’t really have any puns in this post, so I can’t tweet it! My Career Goals I think many people aren’t sure what they want to do in their career. When I first started programming, I wasn’t sure what I wanted to do with my career. But after years of experience,...
about 5 years ago
ESP8266 and Plantower Particle Sensor
Since forest fires have started to become a normal thing in the PNW, I’ve gotten interested in monitoring the air quality in and around my house. I found some sensors that will measure PM2.5 which is a standard for measuring air quality. The sensor I’m using is a PMS5003, and you can see the data sheet for it here. I like this sensor because it supports UART, so I was able to hook it to an FTDI and read data...
about 5 years ago
ESP8266 and Plantower Particle Sensor
Since forest fires have started to become a normal thing in the PNW, I’ve gotten interested in monitoring the air quality in and around my house. I found some sensors that will measure PM2.5 which is a standard for measuring air quality. The sensor I’m using is a PMS5003, and you can see the data sheet for it here. I like this sensor because it supports UART, so I was able to hook it to an FTDI and read data...
about 5 years ago
Instance Variable Performance
Let’s start today’s post with a weird Ruby benchmark: require "benchmark/ips" class Foo def initialize forward forward ? go_forward : go_backward end ivars = ("a".."zz").map { |name| "@#{name} = 5" } # define the go_forward method eval "def go_forward; #{ivars.join("; ")} end" # define the go_backward method eval "def go_backward; #{ivars.reverse.join("; ")} end" end # Heat Foo.new true Foo.new false Benchmark.ips do |x| x.report("backward") { 5000.times { Foo.new false } } x.report("forward") { 5000.times { Foo.new true } } end...
over 5 years ago
Instance Variable Performance
Let’s start today’s post with a weird Ruby benchmark: require "benchmark/ips" class Foo def initialize forward forward ? go_forward : go_backward end ivars = ("a".."zz").map { |name| "@#{name} = 5" } # define the go_forward method eval "def go_forward; #{ivars.join("; ")} end" # define the go_backward method eval "def go_backward; #{ivars.reverse.join("; ")} end" end # Heat Foo.new true Foo.new false Benchmark.ips do |x| x.report("backward") { 5000.times { Foo.new false } } x.report("forward") { 5000.times { Foo.new true } } end...
over 5 years ago
Speeding up Ruby with Shared Strings
It’s not often I am able to write a patch that not only reduces memory usage, but increases speed as well. Usually I find myself trading memory for speed, so it’s a real treat when I can improve both in one patch. Today I want to talk about the patch I submitted to Ruby in this ticket. It decreases “after boot” memory usage of a Rails application by 4% and speeds up require by about 35%. When I was writing...
over 6 years ago
Speeding up Ruby with Shared Strings
It’s not often I am able to write a patch that not only reduces memory usage, but increases speed as well. Usually I find myself trading memory for speed, so it’s a real treat when I can improve both in one patch. Today I want to talk about the patch I submitted to Ruby in this ticket. It decreases “after boot” memory usage of a Rails application by 4% and speeds up require by about 35%. When I was writing...
over 6 years ago
Reducing Memory Usage in Ruby
I’ve been working on building a compacting garbage collector in Ruby for a while now, and one of the biggest hurdles for implementing a compacting GC is updating references. For example, if Object A points to Object B, but the compacting GC moves Object B, how do we make sure that Object A points to the new location? Solving this problem has been fairly straight forward for most objects. Ruby’s garbage collector knows about the internals of most Ruby Objects,...
over 6 years ago
Visualizing Your Ruby Heap
In a previous post, I wrote a bit about how Ruby objects are laid out in memory. Today we’ll use that information to write a program that will allow us to take a Ruby heap dump and visualize the layout and fragmentation of that heap. Ruby Object Layout Recap Just as a recap, Ruby objects are fixed width. That is, every Ruby object is the same size: 40 bytes. Objects are not really allocated with malloc, but instead they are...
about 7 years ago