This story starts at the console debugging a coworker’s issue. They had gotten stuck trying to
run bundle install
. Try, and fail. Try, and fail. And the error was really weird:
1
|
|
We went through the normal steps: turn it off and on again, check wifi, load up google, go for a coffee , etc. Standard debugging techniques.
Finally we realized the y
had inadvertently been dropped in source 'https://rubgems.org'
.
Huh. Go figure. We moved on our way after giggling a little. Rub gems. Hehe.
Then I got to thinking. If my colleague had made this mistake, surely other people have made this error too. I checked the whois record, and it was available! take_my_money
I was now in possession of rubgems.org
, and was left with the question: What can I do with
it? Which lead me to the logical conclusion: I wonder if I can Man in the Middle rubygems.org
and see
if other people make this typo!
It was stupid easy to setup a MITM. Grab a tiny AWS box, configure DNS, setup an Nginx proxy, and add let’s encrypt to the box. The Nginx config looked like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
|
And that was it. I cobbled together some commands to build a simple log file parser to send me a list of unique IPs each day via email, and then I sat back and waited.
What Does a RubyGems MITM Get You?
While I waited, I investigated and tried to determine: What is possible if you’ve got control of a rubygem?
To answer this question I created a trojan_horse
gem. You can see it here on rubygems.org
:
https://rubygems.org/gems/trojan_horse.
The rough contents of which were:
1 2 3 4 5 |
|
Then I made a modification of the gem and added my MITM code at the top:
1 2 3 4 5 6 7 |
|
I put this gem onto my MITM’d server, and tried to download it through various means.
RubyGems
Let’s see what happens with RubyGems. I started looking at gem install
and set the --source
flag,
here’s what got downloaded:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
|
Huh interesting. It reaches out to rubygems.org
and then rubgems.org
, but the download prefers to use
rubygems.org
over my MITM. Wonder why? Checking out the help docs solves that problem:
1 2 3 |
|
From this it looks like the url is appended to the end such that if the gem doesn’t exist on rubygems.org
only then will it reach out to a different source. No MITM possibility there!
Fresh Installation with Bundler
Next I tried my hand at bundler. I started by creating the following Gemfile entry:
1 2 |
|
And received:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
|
As you can see from the curl
output, having never downloaded the gem before, I have RCE through the gem.
Reinstall with Bundler
With the bundler reinstall, I began by uninstalling trojan_horse
, and updating my Gemfile
to the following:
1 2 |
|
And then ran bundle install
:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
|
As you can see the gem downloaded cleanly from rubygems.org
, without a trojan. Next I updated the
Gemfile
back to rubgems
and ran bundle install --verbose
:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
|
And herein I got caught! Bundler does a checksum comparison on the gem verses the server. But if I control
the server, shouldn’t I be able to control the checksum? I checked what /info/trojan_horse
returned:
1 2 3 4 5 6 7 |
|
Those are different, but they don’t match the checksum failure I was getting. Things actually got confusing at this point. And when that happens, I jump into the source code and try to get a debugger around what’s happening.
Grepping for “does not match the checksum” on github lead me to MisMatchedChecksumError. And finally down to line 70.
1
|
|
At this point with a debugger around the code, I can start to look at the two variable values:
local_temp_path
is a path to a local cache of the/info/trojan_horse
file; on my local that path looks like this:
1 2 3 |
|
response_etag
is the ETag value from the request sent to rubgems for/info/trojan_horse
. That value came back as:
1 2 |
|
First mystery solved I found one of the checksums. It’s a partial MD5 hash coming from ETag confused-cat
1 2 |
|
I didn’t update this, because I really think the confused-cat meme is hilarious and wanted to use the emoji.
Still digging, I followed the method etag_for
and ended up at this code:
1 2 3 4 5 6 7 8 9 10 11 |
|
That makes more sense! Bundler is doing an MD5 hash against the contents of the file returned at
/info/trojan_horse
. The MD5 hash for that file looks like this:
1 2 3 4 |
|
And this is the second piece. The ETag is not matching with what I have on the local cache and that’s why the installation is failing. Makes sense. But I control the server, so I can modify the ETag! All that’s expected from the bundler standpoint is an MD5 checksum that matches the contents of the file.
I switched up my Nginx configuration so that when is requested /info/trojan_horse
it reverse proxies to
a local ruby server and sends back the MD5 hash it was expecting.
And viola! The backdoored gem installed. I was actually shocked at this point. I expected additional safety checks to break, and that wasn’t the case. The lesson here is if you own the server, the client can’t do anything to save itself from you.
Summary
To summarize my findings:
- There’s no possibility of MITM against
gem install
- And I have full MITM with RCE against
bundle install
Back to the Story…
After about 3 months I tallied up my stats. I had a collection of around 100 IPs. The geographical layout looked like this:
1 2 3 4 5 6 7 8 9 10 |
|
Not too shabby. You’ll noticed that half of those connections are for AWS. Which should give you pause, because it likely means my MITM was running on a staging or production server. Yikes! At this point I felt like my little experiment had run its course and demonstrated that this was a viable method for running a rubygems MITM.
I wasn’t entirely sure how to properly disclose this info. It was sensitive. But also, kinda not really. Especially since I owned the domain in question. But that said, there could be other domains that could be squatted so I felt it best to start quietly.
I started by submitting a report to HackerOne against the RubyGems program. If I could get paid for this, why not! Rejected. I wasn’t surprised with that. I asked to publicly disclose … it waited for a month until I bumped it, the H1 staff replied “we’ll let you know once we have info” and then closed it. I took that as: “Here is the door, please use it.” man_shrugging
The next avenue I tried was sending an email to rubygems’ security email: security@rubygems.org
. I heard
back within 2 hours, and then nothing. I sent a few follow up emails, but didn’t hear back.
I happened to bump into RenderMan at BSides Edmonton and chatted
with him about the MITM and lack of response. He
fired off a tweet to see if he could get me
a response. Nada.
Finally, I spoke to a colleague that had a connection on the rubygems team. This eventually lead to a response back from my initial email with instructions to file an issue. Progress!
Additionally, since this issue affected bundler, I sent security@bundler.io
an email. That bounced back
as undeliverable. Guess that email doesn’t exist. And sent an email to team@bundler.io
and never got a
response.
That said, Bundler team if you’re reading this … please fix your security email address!
At this point, I submitted issues with the rubygems project and bundler project, and fired up the ol’ text editor to write this post.
This MITM turned out to be a lot of fun to run through. While it wasn’t a super serious issue, had I been a malicious actor, I could have RCE on a bunch of computers right now so… Achievement unlocked … Kinda! trophy