Mar 27

Writing Python in Scheme

Python was the first language I learned (second if you count QBasic, and I don’t). Like the classic Fortran programmer who can write Fortran in any language, I have often found myself instinctively writing my Lisp programs with a distinctly Pythonic flavor. To get it out of my system, here are a few simple macros to recreate Python loops in Scheme. This is not serious and should not be seriously used. It’s just a bit of fun!

First, I recreated the beloved Python for loop. Lispers insist that Python’s for is just syntactic sugar for map, but I find that Python style for loops often express themselves less succinctly as map calls.

(define-syntax-rule (for i in lst body ...)
  (map (lambda (x) 
         (let ((i x)) (begin body ...))) 
       lst))

 
And here is how it works when you call it:

(define a 0)

(for i in (range 3)
  (set! a (+ a i))
  (print a))

Output: 136

 
The cool thing about that macro is that it returns a list of the value of each iteration. This allows us to easily emulate Python’s list comprehensions as well, like so:

> (for i in (range 3)
    (number->string (+ i 1)))
'("1" "2" "3")

 
To appreciate the elegance of that macro, here is the same code using a traditional call to map:

(map (lambda (x) 
       (number->string (+ x 1))) 
     (range 3))

 
Maybe I am just biased, but the Pythonic style just seems more clear. Anyway, let’s do one more loop – the while loop. The while loop has been implemented thousands of different ways in Scheme, but we want to mimic the Python while loop as much as possible, so we will use the following macro:

(define-syntax-rule (while condition body ...)
  (let loop ((iter-vals null))
    (if condition
        (loop (append iter-vals (list (begin body ...))))
        iter-vals)))

 
Here is our macro being used:

(define p 0)

(while (< p 5)
  (set! p (+ p 1))
  (print p))

Output: 12345

 
Interestingly, because this macro is so similar to our for loop, we can even use comprehensions with it, like this:

> (while (< p 5) (set! p (+ p 1)) (* p 10))
'(10 20 30 40 50)

 
Scheme is such a wonderfully expressive language that recreating your favorite features from other languages is extremely easy, although I recommend against it. If you want to program in every language simultaneously learn C++. For the rest of us, source code is already obfuscated enough. That’s enough fun for now. Time to figure out what I was actually working on before I got distracted by this!

Feb 20

Why governments are not attacking Silk Road

When people are introduced to Silk Road, one of their earliest instincts is to articulate a few obvious methods by which law enforcers could potentially subvert the community or stability of the site. The feasibility of the attacks usually falls apart given meticulous scrutiny. Most often the techniques rely on social engineering, but occasionally technical attacks are suggested. The technical approaches can be mitigated in the conventional ways and the fortification of web servers has been described in great volume elsewhere, so we will focus on the most commonly suggested social engineering attack.

The social engineering scenario most often contrived involves a government (in practice it is almost always the US government) making thousands of vendor and customer accounts, buying things from themselves, and leaving very positive reviews. When real customers buy from these apparently well established merchants government agents arrest them. The real impact here is not the arrests, but the destruction of confidence in merchants. How can you know who is or is not a government sockpuppet if the majority of the well established vendors may very well be honeypots? The elephant in the room for those familiar with Silk Road’s merchant system is that vendor accounts cost over 500 dollars each, and even that is subject to increase if administrators feel the barrier to entry is too low to stop fakes. Let’s take a quick look at the economics at play here.

Let V be the price of a single vendor account, let G be the number of vendor accounts the government will buy, and let P be the price. Basic arithmetic shows that V * G = P. This means that if the government wants to poison the well with 5000 accounts, and each account is $500, the price just to start out will be $2,500,000. Keep in mind that the price is liable to increase once the administrators feel threatened, and the attackers would need to continuously buy new accounts for each sockpuppet they use to make an arrest.

Another problem with this technique is the administrators would be able to see which accounts are interacting with each other, and it would be possible to figure out which accounts were not buying and selling outside of the government bubble. Thus, this would be an extremely expensive operation and even an organization that had the money could be easily stopped by diligent administrators.

Jan 23

WP Super Cache’s gzip Compression Bug Explained

If you tried to go to certain WordPress sites recently you may have noticed a very strange bug. Especially if that site used WordPress 3.5 and WP Super Cache 1.2 with compression turned on and is hosted on Apache servers that don’t have mod_gzip or mod_deflate enabled. You click a link on the site and it would fail to load. Then once you refreshed it would load fine. And it would continue to load fine as long as you had been to that page on the site before. This happened to me last night when I tried to visit the Noisebridge blog, and the debugging fun began!

First I tried loading it from a different browser, then from a proxy, then from Tor Browser, then from my iPhone, to make sure it wasn’t something local to me. Other users in the #noisebridge IRC channel on Freenode confirmed they were sporadically experiencing the same problem, and one user told me that from Safari they got an error saying”‘cannot decode raw data”. Tor Browser (which I believe is simply a modified fork of Firefox) was the only browser that actually gave me a meaningful error, saying “Content Encoding Error: The page you are trying to view cannot be shown because it uses an invalid or unsupported form of compression.”

I idled for a bit pondering where to start, and finally I decided to check out the HTTP headers coming from Noisebridge’s server to see if they were telling browsers incorrect information about what form of compression to use, which yielded this:

HTTP/1.1 200 OK
Date: Wed, 23 Jan 2013 06:21:12 GMT
Server: Apache/2.2.22 (Debian)
X-Powered-By: PHP/5.4.4-9
Vary: Cookie,Accept-Encoding
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8

As we can see, Apache’s Content-Type was “Content-Type: text/html; charset=UTF-8″; it didn’t even send Content-Encoding. However, when I looked at the bottom of the body of the response I noticed a very conspicuous comment generated by WP Super Cache, that read as follows:

<!– Dynamic page generated in 0.442 seconds. –>
<!– Cached page generated by WP-Super-Cache on 2013-01-22 22:21:20 –>

<!– Compression = gzip –>
<!– super cache –>

From there I had a pretty good idea of what was happening. The HTTP headers informed the browser that the content was not encoded, so the browser would specifically not decode the data. But because the browser would then cache the page, subsequent browsing to that page worked fine (since you were just loading the data from your hard disk, and there were no incorrect HTTP headers for your browser to misinterpret). The immediate solution was simple – just disable compression in WP Super Cache. But I was still curious as to the root cause of the bug.

I logged in to my own server, installed WP Super Cache and played around in the Apache config until I found some relevant modules. The most probably suspects seemed to be mod_gzip and mod_deflate since those both deal with compression. Disabling either of them yielded the results I was after: As long as you had either mod_deflate or mod_gzip set up, the bug would not occur. But if you disabled both of them, the bug would occur. So there is our solution! For the 90% of you out there who don’t care about the underlying cause you can just disable compression in WP Super Cache and move on with your life. But for the 10% who will go crazy if they don’t get to the bottom of the mystery, consider this case solved.

Jan 21

The Rise and Fall of Torchan

Torchan was founded in December of 2011 by ‘cerulean’ (an alias I used from then on in the Tor hidden service community). For those old enough to remember it, this article will not deal with Misery’s original Torchan, but rather the recent Torchan incarnation run by OnionWare. Growth was rapid as established Torizens flocked to a nostalgic icon of hidden service history. Post quality was painfully low, but certainly better than the abandoned Anonchan, whose operator had not been seen for some time. But with high spirits intellectual discussion was encouraged and the community grew very conscious; to the point of actively propagating on clearnet. We began writing our own tools from scratch, including a search engine, pastebin, and a tool to check if other hidden services were down. Despite the inevitable rush of 4chan users quality continued to rise and post frequency went up and up.

Minor catastrophes struck in the form of pornographic spam and technical issues but we always got things working again, and each disaster fortified the server with lessons learned (regular backups, detecting malicious posting patterns, etc). Security practices were abysmal but no one had experience running a large hidden service, so we learned as we went. Communication on best practices between operators of hidden services is nearly non-existent, but even that can hardly excuse our behavior. Admins added each other on Facebook, sent each other letters, discussed highly personal identifying matters, some even met in real life. The moderation and userbase was strong enough that I would go months at a time without visiting the site only to find it had doubled in size and the staff had been diligently maintaining the boards. Fancycakes, our first additional staff member, put in so much effort that for a long period most people thought he was the sole owner of the site.

But nothing in the site’s history shook the foundation of the community so strongly as the arrival of Alan Zeni – Cheerilee. I was working as a developer in Tulsa when he started posting, and the memory is vivid. I came home and Steph noted “The entire site is filled with ponies”. I giggled, but the communal reaction was one of desperation and melodrama. We deleted his posts – he didn’t care. We appealed to him rationally – he didn’t care. Use of the site plummeted as Cheerilee dove further and further into amphetamine induced mania. He couldn’t stop. In the depths of obsession I doubt he had the psychological capacity to draw himself away from the keyboard.

But draw himself away he certainly did. His pills ran out, and so began three weeks of his agonizing withdrawal. For us it was a brief window during which a containment strategy could be devised. My personal life was also experiencing some turbelance. My work in Tulsa had reached the edge of what it could teach me, so new work was found in Manhattan, where I quickly moved. Then I did something crazy. Not just “oh wow what a wacky adventure” crazy, but something completely insane. I made a thread saying I needed a place to stay and Cheerilee offered me one. He told me where to come, so I got on a train to his house in West Virginia without even a vague idea of what to expect.

The secret of my residency with Cheerilee slowly trickled out until it was finally admitted via a series of pictures posted to the site that we had taken. By that time we had given up any pretenses of anonymity, so sharing the pictures didn’t feel inappropriate. The community was oddly receptive and the site began to explode. It is always hard to tell how many people use Tor hidden services and operators tend to be apprehensive about giving out server statistics. Since our server is dead and gone along with the code, community, and everything else, I have no qualms about sharing information. At last count via Apache’s access_log the site was generating an average of 150k hits daily, with our highest peak being 300k. I had not analyzed traffic at all until the site was very close to it’s death, although none of us knew it was going to die, so I was shocked when I saw that. Torchan was very likely the most popular non-pedo, non-commercial hidden service in existence.

The sudden fever which plunged the site into it’s death throes came abruptly. A cracker or group of crackers calling themselves PonySquad demanded that if cerulean (that is me) is not restored as admin, they will crack into the server. This made no sense at all since I had been the admin for over a year. I explained this in the thread and there was no response. The next day I checked on the site and every post is deleted but one: another demand from PonySquad that cerulean be restored as admin, claiming that Nekro (an administrator of Torchan) and Fancycakes had seized control of the site. Kusaba X’s code had not been updated by the developers in several months and we were on the latest version. It was compromised and there was nothing to be done about it. I went on IRC and offered a backup of the server to whoever wanted to run the site for themselves and told everyone goodbye.

On my VPS administration panel I sat for a bit staring at the server stats, hovering above the button to turn the server off. A bright green “Online” buzzed above the hostname. I clicked the mouse, the server prepared to shut down, and the bright green buzz faded into a calm red “Offline”.