debugging (2024)

Background

I was contacted this week by a geocacher called Dominik who, like me, loves geocaching…. but hates it when the coordinates for a cache are hidden behind a virtual jigsaw puzzle.

A popular online jigsaw tool used by lazy geocache owners is Jigidi: I’ve come up with several techniques for bypassing their puzzles or at least makingthem easier.

Dominik had been looking at a geocache hidden last week inEastern France and had discovered that it used JigsawExplorer, not Jigidi, to conceal the coordinates. Let’s take a look…

I experimented with a few ways to work-around the jigsaw, e.g. dramatically increasing the “snap range” so dragging a piece any distance would result in it jumping to aneighbour, and extracting original image URLs from localStorage. All were good, but none wereperfect.

Then I realised that – unlike Jigidi, where there can be a congratulatory “completion message” (with e.g. geocache coordinates in) – in JigsawExplorer the prize is seeing thecompleted jigsaw.

Let’s work on attacking that bit of functionality. After all: if we can bypass the “added challenge” we’ll be able to see the finished jigsaw and, therefore, the geocachecoordinates. Like this:

Hackaround

1. Open a jigsaw and try the “box cover” button at the top. If you get the message “This puzzle’s box top preview is disabled for added challenge.”, carry on.
2. Open your browser’s debug tools (F12) and navigate to the Sources tab.
3. Find the jigex-prog.js file. Right-click and select Override Content (or Add Script Override).
4. In the overridden version of the file, search for the string – e&&e.customMystery?tt.msgbox("This puzzle's box top preview is disabled for added challenge."): –this code checks if the puzzle has the “custom mystery” setting switched on and if so shows the message, otherwise (after the :) shows the box cover.
5. Carefully delete that entire string. It’ll probably appear twice.
6. Reload the page. Now the “box cover” button will work.

The moral, as always, might be: don’t put functionality into the client-side JavaScript if you don’t want the user to be able to bypass it.

Or maybe the moral is: if you’re going to make a puzzle geocache, put some work in and do something clever, original, and ideally with fieldwork rather than yet another low-effort“upload a picture and choose the highest number of jigsaw pieces to cut it into from the dropdown”.

(Just want the instructions? Scroll down.)

A year and a half ago I came up with a technique for intercepting the “shuffle” operationon jigsaw website Jigidi, allowing players to force the pieces to appear in a consecutive “stack” for ludicrously easy solving. I did thispartially because I was annoyed that a collection of geocaches near me used Jigidi puzzles as a barrier to their coordinates¹…but also because I enjoy hacking my way around artificially-imposed constraints on the Web (see, for example, my efforts last week to circumvent region-blocking on radio.garden).

My solver didn’t work for long: code changes at Jigidi’s end first made it harder, then made it impossible, to use the approach I suggested. That’s fine by me – I’d already got what Iwanted – but the comments thread on that post suggests that there’sa lot of people who wish it still worked!²And so I ignored the pleas of people who wanted me to re-develop a “Jigidi solver”. Until recently, when I once again needed to solve a jigsaw puzzle in order to find a geocache’scoordinates.

Making A Jigidi Helper

Rather than interfere with the code provided by Jigidi, I decided to take a more-abstract approach: swapping out the jigsaw’s image for one that would be easier.

This approach benefits from (a) having multiple mechanisms of application: query interception, DNS hijacking, etc., meaning that if one stops working then another one can be easilyrolled-out, and (b) not relying so-heavily on the structure of Jigidi’s code (and therefore not being likely to “break” as a result of future upgrades to Jigidi’s platform).

Watch a video demonstrating the approach:

It’s not as powerful as my previous technique – more a “helper” than a “solver” – but it’s good enough to shave at least half the time off that I’d otherwise spend solving a Jigidijigsaw, which means I get to spend more time out in the rain looking for lost tupperware. (If only geocaching were even the weirdest of my hobbies…)

What makes it easier to solve?

The replacement image has the following characteristics that make it easier to solve than it might otherwise be:

• Every piece has written on it the row and column it belongs in.
• Every “column” is striped in a different colour.
• Striped “bands” run along entire rows and columns.

To solve the jigsaw, start by grouping colours together, then start combining those that belong in the same column (based on the second digit on the piece). Join whole or partialcolumns together as you go.

I’ve been using this technique or related ones for over six months now and no code changes on Jigidi’s side have impacted upon it at all, so it’s probably got better longevity than theprevious approach. I’m not entirely happy with it, and you might not be either, so feel free to fork my code and improve it: the legiblity of the numbers is sometimes suboptimal, andthe colour banding repeats on larger jigsaws which I’d rather avoid. There’s probably also potential to improve colour-recognition by making the colour bands span the gapsbetween rows or columns of pieces, too, but more experiments are needed and, frankly, I’m not the right person for the job. For the second time, I’m going to abandon a toolthat streamlines Jigidi solving because I’ve already gotten what I needed out of it, and I’ll leave it up to you if you want to come up with an improvement and share it with thecommunity.

tl;dr? Just want instructions on how to solve Jigidi puzzles really fast with the help of your browser’s dev tools? Skip to that bit.

I don’t enjoy jigsaw puzzles

I enjoy geocaching. I don’t enjoy jigsaw puzzles. So mystery caches that require you to solve an online jigsaw puzzle in order to get the coordinates reallydon’t do it for me. When I’m geocaching I want to be outdoors exploring, not sitting at my computer gradually dragging pixels around!

Many of these mystery caches use Jigidi to host these jigsaw puzzles. An earlier version of Jigidi was auto-solvable with a userscript, but the service has continued to be developed and evolve and the current version works quite hard tomake it hard for simple scripts to solve. For example, it uses a WebSocket connection to telegraph back to the server how pieces are moved around and connected to one another and theserver only releases the secret “you’ve solved it” message after it detects that the pieces have been arranged in the appropriate relative configuration.

If there’s one thing I enjoy more than jigsaw puzzles – and as previously established there are about a billion things I enjoy more than jigsaw puzzles – it’s reverse-engineering acomputer system to exploit its weaknesses. So I took a dive into Jigidi’s client-side source code. Here’s what it does:

1. Get from the server the completed image and the dimensions (number of pieces).
2. Cut the image up into the appropriate number of pieces.
3. Shuffle the pieces.
4. Establish a WebSocket connection to keep the server up-to-date with the relative position of the pieces.
5. Start the game: the player can drag-and-drop pieces and if two adjacent pieces can be connected they lock together. Both pieces have to be mostly-visible (not buried under otherpieces), presumably to prevent players from just making a stack and then holding a piece against each edge of it to “fish” for its adjacent partners.

Looking at that process, there’s an obvious weak point – the shuffling (point 3) happens client-side, and before the WebSocket sync begins. We could override theshuffling function to lay the pieces out in a grid, but we’d still have to click each of them in turn to trigger the connection. Or we could skip the shuffling entirely and just leavethe pieces in their default positions.

And what are the default positions? It’s a stack with the bottom-right jigsaw piece on the top, the piece to the left of it below it, then the piece to the left of that and son onthrough the first row… then the rightmost piece from the second-to-bottom row, then the piece to the left of that, and so on.

That’s… a pretty convenient order if you want to solve a jigsaw. All you have to do is drag the top piece to the right to join it to the piece below that. Then move those two to theright to join to the piece below them. And so on through the bottom row before moving back – like a typewriter’s carriage return – to collect the second-to-bottom row and so on.

How can I do this?

If you’d like to cheat at Jigidi jigsaws, this approach works as of the time of writing. I used Firefox, but the same basic approach should work with virtually any modern desktop webbrowser.

1. Go to a Jigidi jigsaw in your web browser.
2. Pop up your browser’s developer tools (F12, usually) and switch to the Debugger tab. Open the file game/js/release.js and uncompress it by pressing the {} button, if necessary.
3. Find the line where the code considers shuffling; right now for me it’s like 3671 and looks like this:
   return this.j ? (V.info('board-data-bytes already exists, no need to send SHUFFLE'), Promise.resolve(this.j)) : new Promise(function (d, e) {

   

4. Set a breakpoint on that line by clicking its line number.
5. Restart the puzzle by clicking the restart button to the right of the timer. The puzzle will reload but then stop with a “Paused on breakpoint” message. At this point theapplication is considering whether or not to shuffle the pieces, which normally depends on whether you’ve started the puzzle for the first time or you’re continuing a saved puzzle fromwhere you left off.
   
6. In the developer tools, switch to the Console tab.
7. Type: this.j = true (this ensures that the ternary operation we set the breakpoint on will resolve to the true condition, i.e. not shuffle the pieces).
   
8. Press the play button to continue running the code from the breakpoint. You can now close the developer tools if you like.
9. Solve the puzzle as described/shown above, by moving the top piece on the stack slightly to the right, repeatedly, and then down and left at the end of each full row.
   

Abraxas observes that Jigidi have changedtheir code, possibly in response to this shortcut. Unfortunately for them, while they continue to perform shuffling on the client-side they’ll always be vulnerable to this kind ofsimple exploit. Their new code seems to be named not release.js but given a version number; right now it’s 14.3.1977. You can still expand it in the same way,and find the shuffling code: right now for me this starts on line 1129:

Put a breakpoint on line 1129. This code gets called twice, so the first time the breakpoint gets hit just hit continue and play on until the second time. The second time it gets hit,move the breakpoint to line 1130 and press continue. Then use the console to enter the code d = a.G and continue. Only one piece of jigsaw will be shuffled; the rest willbe arranged in a neat stack like before (I’m sure you can work out where the one piece goes when you get to it).

I’ve not had time nor inclination to re-“break” Jigidi’s shuffler, but on the rare ocassions I’veneeded to solve a Jigidi, I’ve come up with a technique that replaces a jigsaw’s pieces with ones that eachshow the row and column number they belong to, as well as colour-coding the rows and columns and drawing horizontal and vertical bars to help visual alignment. It makes the processsignificantly less-painful. It’s still pretty buggy code though and I end up tweaking it each and every time I use it, but it certainly works and makes jigsaws that lack clear visualmarkers (e.g. large areas the same colour) a lot easier.

Earlier this week, I mentioned the exciting hackathon that produced a moderately-faithful reimagining of the world’s first Web browser. I was sufficientlyexcited about it that I not only blogged here but I also posted about it to MetaFilter. Of course, the veryfirst thing that everybody there did was try to load MetaFilter in it, which… didn’t work.

People were quick to point this out and assume that it was something to do with the modernity of MetaFilter:

honestly, the disheartening thing is that many metafilter pages don’t seem to work. Oh, the modern web.

Some even went so far as to speculate that the reason related to MetaFilter’s use of CSS and JS:

CSS and JS. They do things. Important things.

This is, of course, complete baloney, and it’s easy to prove to oneself. Firstly, simply using the View Source tool in your browser on a MetaFilter page reveals source code that’s quitecomprehensible, even human-readable, without going anywhere near any CSS or JavaScript.

Secondly, it’s pretty simple to try browsing MetaFilter without CSS or JavaScript enabled! I tried in two ways: first,by using Lynx, a text-based browser that’s never supported either of those technologies. I also tried by usingFirefox but with them disabled (honestly, I slightly miss when the Web used to look like this):

And thirdly: the error code being returned by the simulated WorldWideWeb browser is aHTTP code 500. Even if you don’tknow your HTTP codes (I mean, what kind of weirdo would take the time to memorise them all anyway <ahem>),it’s worth learning this: the first digit of a HTTP response code tells you what happened:

• 1xx means “everything’s fine, keep going”;
• 2xx means “everything’s fine and we’re done”;
• 3xx means “try over there”;
• 4xx means “you did something wrong” (the infamous 404, for example, means you asked for a page that doesn’t exist);
• 5xx means “the server did something wrong”.

Simple! The fact that the error code begins with a 5 strongly implies that the problem isn’t in the (client-side) reimplementation of WorldWideWeb: if this had have been aCSS/JS problem, I’d expect to see a blank page, scrambled content, “filler”content, or incomplete content.

So I found myself wondering what the real problem was. This is, of course, where my geek flag becomes most-visible: what we’re talking about, let’s not forget, is a fringeproblem in an incomplete simulation of an ancient computer program that nobody uses. Odds are incredibly good that nobody on Earth cares about this except, right now, for me.

Luckily, I spotted Jeremy’s note that the source code for the WorldWideWebsimulator was now available, so I downloaded a copy to take a look. Here’s what’s happening:

1. The (simulated) copy of WorldWideWeb is asked to open a document by reference, e.g. “https://www.metafilter.com/”.
2. To work around same-origin policy restrictions, the request is sent to an API which acts as a proxy server.
3. The API makes a request using the Node package “request” with this line of code: request(url, (error, response, body) =>{ ... }). When the first parameter to request is a (string) URL, the module uses its default settings for all ofthe other options, which means that itdoesn’t set the User-Agent header (an optional part of a Web request where the computer making the request identifies the softwarethat’s asking).
4. MetaFilter, for some reason, blocks requests whose User-Agent isn’t set. This is weird! And nonstandard: while web browsers should – in RFC2119 terms – set their User-Agent: header, web servers shouldn’t requirethat they do so. MetaFilter returns a 403 and a message to say “Forbidden”; usually a message you only see if you’re trying to access a resource that requires session authentication andyou haven’t logged-in yet.
5. The API is programmed to handle response codes 200 (okay!) and 404 (not found), but if it gets anything else backit’s supposed to throw a 400 (bad request). Except there’s a bug: when trying to throw a 400, it requires that an error message has been set by the request module and if therehasn’t… it instead throws a 500 with the message “Internal Server Fangle” and no clue what actually went wrong. So MetaFilter’s 403 gets translated by the proxy into a 400 whichit fails to render because a 403 doesn’t actually produce an error message and so it gets translated again into the 500 that you eventually see. What a knock-on effect!

The fix is simple: simply change the line:

request(url, (error, response, body) => { ... })

to:

request({ url: url, headers: { 'User-Agent': 'WorldWideWeb' } }, (error, response, body) => { ... })

This then sets a User-Agent header and makes servers that require one, such as MetaFilter, respond appropriately. I don’t know whether WorldWideWeb originally set a User-Agent header(CERN’s source file archive seems to be missing the relevant C sources so I can’t check) but Isuspect that it did, so this change actually improves the fidelity of the emulation as a bonus. A better fix would also add support for and appropriate handling of other HTTP responsecodes, but that’s a story for another day, I guess.

I know the hackathon’s over, but I wonder if they’re taking pull requests…

If you’re reading this post via my blog and using a desktop computer, try opening your browser’s debug console (don’t worry; I’ll wait). If you don’t know how, here’s instructions for Firefox and instructions for Chrome. Other browsers may vary. You ought to see something like this in yourdebugger:

What sorcery is this?

The debug console is designed to be used by web developers so that they can write Javascript code right in their browser as well as to investigate any problems with the code run by aweb page. The web page itself can also output to the console, which is usually used for what I call “hello-based debugging”: printing out messages throughout a process so that the flowand progress can be monitored by the developer without having to do “proper” debugging. And it gets used by some web pages to deliver secret messages to any of the site users who opentheir debugger.

Principally, though, the console is designed for textual content and nothing else. That said, both Firefox and Chrome’s consoles permit the use of CSS to style blocks of debug output by using the %c escape sequence. For example, I could style some of a message with italic text:

>> console.log('I have some %citalic %ctext', 'font-style:italic;', '');
I have someitalic text

Using CSS directives like background, then, it’s easyto see how one could embed an image into the console, and that’s been done before. Instead, though, I wanted to usethe lessons I’d learned developing PicInHTML 8¾ years ago to use text and CSS(only) to render a colour picture to the console. First, I created my template image – a hackergotchi of me and an accompanyingspeech bubble, shrunk to a tiny size and posterised to reduce the number of colours used and saved as a PNG.

Next, I wrote a quick Ruby program, consolepic.rb, to do the hard work. It analyses each pixel of the imageand for each distinct colour assigns to a variable the CSS code used to set the background colour to that colour. It looks for“strings” of like pixels and combines them into one, and then outputs the Javascript necessary to write out all of the above. Finally, I made a few hand-tweaks to insert the text intothe speech bubble.

The resulting output weighs in at 31.6kB – about a quarter of the size of the custom Javascript on the frontend of my site and so quite abit larger than I’d have liked and significantly less-efficient than the image itself, even base64-encoded for embedding directly into the code, but that really wasn’t thepoint of the exercise, was it? (I’m pretty sure there’s significant room for improvement from a performance perspective…)

What it achieved was an interesting experiment into what can be achieved with Javascript, CSS, the browser console, and a littleimagination. An experiment that can live here on my site, for anybody who looks in the direction of their debugger, for the foreseeable future (or until I get bored of it). Anybody withany more-exotic/silly ideas about what this technique could be used for is welcome to let me know!

Investigating a possible new bug in @firefox 57: after installing the service worker, going to an uncached page on a site like adactio.com (@adactio), danq.me, or 3r.org.uk results in a“NetworkError” and the offline page, even though the connection is fine…

Chrome comes with built-in developer tools. This comes with a wide variety of features, such as Elements, Network, and Security. Today, we’ll focus 100% on its JavaScript console.

When I started coding, I only used the JavaScript console for logging values like responses from the server, or the value of variables. But over time, and with the help of tutorials,I discovered that the console can do way more than I ever imagined…

This link was originally posted to /r/MegaMegaMonitor. See more things from Dan's Reddit account.

The original link was: http://i.imgur.com/KxcrKsY.png