By Chad Richts & Martyn Ruks
There are few people in the World who won’t have been exposed in some way, shape or form to the latest craze in live experience challenges, known to many as “Escape Games”. The popularity has skyrocketed of an activity that involves a group of people paying to be locked in a room with the challenge of reaching freedom by solving puzzles. The Mad Scientist’s office challenge at HackFu ZA 2016 draws much of its inspiration from this phenomenon as well as the entire history of puzzling and gaming. However, in true HackFu style the re-imaging of an escape room in the guise of a HackFu challenge included a few surprises. The results of observing those completing it also shone a light on the way people interact and operate when working on problems together.
Over the course of HackFu “The Mad Scientist’s Office”
challenge was able to be run around 8 times, each one involving a group of
between 4 and 6 people. Each team had a mix of consultants, Project Managers and Account Directors. The concept of the challenge was a simple one; search the abandoned scientist’s office for clues and put them together to reveal the identity of “patient zero”.
By observing these groups attempting the challenge it was possible to draw out both common themes and interesting edge cases that can be used to penetrate the seemingly simple concept of the escape room and reveal what lies within it. What we observed is presented here and reveals more than you might expect about skills and approaches needed within the field of Cyber Security.
Completing the office challenge
required very little technical knowledge, making it accessible to a wide range
of people and groups thereof. However, it did contain a number of constraints
that were designed to encourage certain behaviours and make the optimal
approach one that exercises the skills we believe to be important in the real
world. For example, limiting the time that teams had to complete the challenge
means that efficiency in solving problems was important and promoted an
effective team dynamic and structure with a role for a leader. However, the
time pressure also had a noticeable negative impact on some behaviours with tunnel
vision and fixation on the wrong problems being the relevant outcomes.
It was immediately apparent that groups who worked well together found the challenge much easier than groups where the individuals operated independently. In practical terms those working well together self-organised and created roles for team members at different points in the challenge. The clear winners on this front were those with someone leading the team, delegating responsibility and maintaining a view of the bigger picture.
This turns out to be a key approach as individual puzzles often required multiple pieces of information or equipment to solve and needed findings to be shared, interpreted and applied back into the wider context of the game. It was often observed that individuals would get caught up with a particular puzzle for an extended amount of time, and as soon as another team member stepped in to help, the puzzle would be solved within minutes.
In all groups we observed the leader evolved out of individual behaviours and was not specifically selected by the team, an outcome believed to be primarily as a result of the challenge environment being a new one for the participants. It is commonly observed that the first time a team approaches a problem they get caught up in the detail and urgency of it and do not immediately take time to find the optimal solution.
In subsequent running of similar puzzles it has been shown that the team approach will develop and the value of a leader and roles within the team is learned and then implemented. The value that a leader brings is clear when examining all some observations made during the game. Indeed the value of leadership when approaching challenges in cyber security should also be clear as many of the same elements seen in games and puzzles like this are present. This theme of leadership is one you'll see woven throughout this article, another indication of its critical importance.
Although the challenge had a path that teams could follow towards their objective, on some occasions the challenge to be solved was actually to identify the direction within the game to head next. The game provided clues and feedback to the participants if they looked for this pathway but again, without someone taking responsibility for keeping it in view, this direction of travel was obscured or even lost. The effect was that the teams then spent their time working on the "wrong problems".
For example, at several stages of the challenge there were puzzles where progress could be achieved simultaneously. However, the teams often got caught up working together on individual puzzles, sometimes ones where there was insufficient information for them to be completed, or sometimes complete red herrings, at the expense of other puzzles for which signposting showed that they were in fact the direction to head in.
In this regard the skill of the puzzle maker needs to come to the forefront if the contestant experience is to be maintained. It is important that competitors are able to identify if the item or puzzle they are working on will advance them in the game, or is a dead end.
Assuming the puzzle maker has done their job correctly, there are two different approaches teams can take in these situations that can enable them to advance. The first is parallel working, where different team members work on different problems, looking to achieve a breakthrough on any one of them. The second is for the leader to use the feedback the game is providing to determine which is the most important puzzle to focus on at that point in time and ensure the right resources are deployed on that task.
The game can provide feedback to players in many ways but some common ones are showing how many pieces a puzzle requires, for example, a torch requiring 3 batteries but where only one is present. This shows that the item is probably intended to be used and that 2 more batteries are required before this can happen. Your approach should then be to search harder for a battery or leave the item until one is found.
Similarly, the revealing of an object by opening a lock on a drawer or a box can reveal its important in the timeline of the game. If a connection can be identified between the item and others then again the number of required pieces for completing the puzzle can again be assessed and the need to solve that component can then be assessed.
Due to the time constraints enforced on the challenge, teams had to manage the time that was spent on each puzzle. Teams obviously performed better when they did not spend too much time focused on a single puzzle at the expense of others. Often stepping back and attempting another puzzle when progress slowed, resulted in teams completing the challenge much quicker. Those that took account of the feedback the game was providing them were by far the best and avoided being stuck in dead ends for too long.
This ability to see the pathway ahead and to understand the information or capabilities needed to solve a challenge, whose solution pathway may be obscured, is also crucial for cyber security problem solving. For example, when conducting research or looking for solutions to complex problems there are a range of capabilities that if used effectively will result in the route to the solution being shorter, faster and cheaper. However, if you are stuck in the detail without a view of the bigger picture you can often spend your time working in a dead-end without realising.
In many instances we observed the competitors making similar mistakes when attempting to complete challenges. These are mistakes that would not have occurred if the next goals of the challenge were clearly understood. This can be neatly illustrated in the following example of one of the puzzle components. The contestants were provided with the items in the following picture, the two of most relevance being the picture with red tape on three of the corners and the strange black construction with a pinhole cut in one end.
Additionally, by moving a rug aside in the room it would have revealed the strange marks on the floor observed in the following picture.
At this point in the game the competitors should have been focused on the fact that to progress they needed to identify another 4 digit numeric code to be able to open the lock on a desk drawer.
If this goal was at the forefront of thinking the logical approach would be to look at the pictured items and find a method through which such a code might be revealed. One clue is the tape on the floor that indicates the positions that the objects should be placed, with colour coding directly linking them to the objects. However, the presence of a pinhole in the strange black object and the street-map in the picture frame containing numerous 4 digit numbers are indicators towards the intended solution. This is to “correctly position the hole above the map and look through it to see which number is correct”.
Without making this deduction about what the goal was, the majority of teams spent a lot of time investigating incorrect solutions to the problem. This resulted in them attempting to position the black wooden piece on its side or directly on top of the picture, as well as a variety of other humorous approaches that will not be repeated here to protect the dignity of those involved. This was all determined to be directly as a result of them lacking a sense of what their goal was. Those that understood their objective put the pieces, of what in reality is a very simple 2-piece jigsaw puzzle, together easily and recovered the 4-digit code to unlock the drawer with ease.
Understanding what the bigger picture looks like an what we are trying to achieve is another key cyber security capability. We often observe situations where the security of a system hasn't been correctly assured because those responsible for doing do didn't understand what they were trying to achieve. For example, solely investing time and effort proving whether the most complex attack path is viable or not whilst completely ignoring the easiest and most likely route the attacker will take. In many cases the reason for doing so was because those involved believed that particular activities, were absolute requirements in order to do security well, rather than focusing on what was most important to protect the system in question.
The phenomena of positive re-enforcement is well known, it’s where one person makes an assertion and then another person agreeing that it is correct causes others to also believe this to be the truth. This then acts as further re-enforcement and makes it difficult to break this perpetuating cycle of implied truth and to challenge where the initial assertion was correct.
One interesting example of this was observed during the challenge, when an audio clip of a disguised voice was played as a result of solving one puzzle. One member of the team upon listening to the sound expressed out loud what they thought the voice was saying; however, what they thought they had heard was completely wrong. Every team member who had heard them say this phrase before listening to the sound then agreed it was what was being said every time the voice clip was subsequently played. This example is slightly more than just positive re-enforcement as effects such as Mondegreen are also at play in this example.
After 5 minutes of being stuck on this issue, it took another team member who had been paying no attention to them and who had not heard the interpretation to listen to the voice and correctly identified what was being said on the first time of listening. This enabled the untangling of both the Mondegreen effect as well as the positive re-enforcement that was preventing the other contestants from challenging the original assertion.
This phenomena is often witnessed in cyber security where one person makes a false assertion, often based on an incorrect underlying assumption that others then believe and don’t question. During security research, for example, someone will say that a technology component doesn't have vulnerabilities because it "must have been looked at by lots of people in the past". It often takes someone not involved or more interestingly with an approach to naturally challenging assumptions to break down the misconceptions and arrive at the truth. In this example someone who refuses to believe that anything can be free from vulnerabilities or someone who has heard that false assumption before will naturally challenge it. But at a more fundamental level this ability to challenge the assumptions that have the most chance of being incorrect is something that is commonly observed in people who excel in cyber security roles.
The same challenge that delivered the positive re-enforcement also highlighted another aspect of how analysis of the clues and applying them in the game context differed between teams. The clue revealed in the aforementioned distorted voice clip was the phrase “To be or not to be”, the origin of which was immediately understood by every team. However, the majority of them initially looked at this phrase literally and immediately began searching the room for a book about Shakespeare (something that was in fact predicted by the puzzle makers). This was an incorrect approach although one that can be quickly eliminated once no books even remotely connected to Shakespeare can be found, an example of the game providing feedback.
At this point its worth a slight digression to mention the fact that having a book about Shakespeare in the room may outwardly seem like an opportunity the puzzle maker missed to make the game harder to play. But doing so without another clear indication that this route was a dead-end would be misleading and would reduce the rewards the players take from the game. This is a subject best saved for another article but one key takeaway here is that if you create dead-ends for players, with no feedback that they are, you will frustrate, confuse and demoralise the players. Outcomes that should never be desired by the puzzle maker.
Returning to the puzzle in the Office challenge, what the teams should have been doing is looking at alternative interpretations of the phrase and how they might lead to solving the puzzle. To some this might be obvious, to others it might be more obvious had I written the clue as “2B or not 2B”. However, only those who looked at such alternate interpretations of the data in front of them realised that their next search of the room should relate to “2B” instead of Shakespeare.
This clue would quickly lead the teams to a map and the contents of the square at the grid reference “2B”. This phrase was then required to be used in combination with another clue obtained from another puzzle.
There are also good parallels to draw between this and the real world. For example, the interpretation of patterns or information in multiple ways until a useful form is found is another useful skill to possess within a cyber security context. For example, when using anomalies in detecting signs of a compromise there pieces of data are often encountered that are clearly unusual and worthy of further investigation. However, their relevance in the attacker's context may require interpreting the data in different ways. This is therefore another interesting observation of how thought processes revealed by this challenge map directly back to the real world.
Another constraint introduced to the challenge was that no laptops, phones or other electronic aids could be used to solve the puzzles. Teams were provided with the somewhat old fashioned tool of a pen and paper, to make notes of interesting things found. While it is now 2016, a pen and paper is still incredibly useful in this situation, that is if they are used. It was consistently observed that teams would not use this resource effectively, some would not use it at all. Often, after writing down many irrelevant facts on the paper they would find what was obviously a clue or code to open a lock and then choose not write it down, with the result of having completely forgotten about it by the time it was necessary to use it.
In effective teams one person is often assigned the role of the scribe, on the outside a seemingly mundane role, but a critical one in the context of the game. By having visibility over all the clues and codes that are recorded, the ability to progress further in the challenge can often rest in the hands of this person.
In order to solve the Mad Scientist's challenge in the required time, teams had to divide and conquer by each searching a different part of the office. This was needed in order to get full coverage in the shortest time possible, and to be effective naturally forced the team members to communicate with each other. A big stumbling block for a lot of teams was lack of this communication. They would split up to look for clues, then when found they would neglect to share the clues they found with the rest of the team. In some cases it was observed that a team member would complete a puzzle to retrieve a code, then neglect to share it with the rest of the team. the result being that the power to use the code was concentrated in a small percentage of the overall team's capability.
The use of the aforementioned scribe role would help to address this problem, but only if their interactions with the team and ability to influence team member behaviours were correct. This is further illustration as to why the role can be critical and not simply something to give to the "rookie". So we are drawn back to role of the leader being to deploy their resources most effectively to ensure a key part of operational effectiveness is addressed.
Again in a cyber security context the ability to record and access information from all parts of a team is critical, a breakthrough, clever thinking or new piece of code implementing a game-changing capability is no use if others cannot benefit from it.
All of the observations we made while running this challenge at HackFu ZA were fascinating and while the nuances of the teams’ approaches may have given rise to some classic comedy moments, the problem solving capability of those participating was nothing short of impressive. Yes the teams made the mistakes highlighted here, but lets put those into context.
The examples we have drawn on in this article are made with the visibility of the puzzle makers who collectively possess a wealth of knowledge and experience in this field. Also, as with all challenges at HackFu, the contestants are always pushed outside their comfort zone and and are allowed to learnt through some of the mistakes made in their first attempt. Remember that one of the reasons for running HackFu is to provide an environment where people can fail-safe the first time they attempt something. This challenge is no different!
The challenge was created with the aim of teams finding a specific solution using a pre-determined pathway; however, teams were incredibly creative in how they made some of the leaps needed to solve each component and in many cases each team solved the same problem in a slightly different manner. Some teams also found unforeseen creative loopholes in the challenge allowing them to skip certain puzzles, but that's another subject.
It was, however, observed across all the teams that teamwork and leadership within the group, most specifically in-game communication within the team, was either the biggest stumbling block for the team or their greatest advantage. As we have explained, many of our observations about good solutions for solving the challenge map directly to the use of skills or capabilities that are important in the cyber security industry and for solving the challenges we face every day.
As a result of the challenge, our observations and further analysis of them, we’re pleased to be able to draw some conclusions. We believe that anyone involved in cyber security should challenge themselves against some of the multitude of escape games and live action puzzles that now exist.
In fact, if you’re not putting at least one of them in you and your team’s training objectives for the coming year then something isn’t right. One thing that’s certain though is that the concept as well as some fiendish new puzzle ideas will be returning to a HackFu event near you very soon!
In our opening article we discuss how we see the problem space and what some of the challenges are so given all of that, what are we looking to achieve with this website.
As we said previously we don’t claim to know all the solutions but we are certainly trying things that will enable us to find them. Along the way we’ll learn lots of lessons, we’ll no doubt have a few false starts but we’ll also get some stuff right. It’s that journey that we’re going to share on this site, including all the initiatives and events that we’re planning to run to support it all.
We don’t know yet exactly what will be on this site but the one thing we’re certain of is that it will be thought provoking, philosophical, challenging and most importantly it will be fun and engaging. We’ll get techie at times, something we make no apology for, as well as extracting the concepts and key points that we encounter along the way.
We also want your help on our journey. We want you to tell us your thoughts and experiences of what we’re doing and hopefully some stories about how you’ve taken our ideas and turned them into your own projects, events and general awesomeness. We want to know the good, the bad and the ugly and to get involved please get in contact via Twitter and then keep coming back to this hub for more insight and information.
So hold on tight, this will be a wild ride for all of us.