Security Research Techniques: Build Before You Break

I’ve been in security research for quite a while now, so I’ve had more than a few opportunities to guide researchers during their security research endeavors and help them refine their security research techniques.

I’m sure that you’ve had the experience of swinging between feeling stuck with no ideas and being overwhelmed with too many options. Deciding whether to explore a new area, picking between reverse engineering and fuzzing, or starting with network protocol sniffing versus OS API monitoring can be tricky.

Understanding the Role of Intuition in Research

As security researchers, we deal with some of the most complex systems out there. While our work seems purely analytical, a lot of our decisions come down to gut feelings. Explaining these choices can be tough because they’re not just about data, they’re about intuition guiding us through uncertainty. This intuitive side of our work is crucial and can often be the key to a breakthrough, rather than hitting a dead end.

How can you develop such intuition faster? Try to build before you break.

Build Before You Break: A Core Security Research Technique

Before trying to find vulnerabilities, you first need to understand how the system is built. By reconstructing the system from the ground up, you gain valuable insights into its design, functionality, and potential vulnerabilities. This process enhances your intuition and helps you see the system from a new perspective, making it easier to identify and exploit vulnerabilities. In this blog post, we’ll demonstrate this concept with a specific technical example: building a Linux container from scratch.

Imagine you’re trying to fix a car engine. If you understand how each part of the engine works and how they fit together, it’s easier to figure out what’s broken or which mechanism will be the first to fail. By learning how to build an engine, you gain the intuition needed to diagnose and repair it. The same goes for cybersecurity – if we understand how systems are built, we can more effectively identify where vulnerabilities might be.

As a network engineer, I spent years designing networks to be as secure and resilient as possible, understanding every layer of defense and the intricacies of how a well-protected system should function. That experience became my foundation. Once I knew how to build a network to withstand attacks, it became almost second nature to figure out how to break into one. I could see the gaps, the places where things might not hold up under pressure, because I had already been on the other side, ensuring they were fortified. This foundational knowledge enabled me to make a transition into red teaming operations quite successfully, even on unfamiliar systems. Understanding how to build it meant I knew exactly where to look for exploitable gaps.

Build a Linux Container to Practice Security Research Techniques

Let’s dive into a specific example: Containers. Containers are basically an isolation suite for processes and have been part of Linux since 2002. All you need to build a basic container is a handful of Linux syscalls and utilize features like namespaces, chroot, cgroups, and Linux capabilities. Container images are essentially just a bunch of files and folders that include the needed dependencies for your dedicated process to run.

Containers are a great playground to practice hands-on security research techniques that go beyond theory.

You can create a simple container using these commands:

1. Pull the Ubuntu image (that’s the only step where we’ll utilize Docker just for the sake of simplicity instead of building the image from scratch):

This will be the image of our own container. To use it, let’s look it up in the Docker image directory:

ls -la /var/lib/docker/overlay2

You’ll find a directory with a weird name (something like 943d40f825b59697c59bf5f822cb5a479b0610e1e51c535c557418be830d6138). You can run:

And you’ll find something that looks a lot like your Ubuntu machine’s root directory.

Let’s copy this image to the /tmp/container directory:

Now we have an Ubuntu image ready to use.

2. Run an isolated process on top of this image.We’ll use /bin/bash as the isolated process and the unshare executable, which uses the unshare syscall to run a process with separate namespaces.

# print current namespace IDs

ls -la /proc/1/ns

unshare –fork –uts –pid –mount-proc /tmp/container/bin/bash

3. Compare the previous namespace list with the current:

ls -la /proc/1/ns

You’ll see that the UTS, mount, and PID namespaces have different IDs and that the PID namespace took place because there are no host-related processes visible within this container:

You no longer have access to the root PID namespace. Since in Linux everything is a file, listing all the processes managed in /proc:

Will give you a sense of a « fake » proc directory.

To further isolate the container, use the chroot command:

hostname Pentera-Demo

chroot /tmp/container

Voilà! Our container is ready.

👉 Watch the demo video of the entire configuration

With this knowledge, we have a better understanding of the role of namespaces and chroot and can better grasp containers’ security boundaries. It’s a simple example, so we didn’t cover Linux capabilities, cgroups, and more, but you get the idea.

The point is now that you understand how namespaces are set, you can ask all sorts of security research questions with a lot more insight:

1. Can I move from one namespace to another?
2. What can I do within my namespace that will affect the root namespace?
3. What might be a sensitive mount path?
4. Can I use symlinks to get access to files outside of my chroot? (CVE-2024-21626)
5. What can I do with a SUID file or a file with capabilities that are part of the container’s image?

These kinds of questions will help you move the research to its next step. By understanding more about different technologies, how they are built, and becoming better at asking edge-case questions, you’ll refine your research skills. That’s an example of what research intuition feels like.

Security Research Intuition: The Cure for Research Dead Ends

A poor research decision can lead to vast amounts of time wasted on irrelevant research directions. By adopting the « build before you break » mindset, you develop an intuition that guides you to the most promising areas to investigate. With each new system you study, your intuition grows stronger, and soon, you won’t need to build it from scratch each time. It won’t take long before you’ll have a much better feel for where the problems might lie.

By adopting this approach and leveraging the power of research intuition, you’ll be able to enhance your skills, apply better techniques, and make stronger decisions. At various crossroads in your research, you need to make significant choices that will determine whether your research is a success or hits a dead end.

Want to see how real-world techniques are applied at scale? Explore Pentera’s platform to learn more about continuous testing in live environments.

Originally published by The Stack as « The Building Blocks of Destruction: How to Develop Your Security Research Intuition. »