Skip to content

Run Fuzzing

Fuzz testing or fuzzing is a software testing technique to discover vulnerabilities, crashes, and unexpected behaviors in a program. EmbedOps CLI can be used to add a fuzz testing template to get you started on running a fuzzer on your project.

Prerequisites

Steps

1. Connect Your Project To The EmbedOps Platform

Running the following command in the root of your project connects it to the EmbedOps platform and creates the configuration files for a dev container and a Continuous Integration (CI) pipeline.

host> eo init

2. Add The Fuzzing Template

Running the following command adds the scripts to run a fuzzing example to your project. It will prompt for the pipeline provider, and what name you'd like the fuzzing example directory to be called. This example directory can be called anything, but ideally you'll want to use a new folder name (i.e. one that doesn't yet exist).

host> eo add fuzzing

Once these files are added you'll see the following:

fuzzing
├── ci
│   ├── fuzzing.gitlab.yml
│   ├── fuzzing.sh
│   ...
└── fuzzingExample
    ├── CMakeLists.txt
    ├── ReadMe.md
    ├── corpus
    │   ├── overflow_int64_seed
    │   ├── overflow_uint32_seed
    │   ├── starting_seed_1
    │   └── string_finder_seed
    ├── main_no_crash.c
    ├── main_with_crash.c
    ├── more_fuzzing.c
    └── more_fuzzing.h

  1. A ci folder that contains the fuzzing.sh script. This is what gets called when you run eo run fuzzing. The fuzzing.sh script uses CMake to build the example project and run the fuzzing target.

  2. The folder name that you specified in eo add fuzzing. This folder contains the following:

    • Directory named corpus with a few files. These files are used as initial seed data for the fuzz run
    • A CMakeLists.txt used as part of the cmake build process to build the fuzzing project with llvm (clang). libFuzzer is used under the covers, and thus whatever is desired to be fuzzed must be built with clang.

3. Run The Fuzzing Example

  1. Open your project in Visual Studio Code.

  2. If the Dev Containers extension has not been installed, go to the Extensions tab on the left column, search for Dev Containers, and install it.

  3. Open the Command Palette with command + shift + P, search for Dev Containers: Reopen in Container, and press enter. This will create a development container with the directories and files in your project and the necessary dependencies to run the fuzzing example.

  4. Open the Command Palette again, search for View: Toggle Terminal, and press enter. This will open a terminal in the development container.

  5. Run the fuzzing example with:

    container> eo run fuzzing

  6. A few reports are generated. All this information is sent to EmbedOps if using a CI run. (i.e. on GitHub Actions, etc)

    • fuzz_output.log
      • Copy of the build log that was outputted to the terminal screen.
    • coverage.lcov
      • Line and Branch coverage file.

Note: You can also run this exact command outside of the devcontainer. To do so you'll want to install the devcontainer CLI:

Info

Install the devcontainer CLI and run it outside of the container.

macOS
host> brew install npm
host> sudo npm install -g @devcontainers/cli
host> devcontainer up --workspace-folder .
host> eo run fuzzing

For installation on other OSes, please follow the official installation guide

4. Setting and Overriding Configurations

The eo run fuzzing command is a wrapper around eo fuzz. eo run fuzzing is intended to be used with the example template and will look for and execute ci/fuzzing.sh which will eventually execute eo fuzz. The eo fuzz command takes optional arguments that specify things like corpus directory, total fuzzing runtime, maximum crashes allowed, and more. To see a full list of possible arguments run: 

host> eo fuzz --help

eo fuzz has an option called --bin-path. This option specifies the binary you'll run fuzzing on. If this option is not present, then eo fuzz will attempt to locate the ci/fuzzing.sh script and run that.

On a normal run, eo fuzz pulls all of its configuration from the EmbedOps platform Fuzzing Configuration page for the repo (created when you ran eo init).

Fuzzing Configuration Page

You can reach this page by running eo open at the root of the repo (at the same level the .embedops directory exists) and navigate to Security > Fuzzing Configuration

Whatever is set on this page will be used for the given eo fuzz run. Any of these arguments can be overridden locally on the command line. To see the options available to override use eo fuzz --help. Additionally, eo fuzz attempts to ensure that there is no architecture mismatch between fuzzing binaries that have been built and the system (for example, one built on the devcontainer won't work locally). If this is giving issues, you can override this check by adding the --ignore-arch 1 option to eo fuzz.

5. Run The Fuzzing Example With Crashes

The fuzzing example is configured to run a program without crashes. To run fuzzing with crashes, update fuzzingExample/CMakeLists.txt as shown below to point to main_with_crash.c (Here this is assuming that you've named your directory for fuzzing from Step 2 fuzzingExample. Replace fuzzingExample below with whatever you've named it from Step 2.):

--- a/fuzzingExample/CMakeLists.txt
+++ b/fuzzingExample/CMakeLists.txt
@@ -13,7 +13,7 @@ add_executable(${LOCAL_PROJ_NAME})

 # Add all the files (here you can add your own files or add more)
 target_sources(${LOCAL_PROJ_NAME} PRIVATE
-  ${CMAKE_CURRENT_LIST_DIR}/main_no_crash.c
+  ${CMAKE_CURRENT_LIST_DIR}/main_with_crash.c
   ${CMAKE_CURRENT_LIST_DIR}/more_fuzzing.c
 )

You can update the file in a text editor or with the following bash command:

host> sed -i.bak 's/main_no_crash/main_with_crash/' fuzzingExample/CMakeLists.txt

Then, repeat Step 3 to rebuild and rerun the fuzzing example.

6. Fuzz Reports on the EmbedOps Platform

The fuzz data is automatically sent to the EmbedOps Platform while running.

Fuzzing Report Example

You can reach this page by running eo open at the root of the repo (at the same level the .embedops directory exists) and navigate to Security > Dashboard. Running eo fuzz in your local terminal will generate a single data point in the report. If ran in CI, you will get historical data of fuzzing runs for that branch.

Fuzzing Report History Example

7. Run Fuzzer On Your Project

This example is a quick intro on how to use fuzzing. Use the existing CMake file to compile certain sections of your own code base with Clang and run fuzzing on your own code. It is suggested to start small as trying to compile your entire project with Clang may take some time since you'll likely need to fake out different compiler-specific and embedded targeted items.

One example would be taking a single static function and getting that to compile with Clang. Then call it from LLVMFuzzerTestOneInput() and watch it fuzz.

Because eo run fuzzing relies on libFuzzer, you must compile with a version of Clang that contains libFuzzer. The compiler options specified in the example CMakeLists.txt (specifically the -fsanitize=fuzzer, -fcoverage-mapping, and -fprofile-instr-generate MUST be used when building). In addition, compiling with debug symbols on (-g) will make it easier to debug any crashes that fuzzing finds.

Please refer to the fuzzingExample/ReadMe.md (the ReadMe.md file generated in Step 2) to learn more about how to add fuzzing to your project.