Generate Reports

The EmbedOps CLI uses the eo report to send metrics to the EmbedOps Platform so they can be tracked over time. They're most commonly expected to be used in a CI pipeline, as a means of tracking either official releases or per-commit changes.

Memory Usage

The eo report memusage [path to the build log file] sends memory usage information to the EmbedOps platform from CI builds. Memory usage is parsed from the output of standard size tools normally included with your toolchain, and added to the Memory usage's Historical view on the EmbedOps Platform.

Testing Out Memory Usage Reports

To simply test out this functionality, you can follow the CLI Quick Start and then add memory usage reports to that project using the examples in the steps below.

Prerequisites

Have an EmbedOps project registered on the EmbedOps Platform
Building the project using one of the toolchains that supports outputting the size of the final build file in berkeley format
- arm-none-eabi-size is one example
The EMBEDOPS_API_REPO_KEY is set on the CI that matches the EmbedOps Platform's API Key for that project

Steps

Add to the build process

Update the build process so it outputs the size of the final build file in berkley format.

Arm GCC & CMake

Configure what the size command is for your build:

    find_program(CROSS_GCC_PATH arm-none-eabi-gcc)
    get_filename_component(TOOLCHAIN ${CROSS_GCC_PATH} DIRECTORY)
    set(CMAKE_SIZE              ${TOOLCHAIN}/arm-none-eabi-size   CACHE STRING "arm-none-eabi-size")

Add the below function to your CMake build system:

# print_memory_metrics()
# Description:
#   Adds a custom target that executes the toolchain's CMAKE_SIZE tool for
#   the specified target.
# Parameters:
#   target - IN - Target name as specified to the add_executable() command.
function(print_memory_metrics target)
    add_custom_target(${target}_memory_metrics
                        ALL
                        COMMAND ${CMAKE_SIZE} --format=berkeley ${target}.elf
                        DEPENDS ${target})
endfunction()

Then call this particular function in your main CMakeList.txt. E.g. for the STM32 Quick Start project, this function call is added during the non-unit test build:

CMakeList.txt

if(NOT BUILD_UT)
    # When building for hardware, we are creating an executable.
    add_executable(${APPLICATION_NAME})
    print_memory_metrics(${APPLICATION_NAME})

    # Include everything
    add_subdirectory(app)
    add_subdirectory(lib)
    add_subdirectory(utility)

    # Link application against STM32L4 device library
    target_link_libraries(${APPLICATION_NAME}
        STM32L4
    )
else()

Add to CI pipeline

Update build script for your project to save the build log to a file. E.g. if your build script simply calls make, update to make | tee build.log
Verify the CI pipeline step that runs the build script makes the CI variables available to the build script

CI Variables

CI variables are environment variables that your CI provider sets for any scripts, etc. that run during the CI pipeline. If you're unfamilar with CI variables, Gitlab describes their predefined variables and some uses here.
Update build script for your project so during a CI build, it sends the memusage.

Example resulting bash script that is called in gitlab CI with CI variables available:

build.sh

# add the | tee to the build command so it
# saves the output of the build into the build.log file
make | tee build.log

# use the predefined CI variable $CI to see if we're running in a CI pipeline
if [ "$CI" = "true" ]; then
    # tell embedops to parse the build.log file that was generated above
    # and send the results to the EmbedOps Platform
    eo report memusage build.log
fi

The goal here is to create a log file that saves the build command output, including the berkley formatted size information. This is then fed to the eo report memusage, which parses the log file and sends the information to the EmbedOps Platform.

Push these changes to the CI provider, and the next time the pipeline runs, you'll be able to see the results on the EmbedOps platform. And over time, the EmbedOps dashboard will start to look like this:

Analyze Memory vs. Report MemUsage

If you've been exploring documentation, you may have noticed that the Analyze Memory how-to also mentions calculating memory usage. This is a different tool! The memusage report sends basic size information up to the EmbedOps platform to track overall memory usage over time. The analyze memory functionality provides a more in-depth view, and is a one-off calculation based on your current build.

Multiple Builds

If you have multiple build files that you want to track, you can simply add their size output in the build log and EmbedOps will automatically tag the sizes with the filename on the EmbedOps Platform dashboard. The screenshot above shows how a debug and release build can both be tracked in the same dashboard.

Unit Tests

The eo report unittest [path to JUnit XML file] parses a JUnit XML file that has recorded the results from the run of a unit test suite. It then uploads the info to the EmbedOps platform. This provides the ability to see unit test count and pass/fail metrics over time as the project grows

Testing Out Unit Test Reports

To simply test out this functionality, you can follow the HIL Quick Start and then add unit test reports to that project using the examples in the steps below.

Prerequisites

Have an EmbedOps project registered on the EmbedOps Platform
The EMBEDOPS_API_REPO_KEY is set on the CI that matches the EmbedOps Platform's API Key for that project
Project uses unit tester that supports generating a JUnit XML file. Examples:
- Google test
- Ceedling

Setup Unit Tests

EmbedOps supports adding unit tests with ceedling, go to the Add Unit Tests How-to to learn more

Steps

Update how the unit test suite is run in CI so it outputs a JUnit XML file
1. If you're using ceedling, this can be added by updating the ceedling project.yml to enable junit_tests_report. For Example:
```
:plugins:
    :load_paths:
        - "#{Ceedling.load_path}"
    :enabled:
        - stdout_pretty_tests_report
        - junit_tests_report
```
Info

If you're using the EmbedOps unittest Template, this is already setup!
Verify the CI pipeline step that runs the build script makes the CI variables available to the build script. The default CI variables are used by EmbedOps to link unit test runs to the corresponding CI pipeline and job.
Update the unit test suite script to send the eo report unittest [path to XML file] with the generated JUnit XML file

Example script being run in CI:

unit_test_suite_run.sh

JUNIT_FILE_PATH="test/path/result.xml"

# run unit tests and set output
scripts/run_tests.sh "${JUNIT_FILE_PATH}"

# send report to EmbedOps
if [ "$CI" = "true" ]; then
    eo report unittest "${JUNIT_FILE_PATH}"
fi

Push these changes to the CI provider, and the next time the pipeline runs, you'll be able to see the results on the EmbedOps platform: