Skip to content

Hardware-in-the-Loop (HIL) Quick Start

This page introduces the EmbedOps approach to Hardware-in-the-Loop (HIL) testing.

You will learn how to:

  • Connect a repository to the EmbedOps platform
  • Use an EmbedOps template to add HIL-specific assets
  • Provision a HIL Gateway device
  • Run a HIL test

Recommendation

This HIL Quick Start begins similarly to the EmbedOps CLI (eo) Quick Start. We recommend completing that Quick Start first if you’re new to eo.

Hardware Requirements

This Quick Start requires a Rasbperry Pi 4 and an nRF53 Development Kit. If you don't have this hardware, you can purchase a kit on DigiKey that includes everything you need.

1. Follow Initial Steps from the CLI Quick Start

If you haven’t yet set up eo or familiarized yourself with the basics of EmbedOps, follow Steps 1–4 of the CLI Quick Start to:

  1. Sign up for an EmbedOps account
  2. Install all dependencies
  3. Install the EmbedOps CLI
  4. Log in with eo

Once these prerequisites are met, you can proceed here.

Dev Container vs. Host machine

  • Some commands in this Quick Start are run on your host machine, and some are run in a Dev Container. To differentiate, host> and container> will be used as the terminal prompts to clarify where the command is run.

2. Fork Template Repo

EmbedOps HIL supports creating and running HIL tests on the nRF53 platform. For this Quick Start, fork our nRF53 HIL Quick Start. Forking the project into your own user or organization group within your Git provider is highly recommended so that you can push changes. For the example project, this quick start will assume you're using gitlab.

3. Connect Your Repo to the Platform

From the root of your forked repository, run:

host> eo init

It will prompt you for your preferred CI provider and linux distribution.

For the example project with gitlab, set pipeline provider to "gitlab" and the linux distribution to "ubuntu"

This step associates your repository with the EmbedOps platform and create an initial Dev Container and CI setup. As part of associating your repo with the EmbedOps platform, it generates a file named:

.embedops/repo_id.yml
repo_id: 93080dde-09ee-4ec7-a269-50896a8a3ed6

This file is necessary for HIL functionality.

Info

Commit and push this .embedops/repo_id.yml file so that everyone on your team can leverage the same EmbedOps features.

After running the above commands, you now have:

  • a repo_id.yml that will connect this repository to the EmbedOps platform
  • a template-settings.yml that saves what you selected above.
    • For the example project, the pipeline provider will be set to gitlab
  • an initial empty dev container configuration
    • For the example project, it will use an ubuntu image as the base container
  • an initial empty CI configuration
    • For the example project, it will generate a gitlab-ci.yml file and the /ci folder

4. Apply HIL template

EmbedOps provides a hil template that adds the files needed for HIL testing:

host> eo add hil

The CLI prompts you to select a target platform. Choose D5_NRF5340_HIL_KIT for this Quick Start:

Select a target platform::

[x] D5_NRF5340_HIL_KIT
[ ] Custom


j/k, up/down: select • enter: choose • q, esc: quit

This command creates files in your repository, including:

nrf-hil-quickstart/
├── .embedops/
│   └── hil
│       └── config.yml
└── hil/
    └── hil_sdk/
        └── ...
    ├── README D5 HIL Kit.md
    ├── conftest.py
    ├── test_demo_app.py
    └── ...

Of particular importance is the following configuration file:

.embedops/hil/config.yml
hil_root: hil
hil_extras:
  - name: build/zephyr/merged_domains.hex
    external_module: false

This YAML file configures the HIL setup for your project, specifying which directory contains your HIL tests and which extra files (such as firmware .hex files) to include.

As indicated in hil/README D5 HIL Kit.md:

  • conftest.py - Implements test fixtures for the various interfaces
  • test_demo_app.py - Multiple tests for all demo app features

HIL Configuration

By default, hil_root is set to hil, which is how the example project is organized. On your own projects, you can easily modify it if you organize your tests differently. The hil_extras can also be modified with any additional files you’d like EmbedOps to include when running tests on your hardware. To explore more yaml configurations, go to the YAML Config page.

5. Provision a Device

5a: Generate the Image for the Raspberry Pi

To provision a new HIL device on the EmbedOps platform, run the following from the root of your repository (where .embedops exists):

host> eo hil provision

It will take a short while for it to download the gateway image, and return when it finishes. Below is a GIF demonstrating the terminal output:

Example of running eo hil provision

When provisioning succeeds, EmbedOps:

  1. Registers a HIL device on the EmbedOps platform.
  2. Generates a gateway-image.img file used to flash an SD card.

Use your favorite tool--such as Balena Etcher or Raspberry Pi Imager-to burn gateway-image.img onto an SD card. Insert the SD card into your Raspberry Pi.

Set up your hardware now by connecting your raspberry pi to power, ethernet, and to the nRF5340 dev kit via USB. Setup should look like this:

Example HIL Gateway setup

Ethernet required

The current gateway image requires an Ethernet connection. Make sure your Raspberry Pi is connected via Ethernet for the HIL Gateway to function.

Finally, power on your Raspberry Pi so the EmbedOps platform can communicate with your device for HIL testing.

Provisioning More Devices

To provision more raspberry pi's to make a whole fleet of HIL setups, you'll need to run the eo hil provision command and burn the new gateway image onto a new SD card for each pi.

5b: See Available Devices in the Fleet

You can see the gateway is provisioned correctly by going to the EmbedOps platform and looking at the HIL Fleet page. You can see your provisioned device available online:

EmbedOps Platform screenshot of the available device

To check the status of devices in your fleet via embedOps CLI, run:

host> eo hil fleet

you should see output similar to:

host> eo hil fleet       
Name     Source               Location             Status              
D-1      Trial Project        project              AVAILABLE

Troubleshooting

If either of the above steps display the device as offline, verify the raspberry pi has an ethernet connection to the internet. NOTE there is some latency between when a device is connected/disconnected and its displayed status

EmbedOps Platform screenshot of an offline device

6. Build the Project with a Dev Container

The nRF53-based project in this Quick Start includes source code that generates a HEX file to flash onto the nRF53 target.

6a: Adjust the Dev Container Configuration for Nordic

Edit the start of the file .devcontainer/devcontainer.json (generated from eo add init in Step 3) with the following:

.devcontainer/devcontainer.json
{
    "image": "nordicplayground/nrfconnect-sdk:v2.5-branch",
    "workspaceMount": "source=${localWorkspaceFolder},target=/workdir/project,type=bind",
    "workspaceFolder": "/workdir/project",
    "onCreateCommand": "echo $(nrfutil toolchain-manager env --as-script) >> ~/.bashrc"
}

This configuration change:

  1. replaces the default image for the Dev Container to one using the Nordic nRF Connect SDK, so you can build your nrf5340's project with the standard nRF Connect SDK.
  2. bind the local workspace folder to the /workdir/project folder
  3. Update the default workspace folder in the container to /workdir/project
  4. when the container is created, it will run a nrfutil command that will setup the toolchain environment for the nRF5340

6b: Build the Project

For the example project, you can now build the project yourself manually in the Dev Container. To get into the Dev container, go to the Dev Container extension in VS Code and click the "+" at the top to open the drop down selection. Select "Open Current Folder in Container", and it will auto create the dev container and restart VS Code with the terminal running in the container. To learn more about dev containers in VSCode, see these Dev Container docs.

Workspace Folder

When you start a terminal in VSCode inside the Dev Container, notice it drops you into the /workdir/project folder that you defined in the devcontainer.json above!

Once you're in the Dev Container, run:

container> west build --pristine --board nrf5340dk_nrf5340_cpuapp

Running commands outside the Dev Container

  • If you are NOT using VS Code, you can install the devcontainer CLI and run commands like so: 
    host> devcontainer up --workspace-folder .
host> devcontainer exec --workspace-folder . west build --pristine --board nrf5340dk_nrf5340_cpuapp

This command generates a build folder. The key file for flashing the nRF53 board is build/zephyr/merged_domains.hex, which you saw earlier at Step 4 as the name of the hex file in hil_extras.

7. Use eo for HIL

With your project now set up for EmbedOps HIL, you can run HIL-related commands using the eo CLI.

7a: Run HIL Tests

To trigger your HIL tests, simply run:

container> eo hil run

Logging Results Locally

By default, EmbedOps doesn't log the output of the hil tests locally, it simply prints them to the terminal. The logs will still be available after the run on the EmbedOps Platform, but if you want to keep them locally, you can change the command to something like this eo hil run 2>&1 | tee hil.log to store the results in the hil.log file.

The eo add hil command (from Step 4) added Python files containing the logic needed to program the device and execute tests. If everything is set up correctly, you should the HIL run start, with logs of the tests from the Gateway:

Example of running eo hil run

The eo hil run command

  • Uploads all necessary assets for the tests
  • Establishes communication with the HIL Gateway
  • Executes the specified tests
  • Returns the output from the Gateway

There's a couple flags you can set that allow you to limit which tests you run and where you run them.

  • -m [string]: Marker that will restrict which tests are run to only tests marked with @pytest.makr.<marker>. See test selection
  • -d device1,device2,...,deviceN: Comma delimited list of HIL Gateway device names to restrict the run to

7b: Examining HIL Test Results

When the run command finishes, you'll be able to see the final number of tests passed in the terminal. You can also view all the logs for this HIL test run on the EmbedOps Platform. On the EmbedOps Platform, go into the nrf53-hil-quick-start repo and select "HIL Tests" to see a list of the most recent HIL runs, like below:

Example of HIL run results

Clicking into a particular run, you can then see all the logs that were output on the terminal during the eo hil run command:

Example of a HIL run logs

Next Steps

Congratulations on completing the HIL Quick Start! You’ve successfully configured your repository for HIL and run your first hardware-in-the-loop tests.

To dive deeper into the capabilities of our HIL framework, refer to the links on the left. There, you’ll find:

  • How-To Guides for advanced configurations
  • Reference Documentation for all HIL commands and features
  • HIL SDK Details (one of the modules auto-imported by the eo add hil command)