EC: Phone Home

Precursor is a hardware reference platform that is the first iteration of Betrusted. Betrusted aims to be an accessible communications device that can ensure your conversations are private, and Precursor plants the seeds of that idea.

The Betrusted platform is designed to be a secure enclave with a screen, and while it’s nice to have a verifiable platform on which to keep secrets, it’s important to be able to communicate with the outside world. It is for this reason that the Betrusted architecture is divided into two domains: Trusted, and Untrusted. Each of these domains has its own CPU core running on a dedicated FPGA. The Trusted domain runs on a relatively large Xilinx Spartan7 FPGA is referred to as the System-on-Chip, or SoC. The Untrusted domain runs on a much smaller ICE40 which is called the Embedded Controller, or EC.

The COM bus is the only electronic means of communication with the outside world, and it’s routed completely through the EC. I’d like to discuss the reasoning behind this design decision, as well as some of the practical implications.

The Network is Untrustworthy

You may notice from the diagram above that the EC is in the untrusted domain. The idea here is that the network is completely untrustworthy – once data exits the SoC via the COM bus, assume that it can be tampered with. Therefore, nothing of value should leave the SoC without first encrypting it. Once the valuable data is encrypted, it doesn’t matter how public it is. This frees the EC from needing to worry about trusting anything, which greatly reduces the resources required.

Networking traffic that flows across the COM bus is simply raw Ethernet frames. This allows us to use standard, off-the-shelf networking stacks such as smoltcp to get full network support by simply gluing projects together.

Xous-NoMMU: A Trimmed-down Interface

The EC is a very limited FPGA. While we could fit a full MMU in the tiny 5000-LUT FPGA, doing so would use a large number of resources for very little benefit. While some projects have gotten a full Linux running on an ICE40, this requires external RAM that is not present on the EC, which means any MMU implementation on an ICE40 would need to waste at least two pages of memory (8 kB) on pagetables alone. The ICE40 comes with 128 kilobytes of on-chip SRAM, which means 16% of the available memory would need to be dedicated to an MMU.

Even worse, a context switch on RISC-V requires saving 32 registers, then restoring a different set of 32 registers. This can take a relatively long time on the slow EC, so it’s best to avoid doing a gratuitous context switch whenever possible. Therefore, there should be no ticktimer generating interrupts at regular intervals.

Because of this waste of resources, and because the EC is an inherently untrusted device, we instead opt for a stripped down version of Xous. We’re mostly interested in interrupts, which allow the CPU to pause what it’s doing to service a request from either the wifi or the SoC. There is no memory management or processes, but we do get a similar interface to what we have on the full Xous build.

Controlling the Network

The SoC is very power-hungry, so it ideally would be off most of the time. The EC is powered by an ICE40, which is extremely power-efficient. Likewise, the WF200 was chosen for very low quiescent current.

The EC is responsible for managing the WF200 which it does by calling the driver software from SiLabs. This software knows how to load firmware, and has several functions to assist with tasks such as joining wifi networks and sending and receiving Ethernet frames.

The EC also relays wifi management commands from the SoC. For example, the SoC would send a “Join” command to connect to an access point, along with the SSID and password for that access point.

Because we’d like the SoC to spend most of its time asleep or powered off, the EC actually contains a small DHCP server. That is, while most packets will be forwarded on to the SoC, the EC is able to obtain its own IP and renew the DHCP lease.

With this architecture, the EC is able to autonomously maintain a connection to wifi even if the SoC is powered off and the access point goes away.

Emulating the EC (and Everything Else)

The diagram above depicts a complicated system, and indeed we didn’t have this system emulated until relatively recently. However, thanks to Renode, we’re able to emulate the whole thing.

The EC is just a standard RV32I chip running at around 8 MIPS. The WF200 is a SPI device. It’s connected to the EC via the WIFI bus, and the EC is connected to the SoC via the COM bus. The SoC has its own view of the COM bus, which we also implemented.

Antmicro have also been working on getting TUN/TAP support working on Windows, meaning all of the supported platforms can now connect an emulated Betrusted device to an Ethernet network and perform networking operations.

Standard Networking Support

Thanks to the relatively standard nature of these projects, we’ve been able to add support for TCP connections in the Xous port of Rust’s libstd. The EC is instrumental in controlling the network, and thanks to our Renode models we can inspect and instrument the code very thoroughly. We can load real firmware into a virtual environment and debug it to ensure it’s doing exactly what we think it is. We can also run it in a CI environment to ensure code works when we make changes. And thanks to the implementations we’ve made, the following actually works now when compiling for our riscv32imac-unknown-xous-elf target:

use std::io::prelude::*;
use std::net::TcpStream;

fn main() -> std::io::Result<()> {
    let mut stream = TcpStream::connect("betrusted.io:80")?;
    let mut output = vec![];

    stream.write_all(b"GET / HTTP/1.1\nHost: betrusted.io\nConnection: Close\n\n")?;
    stream.read_to_end(&mut output)?;
    println!("Data: {}", std::str::from_utf8(&output).unwrap_or("<error>"));
    Ok(())
}

Future Work

While we have TcpStream working, we don’t yet have TcpListener or UdpSocket working. These are the next tasks to be done once we have a need for them.

The Betrusted project, including the Xous operating system, are made possible thanks to financial assistance from NLNet and the NG10 Privacy & Trust Enhancing Technologies Fund. Thank you to them for their support.

This work was funded by NLNet.