Shadowgraph
TrendingAbout
Sign in

carlosedp/riscv-bringup

Risc-V journey thru containers and new projects

Shell

281

151 commits

updated May 17, 2024

See the code

README

RISC-V bring-up tracker

The objective of this repository is to track the progress and pre-requisites to allow containers and Go applications on RISC-V.

The repo also hosts multiple files and images suited for the SiFive Unmatched and QEmu VMs.

There is a companion article available on https://medium.com/@carlosedp/docker-containers-on-RISC-V-architecture-5bc45725624b.

This page is also linked from http://bit.ly/riscvtracker.

If you like this project and others I've been contributing and would like to support me, please check-out my Patreon page!

Contents

RISC-V Unleashed SBC, Virtual Machines and pre-built binaries

To make the development easier, there are Qemu virtual machines based on Debian and Ubuntu with some developer tools already installed.

For the SiFive Unmatched, there is a prebuilt SDcard image at https://github.com/carlosedp/riscv-bringup/releases/download/v1.0/UbuntuHippo-RISC-V.img.gz

For QEmu, there are three distributions of RISC-V pre-packaged VM images:

The user is root and password riscv. For more information, check the readme.

A prebuilt Go 1.16 tarball can be downloaded here.

If required to build the complete boot stack composed of OpenSBI, U-Boot, Linux, checkout the guides for SiFive Unmatched, SiFive Unleashed and Qemu.

To run Go on the VM or board, install with:

# Start the VM
./run_debian.sh

# Download Golang tarball
wget https://github.com/carlosedp/riscv-bringup/releases/download/v1.0/go1.16.5-riscv64.tar.gz

# In the VM, unpack (in root dir for example)
tar vxf go1.16.5-riscv64.tar.gz -C /usr/local/

# Add to your PATH
export PATH="/usr/local/go/bin:$PATH"

# Addto bashrc
echo "export PATH=/usr/local/go/bin:$PATH" >> ~/.bashrc

To run Docker on your RISC-V Debian or Ubuntu environment, download a deb package and install with sudo apt install ./docker-v20.10.2-dev_riscv64.deb.

For other distros get the tarball here and unpack to your / dir. If the docker service doesn't start on install script, re-run systemctl start docker.

Docker-compose install instructions 
# In Debian image
sudo apt-get install python3 python3-dev python3-pip

# For Fedora image
sudo dnf install python3-devel

sudo pip3 install docker-compose

To test it out after install, just run docker run -d -p 8080:8080 carlosedp/echo-riscv and then curl http://localhost:8080.

There are a couple of projects that build on RISC-V in my go-playground repo.

There is also a Podman package here in both .deb and .tar.gz. Check more info on build-podman-env.md to build the package from scratch.

Golang

Core Golang

Golang has been upstreamed as an experimental architecture in Go 1.14. There are no binaries published officially but the releases section has the tarball.

To build Go from source, check build-golang.md.

Go Std Libraries

External deps

Docker and pre-reqs

To build a complete Docker container stack, check the build-docker-env.md document.

Downloads for prebuilt packages are available on https://github.com/carlosedp/riscv-bringup/releases.

Libseccomp

https://github.com/seccomp/libseccomp

Builds fine from master branch. Will be released with riscv64 support on 2.5.

Runc

https://github.com/opencontainers/runc

  • Upstreamed / Works
  • Depends on CGO (to build nsenter)
  • Support buildmode=pie
  • Add riscv64 to libcontainer/system/syscall_linux_64.go
  • After upstreaming, update x/sys and x/net modules
  • libseccomp-dev
  • apparmor - ($ sudo aa-status -> apparmor module is not loaded.)
  • Add to CI

Crun

https://github.com/giuseppe/crun

No changes required, builds fine even without Kernel support for seccomp. Depends on libseccomp.

  • Upstreamed / Works
  • Rebuild with libseccomp
  • Add to CI

Containerd

https://github.com/containerd/containerd/

Docker

Docker cli

https://github.com/docker/cli

  • Upstreamed / Works (must be built from master)
  • Update x/sys and x/net modules in vendor. PR
  • Add riscv64 to manifest annotation. PR#2084
  • Add support for riscv64 on binfmt. PR#21
  • Docker for Mac - Add RISC-V binfmt. PR#4237
  • Add to CI

Docker daemon

https://github.com/moby/moby

Dependency lib PRs:

docker-init

https://github.com/krallin/tini

No changes required. Just build and copy tini-static to /usr/local/bin/docker-init

  • Upstreamed / Works

docker-proxy

No changes required. https://github.com/docker/libnetwork/cmd/proxy

  • Upstreamed / Works

Alternative is run dockerd as: sudo dockerd --userland-proxy=false

Issues

Podman - libpod

https://github.com/containers/libpod

Podman is a library and tool for running OCI-based containers in Pods

CNI Plugins

https://github.com/containernetworking/plugins

  • Builds and runs.

CNI Plugins Issues

Base Container Images

Docker Community "per-architecture" repository

Docker images for projects

OpenFaaS:

Prometheus:

Traefik:

Kubernetes:

Kubernetes images are multi-arch with manifests to arm, arm64, amd64, riscv64 and ppc64le. Some version mismatches due to Kubernetes hard-coded version check for CoreDNS and etcd.

Misc Images:

Kubernetes

https://github.com/kubernetes/kubernetes/

Building and deploying Kubernetes or K3s on RISC-V is detailed on a dedicated readme. There is a build script(build_images.sh) for custom images in kubernetes dir.

To Do:

  • Cross-platform builder image. Update kubernetes/build/build-image/cross adding riscv64 toolchain. Depends on Ubuntu crossbuild-essential-riscv64 be available.
  • Add riscv64 to ./build/pause Makefile. Depends on Go image with RISC-V support and cross-platform image.
Updating dependencies 
# Update dependency
./hack/pin-dependency.sh github.com/mindprince/gonvml
# Update vendor dir
./hack/update-vendor.sh

# Build all main binaries
make KUBE_BUILD_PLATFORMS=linux/riscv64

# Build specifiv binaries
make WHAT=./cmd/${bin} KUBE_BUILD_PLATFORMS=linux/riscv64

# Binaries will be placed on _output/local/go/bin/linux_riscv64/

K3s

https://github.com/rancher/k3s/

K3s build depends on deploying external etcd database since sqlite embedded DB requires CGO. Another requirement is running k3s with Docker and crun as daemon (see Docker install) since runc, the default runtime for K3s requires CGO as well.

For more info, check file [kubernetes/Readme.md#K3s]

Bump:

Future
  • Embed database
  • Embed runtime

Additional projects / libraries

ETCD

https://github.com/etcd-io/etcd

Build with go build ., run with ETCD_UNSUPPORTED_ARCH=riscv64 ./etcd.

OpenFaaS

OpenFaaS is already upstreamed but still does not build images for RISC-V so I've built them and pushed to my DockerHub as links below. Here are the instructions to deploy OpenFaaS on your RISC-V host or VM.

The PRs do not add functionality to cross-build the images for RISC-V yet since the base images still don't support the architecture. Check the build_images.sh script to build the images manually.

Faas-cli

https://github.com/openfaas/faas-cli/

Faas-swarm

https://github.com/openfaas/faas-swarm

Nats-streaming-server

https://github.com/nats-io/nats-streaming-server

No changes required

Bbolt

https://github.com/etcd-io/bbolt

Pty

https://github.com/kr/pty

Prometheus

https://github.com/prometheus/prometheus

Builds successfully with make build.

Promu

https://github.com/prometheus/promu

AlertManager

https://github.com/prometheus/alertmanager

Already builds successfully with make build.

Traefik

https://github.com/containous/traefik

Building and running 
rm -rf static/ autogen/
make generate-webui
go generate
mkdir dist
GOARCH=riscv64 GOOS=linux go build -o dist/traefik ./cmd/traefik
docker build -t carlosedp/traefik:v2.1-riscv64 .

See also a complete Traefik Dockerfile for cross-compiling.

To run an example stack with Docker Compose, create the file below and start it with docker-compose up -d. To test, you can open the address http://[IP]:8080/dashboard or curl http://localhost:8080/api/rawdata. Prometheus metrics are exposed on http://localhost:8080/metrics.

Create a docker-compose.yml

version: '3'

services:
  reverse-proxy:
    # The official v2.0 Traefik docker image
    image: carlosedp/traefik:v2.0-riscv64
    # Enables the web UI and tells Traefik to listen to docker
    command: --api --providers.docker --metrics.prometheus=true
    ports:
      # The HTTP port
      - "80:80"
      # The Web UI (enabled by --api)
      - "8080:8080"
    volumes:
      # So that Traefik can listen to the Docker events
      - /var/run/docker.sock:/var/run/docker.sock
  whoami:
    # A container that exposes an API to show its IP address
    image: carlosedp/whoami:riscv64
    labels:
      - "traefik.http.routers.whoami.rule=Host(`whoami.docker.localhost`)"

Run with:

docker-compose up -d

SQlite

SQLite builds on RISC-V but requires replacing its building files.

Repository mirror: https://github.com/CanonicalLtd/sqlite

  • Upstreamed / Works
  • Update config.guess and config.sub to newer version. Posted to mailing list.

Go-Jsonnet

https://github.com/google/go-jsonnet

Github Hub tool

https://github.com/github/hub

Labstack Echo Framework

https://github.com/labstack/echo

Labstack Gommon

https://github.com/labstack/gommon

VNDR

https://github.com/LK4D4/vndr

Inlets

https://github.com/alexellis/inlets

Gin web framework

https://github.com/gin-gonic/gin

go-isatty

https://github.com/mattn/go-isatty

Dependency for Gin Framework

ginkgo

https://github.com/onsi/ginkgo

Dependency for building Kubernetes complete binaries

mquery

https://github.com/estesp/mquery

A simple utility and serverless-based backend for querying Docker v2 & OCI v1 container images and reporting on "manifest list"/OCI "index" multi-platform image support.

Community

References

Contributors

carlosedp

122 commits

olof-nord

14 commits

meeDamian

3 commits

arshadaleem-lm

3 commits

Similar repos

unikraft/kraftkit

Build and use highly customized and ultra-lightweight unikernel VMs.

446

veggiemonk/awesome-docker

:whale: A curated list of Docker resources and projects

36,877

slpcat/docker-images

docker images

28

linka-cloud/d2vm

Build Virtual Machine Image from Dockerfile or Docker image

374

opencontainers/runc

CLI tool for spawning and running containers according to the OCI specification

13,459

kata-containers/kata-containers

Kata Containers is an open source project and community working to build a standard implementation…

8,772

containerd/containerd

An open and reliable container runtime

21,328

mgoltzsche/podman-static

static podman binaries and container image

464

Languages

Shell

68.2%

Roff

14.5%

C

6.9%

Dockerfile

3.6%

JavaScript

3.3%

Ruby

2.3%

Go

1.1%

carlosedp/riscv-bringup

Risc-V journey thru containers and new projects

Shell

281

151 commits

updated May 17, 2024

See the code

README

RISC-V bring-up tracker

The objective of this repository is to track the progress and pre-requisites to allow containers and Go applications on RISC-V.

The repo also hosts multiple files and images suited for the SiFive Unmatched and QEmu VMs.

There is a companion article available on https://medium.com/@carlosedp/docker-containers-on-RISC-V-architecture-5bc45725624b.

This page is also linked from http://bit.ly/riscvtracker.

If you like this project and others I've been contributing and would like to support me, please check-out my Patreon page!

Contents

RISC-V Unleashed SBC, Virtual Machines and pre-built binaries

To make the development easier, there are Qemu virtual machines based on Debian and Ubuntu with some developer tools already installed.

For the SiFive Unmatched, there is a prebuilt SDcard image at https://github.com/carlosedp/riscv-bringup/releases/download/v1.0/UbuntuHippo-RISC-V.img.gz

For QEmu, there are three distributions of RISC-V pre-packaged VM images:

The user is root and password riscv. For more information, check the readme.

A prebuilt Go 1.16 tarball can be downloaded here.

If required to build the complete boot stack composed of OpenSBI, U-Boot, Linux, checkout the guides for SiFive Unmatched, SiFive Unleashed and Qemu.

To run Go on the VM or board, install with:

# Start the VM
./run_debian.sh

# Download Golang tarball
wget https://github.com/carlosedp/riscv-bringup/releases/download/v1.0/go1.16.5-riscv64.tar.gz

# In the VM, unpack (in root dir for example)
tar vxf go1.16.5-riscv64.tar.gz -C /usr/local/

# Add to your PATH
export PATH="/usr/local/go/bin:$PATH"

# Addto bashrc
echo "export PATH=/usr/local/go/bin:$PATH" >> ~/.bashrc

To run Docker on your RISC-V Debian or Ubuntu environment, download a deb package and install with sudo apt install ./docker-v20.10.2-dev_riscv64.deb.

For other distros get the tarball here and unpack to your / dir. If the docker service doesn't start on install script, re-run systemctl start docker.

Docker-compose install instructions 
# In Debian image
sudo apt-get install python3 python3-dev python3-pip

# For Fedora image
sudo dnf install python3-devel

sudo pip3 install docker-compose

To test it out after install, just run docker run -d -p 8080:8080 carlosedp/echo-riscv and then curl http://localhost:8080.

There are a couple of projects that build on RISC-V in my go-playground repo.

There is also a Podman package here in both .deb and .tar.gz. Check more info on build-podman-env.md to build the package from scratch.

Golang

Core Golang

Golang has been upstreamed as an experimental architecture in Go 1.14. There are no binaries published officially but the releases section has the tarball.

To build Go from source, check build-golang.md.

Go Std Libraries

External deps

Docker and pre-reqs

To build a complete Docker container stack, check the build-docker-env.md document.

Downloads for prebuilt packages are available on https://github.com/carlosedp/riscv-bringup/releases.

Libseccomp

https://github.com/seccomp/libseccomp

Builds fine from master branch. Will be released with riscv64 support on 2.5.

Runc

https://github.com/opencontainers/runc

  • Upstreamed / Works
  • Depends on CGO (to build nsenter)
  • Support buildmode=pie
  • Add riscv64 to libcontainer/system/syscall_linux_64.go
  • After upstreaming, update x/sys and x/net modules
  • libseccomp-dev
  • apparmor - ($ sudo aa-status -> apparmor module is not loaded.)
  • Add to CI

Crun

https://github.com/giuseppe/crun

No changes required, builds fine even without Kernel support for seccomp. Depends on libseccomp.

  • Upstreamed / Works
  • Rebuild with libseccomp
  • Add to CI

Containerd

https://github.com/containerd/containerd/

Docker

Docker cli

https://github.com/docker/cli

  • Upstreamed / Works (must be built from master)
  • Update x/sys and x/net modules in vendor. PR
  • Add riscv64 to manifest annotation. PR#2084
  • Add support for riscv64 on binfmt. PR#21
  • Docker for Mac - Add RISC-V binfmt. PR#4237
  • Add to CI

Docker daemon

https://github.com/moby/moby

Dependency lib PRs:

docker-init

https://github.com/krallin/tini

No changes required. Just build and copy tini-static to /usr/local/bin/docker-init

  • Upstreamed / Works

docker-proxy

No changes required. https://github.com/docker/libnetwork/cmd/proxy

  • Upstreamed / Works

Alternative is run dockerd as: sudo dockerd --userland-proxy=false

Issues

Podman - libpod

https://github.com/containers/libpod

Podman is a library and tool for running OCI-based containers in Pods

CNI Plugins

https://github.com/containernetworking/plugins

  • Builds and runs.

CNI Plugins Issues

Base Container Images

Docker Community "per-architecture" repository

Docker images for projects

OpenFaaS:

Prometheus:

Traefik:

Kubernetes:

Kubernetes images are multi-arch with manifests to arm, arm64, amd64, riscv64 and ppc64le. Some version mismatches due to Kubernetes hard-coded version check for CoreDNS and etcd.

Misc Images:

Kubernetes

https://github.com/kubernetes/kubernetes/

Building and deploying Kubernetes or K3s on RISC-V is detailed on a dedicated readme. There is a build script(build_images.sh) for custom images in kubernetes dir.

To Do:

  • Cross-platform builder image. Update kubernetes/build/build-image/cross adding riscv64 toolchain. Depends on Ubuntu crossbuild-essential-riscv64 be available.
  • Add riscv64 to ./build/pause Makefile. Depends on Go image with RISC-V support and cross-platform image.
Updating dependencies 
# Update dependency
./hack/pin-dependency.sh github.com/mindprince/gonvml
# Update vendor dir
./hack/update-vendor.sh

# Build all main binaries
make KUBE_BUILD_PLATFORMS=linux/riscv64

# Build specifiv binaries
make WHAT=./cmd/${bin} KUBE_BUILD_PLATFORMS=linux/riscv64

# Binaries will be placed on _output/local/go/bin/linux_riscv64/

K3s

https://github.com/rancher/k3s/

K3s build depends on deploying external etcd database since sqlite embedded DB requires CGO. Another requirement is running k3s with Docker and crun as daemon (see Docker install) since runc, the default runtime for K3s requires CGO as well.

For more info, check file [kubernetes/Readme.md#K3s]

Bump:

Future
  • Embed database
  • Embed runtime

Additional projects / libraries

ETCD

https://github.com/etcd-io/etcd

Build with go build ., run with ETCD_UNSUPPORTED_ARCH=riscv64 ./etcd.

OpenFaaS

OpenFaaS is already upstreamed but still does not build images for RISC-V so I've built them and pushed to my DockerHub as links below. Here are the instructions to deploy OpenFaaS on your RISC-V host or VM.

The PRs do not add functionality to cross-build the images for RISC-V yet since the base images still don't support the architecture. Check the build_images.sh script to build the images manually.

Faas-cli

https://github.com/openfaas/faas-cli/

Faas-swarm

https://github.com/openfaas/faas-swarm

Nats-streaming-server

https://github.com/nats-io/nats-streaming-server

No changes required

Bbolt

https://github.com/etcd-io/bbolt

Pty

https://github.com/kr/pty

Prometheus

https://github.com/prometheus/prometheus

Builds successfully with make build.

Promu

https://github.com/prometheus/promu

AlertManager

https://github.com/prometheus/alertmanager

Already builds successfully with make build.

Traefik

https://github.com/containous/traefik

Building and running 
rm -rf static/ autogen/
make generate-webui
go generate
mkdir dist
GOARCH=riscv64 GOOS=linux go build -o dist/traefik ./cmd/traefik
docker build -t carlosedp/traefik:v2.1-riscv64 .

See also a complete Traefik Dockerfile for cross-compiling.

To run an example stack with Docker Compose, create the file below and start it with docker-compose up -d. To test, you can open the address http://[IP]:8080/dashboard or curl http://localhost:8080/api/rawdata. Prometheus metrics are exposed on http://localhost:8080/metrics.

Create a docker-compose.yml

version: '3'

services:
  reverse-proxy:
    # The official v2.0 Traefik docker image
    image: carlosedp/traefik:v2.0-riscv64
    # Enables the web UI and tells Traefik to listen to docker
    command: --api --providers.docker --metrics.prometheus=true
    ports:
      # The HTTP port
      - "80:80"
      # The Web UI (enabled by --api)
      - "8080:8080"
    volumes:
      # So that Traefik can listen to the Docker events
      - /var/run/docker.sock:/var/run/docker.sock
  whoami:
    # A container that exposes an API to show its IP address
    image: carlosedp/whoami:riscv64
    labels:
      - "traefik.http.routers.whoami.rule=Host(`whoami.docker.localhost`)"

Run with:

docker-compose up -d

SQlite

SQLite builds on RISC-V but requires replacing its building files.

Repository mirror: https://github.com/CanonicalLtd/sqlite

  • Upstreamed / Works
  • Update config.guess and config.sub to newer version. Posted to mailing list.

Go-Jsonnet

https://github.com/google/go-jsonnet

Github Hub tool

https://github.com/github/hub

Labstack Echo Framework

https://github.com/labstack/echo

Labstack Gommon

https://github.com/labstack/gommon

VNDR

https://github.com/LK4D4/vndr

Inlets

https://github.com/alexellis/inlets

Gin web framework

https://github.com/gin-gonic/gin

go-isatty

https://github.com/mattn/go-isatty

Dependency for Gin Framework

ginkgo

https://github.com/onsi/ginkgo

Dependency for building Kubernetes complete binaries

mquery

https://github.com/estesp/mquery

A simple utility and serverless-based backend for querying Docker v2 & OCI v1 container images and reporting on "manifest list"/OCI "index" multi-platform image support.

Community

References

Similar repos

unikraft/kraftkit

Build and use highly customized and ultra-lightweight unikernel VMs.

446

veggiemonk/awesome-docker

:whale: A curated list of Docker resources and projects

36,877

slpcat/docker-images

docker images

28

linka-cloud/d2vm

Build Virtual Machine Image from Dockerfile or Docker image

374

opencontainers/runc

CLI tool for spawning and running containers according to the OCI specification

13,459

kata-containers/kata-containers

Kata Containers is an open source project and community working to build a standard implementation…

8,772

containerd/containerd

An open and reliable container runtime

21,328

mgoltzsche/podman-static

static podman binaries and container image

464

Contributors

carlosedp

122 commits

olof-nord

14 commits

meeDamian

3 commits

arshadaleem-lm

3 commits

Languages

Shell

68.2%

Roff

14.5%

C

6.9%

Dockerfile

3.6%

JavaScript

3.3%

Ruby

2.3%

Go

1.1%