Competition Cloud Infrastructure

Casper O. da Costa-Luis
github/casperdcl

MIC WS-05-02 (2024-11-02) video | html | pdf

GitHub

leaderboard tensorboard

website wiki discord

Problem

CERN employs ten times more engineers and technicians than research physicists.

Do you know why?

https://home.cern/science/engineering

Participating

  1. register
  2. template code & training data
  3. submitting code
  4. standardised & secure GPU-enabled continuous integration (CI)
  5. live leaderboards
  6. test data

Participating

  1. register
  2. template code & training data
  3. submitting code
  4. standardised & secure GPU-enabled continuous integration (CI)
  5. live leaderboards
  6. test data

Registration

register

register

Template Code

Repository

GitHub

Requires SIRF, CIL, CUDA
e.g. via Docker synerbi/sirf:latest-gpu

Template Code

Layout

  • (required) main.py

    from cil.optimisation.algorithms import Algorithm
    class Submission(Algorithm):
        ... # e.g. BSREM, ISTA, OSEM
    submission_callbacks = [] # per-iteration
  • apt.txt/environment.yml/requirements.txt: passed to apt/conda/pip install

Submitting Code

Private mirror

  • https://github.com/SyneRBI/PETRIC-{TEAM_NAME}

Continuous Integration

GitHub (native) runners

  • can run basic (CPU-only) checks

Continuous Integration

GitHub self-hosted runners

Acquire or rent a machine (AWS, GCP, Azure, etc…)

  • GPU (NVIDIA A100 PCIe 40GB GPU)
  • CPU 32-Core
    • 100GB RAM
  • sequential queue

Continuous Integration

Security

GitHub docs1,2 TL;DR:

don’t do this because we can’t be bothered to document how to make it secure

Cloud Services

Public domain

  • public IP (130.246.81.43)
  • (sub)domain name (*.tomography.stfc.ac.uk)
  • docker services & reverse proxy with TLS (https://)

Data

https://petric.tomography.stfc.ac.uk/data

Cloud Services

Leaderboards

During competition

tensorboard

Results

leaderboard

Fin

Want help running your own competition?

Want to compete next year?

GitHub

github/casperdcl


  1. https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners#self-hosted-runner-security↩︎

  2. https://docs.github.com/en/actions/security-for-github-actions/security-guides/security-hardening-for-github-actions#hardening-for-self-hosted-runners↩︎