Discoblocks

The end-2-end build includes:

• gosec scanning
• golang-ci linting
• Docker image build
• Trivy vulnerability scanning

Please note: We take security and users' trust seriously. If you believe you have found a security issue in Discoblocks, please responsibly disclose by following the security policy.

This is the home of Discoblocks, an open-source declarative disk configuration system for Kubernetes helping to automate CRUD (Create, Read, Update, Delete) operations for cloud disk device resources attached to Kubernetes cluster nodes.

• Website: https://discoblocks.io
• Announcement & Forum: GitHub Discussions
• Documentation: GitHub Wiki
• Recording of a demo: Demo

About the name

Some call storage snorage because they believe it is boring... but we could have fun and dance with the block devices!

Why discoblocks

Discoblocks can be leveraged by cloud native data management platform (like Ondat.io) to management the backend disks in the cloud.

When using such data management platform to overcome the block disk device limitation from hyperscalers, a new set of manual operational tasks needs to be considered like:

• provisioning block devices on the Kubernetes worker nodes
• partioning, formating, mounting the block devices within specific path (like /var/lib/vendor)
• capacity management and monitoring
• resizing and optimizing layouts related to capacity management
• decommissioning the devices in secure way
  • by default every resource created by DiscoBlocks has a finalizer, so deletion will be blocked until the corresponding DiskConfig has been deleted
  • by default every additional disk has owner reference to the first disk ever created for pod, Deletion of first PVC terminates all other

At the current stage, Discoblocks is leveraging the available hyperscaler CSI (Container Storage Interface) within the Kubernetes cluster to:

• introduce a CRD (Custom Resource Definition) per workload with
  • StorageClass name
  • capacity
  • mount path within the Pod
  • nodeSelector
  • podSelector
  • access modes: Access mode of PersistentVolume
  • availability mode:
    • ReadWriteOnce: New disk for each pod, including pod restart
    • ReadWriteSame: All pod gets the same volume on the same node
    • ReadWriteDaemon: DaemonSet pods always re-use existing volume on the same node
  • upscale policy
    • upscale trigger percentage
    • maximum capacity of disk
    • maximum number of disks per pod
    • extend capacity
    • cool down period after upscale
    • pause autoscaling
• provision the relevant disk device using the CSI (like EBS on AWS) when the workload deployment will happen
• monitor the volume(s)
• resize automatically the volume based on the upscale policy
• create and mount new volumes in running pod based on maximum number of disks

Note: that an application could be using Discoblocks to get persistent storage but this option would not be safe for production as there will not be any data platform management to address high availability, replication, fencing, encryption, ...

Demo from our core Developer

https://user-images.githubusercontent.com/55788733/168624989-c9b1d469-d3e5-40e7-8858-c9ff4a5b5428.mp4

How to install

Prerequisites

• Kubernetes cluster
• Kubernetes CLI
• Cert Manager
  • kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.8.0/cert-manager.yaml

Install on AWS with EBS backend

cat <