I wrote an OpenMIOP driver for Talos Linux and Debian/Ubuntu, enabling Ethernet communication between Mixtile Blade 3 nodes directly over the ClusterBox internal PCIe fabric.
Important: OpenMIOP requires the compatible ClusterBox BMC/OpenWrt firmware v0.1.0-rc.1 to be installed.
I’ve released the first stable version of the complete OpenMIOP stack for Mixtile Blade 3 / Cluster Box!
This release includes updates for all three components:
OpenMIOP PCIe networking driver v0.3.0
Mixtile Talos v0.3.0
ClusterBox BMC OpenWrt firmware v0.1.0
What’s new and fixed
PCIe root complex hang fixed: Introduced a new push-mode architecture. The BMC no longer reads blade memory, preventing PCIe fabric lockups when a blade disconnects or its link drops.
Independent P2P recovery: Blades can now restore PCIe connectivity automatically after a Cluster Box reboot, typically within ~12 seconds, without waiting for the BMC.
NVMe recovery fixed: Cluster Box reboots previously caused NVMe drives to disappear due to shared PCIe reset signals. NVMe devices now recover automatically, preventing storage failures affecting ZFS and DRBD.
New BMC kernel module (omi-rc): Replaces the userspace helper with kernel-level push-mode support. The old helper remains available as a fallback.
Faster initialization: Fixed a timing issue that could delay blade activation by approximately 5 minutes.
Stalled peer recovery: Connections stalled immediately after establishment are automatically retried.
Improved blade management: Updated nodectl to properly release PCIe resources before reset, poweroff, or hard reboot.
Improved performance: Aggregate throughput increased from 22.2 Gbit/s to 25.0 Gbit/s across four blades.
Performance results
Tested with 4 × Mixtile Blade 3 (RK3588), Talos Linux v1.14.2, Kubernetes, Cozystack, and PCIe Gen3 x2:
