I’ve received my ClusterBox and installed four Mixtile Blade 3s into it. After powering it up and connecting an ethernet cable to the back of it, I was able to log into the OpenWRT management console. I was also able to ssh in, where I executed nodectl list and the response said to try nodectl rescan if there were no nodes visible. There were no nodes visible so I tried rescan and list again and still no nodes.
The ETH_PCI device shows in the OpenWRT list of network interfaces but it shows an error: “Network device is not present.”
I have verified that at least Node 1 has successfully booted to the Debian XFCE desktop and is responsive. It does not, however, have any functioning network interfaces. It does have the factory OS image on it still.
Hello JGiszczak, sorry for the late reply. Your Blade 3s are missing PCIe driver so there are no nodes. We plan to upload the related driver file and prepare the instruction of setting by tomorrow. We will inform you as soon as we finish that.
I had a feeling that was the problem. I’m pleased to hear I can expect a driver update this week. Don’t rush the testing on my account though. The potential for damage is always present when developing hardware drivers. I’d rather they be correct and late rather than early and frying my boards.
Hi, JGiszczak. This instruction is for Blade 3 boards running Debian. I guess the description “Install Android Debug Bridge (ADB) by referring to Install ADB” misled you to think this guide is for Android system, which is not. Although ADB is a tool mainly for Android devices, it also works on Windows, macOS, Linux, ChromeOS, and even other systems.
The link Install ADB introduces how to install this tool on Windows, Ubuntu, and MacOS.
I will modify the doc to remove such misunderstandings. Thanks for reminding.
Since my blades are just running Debian, I used scp instead of ADB to copy the files onto each of them. After installing the driver package and writing the new boot.img, I restarted the cluster box and all four blades are visible and were assigned IP addresses. nodectl list now shows all four devices and OpenWRT’s interface shows the DHCP leases for the internal network.
Unfortunately communication does not work between blade 1 and blade 4. sudo ping -I pci0 10.20.0.x.x works from node 2 to all nodes, from node 3 to all nodes, and from node 1 to nodes 2 and 3, and from node 4 to nodes 2 and 3, but not between 1 and 4, from either end.
I’m not absolutely certain about my node numbering but I’m fairly sure. If necessary, I could verify with certainty.
Sorry that you encountered a new problem. To address the problem of node 1 failing to communicate with node 4, you can edit the /etc/miop_nodes.txt file under Cluster Box as follows:
6=0
5=1
3=2
4=3
To check the mapping between node numbering and IP address, you can navigate to the console of a Blade 3 node by running the command nodectl console -n N (N can be 1, 2, 3, or 4), and then run ifconfig. You will see the IP address of this Blade 3 node.
To exit from the Blade 3 node, press Ctrl+A and then Ctrl+X.
Were the requisite changes merged to the mixtile/linux Github repository? I’ve compiled a custom kernel from the blade3/debian11/kernel5.10 branch and installed it. On reboot, I got all my changes but lost the pci0 device. I can still sudo insmod /lib/modules/5.10.66/miop-control.ko and the kernel loads it but it does nothing.
Hi JGiszczak, the necessary changes haven’t been merged into the mixtile Github repository yet. We’ll commit them by the end of tomorrow, so please bear with us a while longer.
Hi JGiszczak, we just pushed the necessary changes to the mixtile Github repo. However, we’re concerned that the PCIe settings for the Cluster Box might impact the performance of individual Blade 3 units. Therefore, we’ve held off on merging the PCIe changes for now.
You can manually patch the kernel using this file Add-support-for-MIOP-drivers and then compile it. We’ll be conducting thorough testing over the next few days. If everything checks out, we’ll merge the updated patch code to the mixtile Github repo.