I am building my personal private cloud. I am considering using second hand dell optiplexes as worker nodes, but they only have 1 NIC and I'd need a contraption like this for my redundant network.
Then this wish came to my mind. Theoretically, such a one box solution could be faster than gigabit too.
I ain't got that budget man.
Then skip redundant networking and use that cash on the router hardware such that you get enough ports that you won't need a switch
It is for a challenge, the goal is to build a cloud with workload decoupled from servers decoupled from users who'd deploy the workload, with redundant network and storage, no single choke point for network traffic, and I am trying to achieve this with a small budget.
Ah okay. If you don't have any PCI slots available then USB is going to be the only option then. Though if you have WiFi you could use that as your fail over solution.
Basically, from a hardware standpoint, I'd say a minimum viable solution would look like (no wifi):
2 cheap 4-6 port Switches. Preferably with an uplink port and then you can mock redundant ISP connection by putting a splitter on the WAN connection and putting it into each of the two switches.
3-4 worker nodes each connected to both switches. Preferably a PCI nic but if that is a non-starter then USB. Though here is where pretty much every cent should be spent. Important that each has a minimum of 200 GB storage because that enables you to abstract storage into a CEPH pool and still have enough space to run some decent example workloads.
All other redundancy can and should be handled in software. OpenStack is one approach, Proxmox can probably also work.