This (or frankly any other single-board computer releasing this decade) absolutely isn’t something that’d make sense to use as an inference server that you access from a desktop that already has a GPU and plenty of memory.
It’s not even remotely in the same product category as GB10. More akin to a Jetson NX: a low-power device that can do a little local processing for contexts where it doesn’t have a reliable connection to a more powerful server, though even for that purpose it has lower performance than Orin. Probably would’ve been substantially cheaper than Orin if not for the RAM crisis.
I’d guess most of the actual users of this thing probably will ignore the AI cores entirely and just use it as a RISC-V development board. It’s the first hardware that supports RVA23, which is the set of ISA extensions that most distros will treat as the baseline requirement for RISC-V. Also the fastest RISC-V CPU to date, meaning faster code compile (if you can’t cross-compile) and automated tests than any other RISC-V system.
This (or frankly any other single-board computer releasing this decade) absolutely isn’t something that’d make sense to use as an inference server that you access from a desktop that already has a GPU and plenty of memory.
It’s not even remotely in the same product category as GB10. More akin to a Jetson NX: a low-power device that can do a little local processing for contexts where it doesn’t have a reliable connection to a more powerful server, though even for that purpose it has lower performance than Orin. Probably would’ve been substantially cheaper than Orin if not for the RAM crisis.
I’d guess most of the actual users of this thing probably will ignore the AI cores entirely and just use it as a RISC-V development board. It’s the first hardware that supports RVA23, which is the set of ISA extensions that most distros will treat as the baseline requirement for RISC-V. Also the fastest RISC-V CPU to date, meaning faster code compile (if you can’t cross-compile) and automated tests than any other RISC-V system.