Training artificial intelligence models directly on edge devices, known as on-device training, is transforming how AI is deployed in resource-constrained environments. Compared to cloud reliance, this approach offers critical advantages in latency, data privacy, and autonomy, especially in sectors like industrial IoT, mobile robotics, and wearables. However, the traditional bottleneck has been the memory and compute power required to execute backpropagation and weight updates. This is where the combination of the open RISC-V architecture with float16 half-precision, through the Zfh (scalar) and Zvfh (vector) extensions, marks a milestone. This article explores how hardware-software co-design enables up to a 50% memory reduction compared to float32, with minimal model performance degradation, and how companies like Q2BSTUDIO integrate these capabilities into practical solutions.
The RISC-V specification has gained traction due to its modularity and royalty-free nature. The Zfh and Zvfh extensions add native support for 16-bit floating-point numbers, both in scalar and vector units. By using float16, the size of weights and activations is halved, allowing larger models to be trained on devices with tight flash or SRAM budgets. Recent work, such as the AIfES framework, demonstrates that a complete training pipeline can be implemented on a single low-power RISC-V core, including transfer learning and fine-tuning with layer freezing. This is possible because precision reduction is compensated with scaling and normalization techniques that maintain convergence.
From a co-design perspective, implementing Zfh on a superscalar out-of-order RV64GC FPGA softcore incurs minimal area overhead: approximately 1.15% more LUT6 and 0.05% more flip-flops, running at 175 MHz. This low area cost allows manufacturers to add float16 support without penalizing the silicon budget. Additionally, the vector extensions Zvfh enable SIMD operations on 16-bit data, accelerating convolutions and dense layers without the need for external accelerators. The result is an efficient platform for on-site training, ideal for applications where cloud bandwidth is limited or latency is critical.
In the business realm, the combination of float16 on RISC-V opens opportunities to develop custom software that requires continuous learning. For example, an industrial sensor can adjust its anomaly detection model based on new data without sending sensitive information to a central server. Q2BSTUDIO, as a software development and technology company, offers custom software services to integrate these capabilities into embedded systems, leveraging RISC-V scalability and float16 efficiency. Moreover, security of these devices is paramount: keeping data local reduces attack vectors, but firmware and communications still need protection. Therefore, Q2BSTUDIO incorporates cybersecurity practices in its developments, including encryption and edge authentication.
Cloud convergence is also relevant. On-device training can be combined with cloud services like AWS or Azure for model orchestration, remote weight updates, or federated aggregation. Q2BSTUDIO deploys cloud AWS/Azure solutions that facilitate fleet management of RISC-V devices, allowing local training to sync with BI / Power BI dashboards for real-time analysis of model performance and evolution. This integration turns devices into intelligent nodes of a corporate data ecosystem.
Beyond training, the ability to run inference and learning on the same hardware enables so-called AI agents. These autonomous agents can adapt their behavior based on local context, such as a drone learning to optimize its flight path or a voice assistant personalizing responses without cloud dependency. Q2BSTUDIO develops AI agents that use float16 on RISC-V, combining computational efficiency with adaptive intelligence, all within a framework of confidentiality and low energy consumption.
In conclusion, the hardware-software co-design enabled by float16 on RISC-V represents a step forward toward democratizing AI at the edge. The memory reduction, low area overhead, and flexibility of the open architecture make this combination a strategic choice for companies seeking to innovate with local AI. Q2BSTUDIO, with its expertise in software development, cloud, cybersecurity, and business intelligence, is ready to accompany organizations in this transition, offering both the technical tools and the consulting needed to maximize the value of on-device training with float16 on RISC-V.
