On Friday, Huawei Technologies unveiled a groundbreaking open-source software named Flex:ai. This tool aims to enhance the utilization of artificial intelligence chips, a vital step for Chinese companies. This development comes amid challenges accessing advanced Nvidia processors, which are commonplace in AI training.
Key Features of Flex:ai
Flex:ai serves as an orchestration system that integrates various processing units such as:
- Graphics Processing Units (GPUs)
- Neural Processing Units (NPUs)
- Various accelerators from different chip manufacturers
By pooling these processors together, Flex:ai maximizes performance efficiency by allowing multiple AI workloads to run simultaneously. According to Huawei, the tool can improve processor utilization rates by an impressive average of 30%.
Technical Details and Infrastructure
The software is built on Kubernetes, which has become essential for managing AI infrastructures. Kubernetes is an open-source platform that coordinates fleets of containerized applications, making it easier to deploy and scale AI models across different servers and GPU clusters.
Flex:ai’s functionality includes a smart scheduler named Hi Scheduler. This system manages the allocation of GPUs and NPUs throughout the compute cluster and can pool underused processors from various nodes, redistributing their power to assist AI tasks.
Goals and Aspirations
Huawei’s introduction of Flex:ai reflects a broader ambition to create a self-sufficient compute environment. According to Zhou Yuefeng, the vice-president in charge of Huawei’s data storage product line, the software’s design focuses on better resource utilization.
At the launch event held at Huawei’s Lianqiuhu campus in Shanghai, Zhou emphasized the importance of maximizing AI infrastructure capabilities. He mentioned, “Smaller tasks rarely fully utilize a single card’s capability, while larger tasks often require more than one, complicating compute management.”
Through this initiative, Huawei aims to accelerate the “democratization of AI,” unlocking the full potential of AI infrastructure while openly sourcing its technology.
