The following article is from Intelligence Emergence, author Wu Jingjing

36Kr learned that the multimodal large model company "Ruoyu Technology" has completed an angel round of financing of over 50 million yuan. This round of financing was led by Dongfang Jinggong, followed by Kinzhong Capital, and Yuanhe Capital served as the exclusive financing advisor. The company said that the funds will be mainly used for product research and development, business expansion, and team building.

Ruoyu Technology is a team originating from Harbin Institute of Technology, founded in 2023, aiming to build a robot brain through multimodal large model technology.

The company's founder, Nie Liqiang, is currently the executive dean of the School of Computer Science and Technology of Harbin Institute of Technology (Shenzhen). He has been selected as a national talent twice. His scientific research focuses on multimodal content analysis and its implementation in the field of embodied intelligence. He is the first author of the best paper of ACM MM 2022, a top conference of CCF-A in the multimodal field; co-founder Zhang Min serves as the company's chief scientist. He is a specially appointed assistant to the president of Harbin Institute of Technology (Shenzhen) and the first outstanding young scholar in the field of NLP in China. His main research direction is pre-trained language large models and their applications in the field of embodied intelligence; another co-founder, Sun Teng, is a serial entrepreneur and a PhD in artificial intelligence. His research direction is multimedia content analysis and embodied intelligence; at the same time, the company's strategic advisor is Geng Chen, an alumnus of Harbin Institute of Technology, a master of artificial intelligence from the National University of Singapore, deputy director of Huachuang Securities Research Institute, and a well-known analyst.

The opportunity for the company's establishment lies in the new opportunities brought by the technical variable of large models. Ruoyu Technology chose to bet on the robot brain, empowering the robotics industry through its self-developed multimodal large model base combined with massive vertical data. Sun Teng, the company's co-founder, judged: "The robot brain is the pyramid in the core technology stack of the robot, and the robot brain based on multimodal large models is an inevitable trend in the development of the industry ." Ruoyu Technology has taken a fancy to the application opportunities of multimodal large models in the field of robotics.

There are already companies in the international market that are planning to develop this technology. Embodied, founded by Pieter Abbeel, an early founding member of OpenAI and a UC Berkeley expert, and Physical Intelligence, the latest investment of Open AI, are both targeting the direction of robot brains.

Big model technology expands the capabilities of robots. Sun Teng told 36Kr, "Before using AI technology, traditional robots mainly rely on vision and perception technology for body control, such as kinematic and dynamic control. These technologies help robots to plan paths and perform remote control, but in essence, robots still lack thinking and soul, and cannot get rid of the limitation of 'simple mind and strong limbs'."

The introduction of large model technology has changed this situation. "It has endowed robots with intelligence similar to that of human children aged three to six, making a qualitative leap in the ability of robots in reasoning, decision-making, environmental perception and planning," said Sun Teng.

Aiming at equipping robots with multimodal brains, Ruoyu Technology has built its own basic model and used robot data for training.

In terms of the base model, Ruoyu has created a self-developed multimodal model - Ruoyu Jiutian. It is reported that the Jiutian model can support multimodal input and output of text, pictures, audio, and video. At present, the Ruoyu Jiutian multimodal model has been segmented and applied in vertical fields.

The Ruoyu·Jiutian model has also been specially trained for robot application scenarios. "The Ruoyu·Jiutian model includes a planning model, a perception model, and an execution model. It has the ability to understand human language and intentions, as well as multimodal environments. When the robot receives a task instruction, the model can perceive the multimodal environment, decompose the task at multiple granularities, and use the output of the previous step as the input of the next step, forming a cyclic time series planning execution. In this way, even if an unexpected situation occurs, the robot can make corresponding adjustments in the subsequent steps." Sun Teng mentioned.

Sun Teng believes that Ruoyu Technology's core competitiveness lies in the model layer and industrial resource layer: "First of all, our model is completely self-developed, and the team has deep accumulation in the fields of multimodality and natural language processing. It has brought together dozens of national leading talents and young talents. The team has a high talent density and strong professionalism. Secondly, as a school-owned enterprise incubated by the Shenzhen Campus of Harbin Institute of Technology, the school encourages talents to industrialize and provide support to the company in terms of talent introduction and industrial resource introduction. Robotics has always been a dominant discipline of Harbin Institute of Technology, and it has incubated and cultivated a large number of excellent robotics alumni companies. These high-quality industrial resources can support us from product research and development to delivery. "

At the commercial level, Ruoyu Technology currently hopes to adopt a more standardized delivery method. "Our goal is to create a standardized robot brain product, which can be more easily adapted to different robots and application scenarios." Sun Teng said. At present, Ruoyu Technology's "robot brain" is mainly in the form of cloud + terminal, through API + DK (SDK) , and cooperates with robot bodies and scenario companies in the industrial chain. The fields involved include food processing, sorting, assembly, 3C industry and traditional manufacturing, and the company has cooperated with several leading robot body manufacturers and traditional manufacturing companies.

Next, the development of Ruoyu Technology will focus on the core technology research and development and product implementation. Sun Teng, the company's founder, said: "Ruoyu Technology's vision is to give machines the same intelligence as humans. We believe that through continuous technological innovation and product optimization, we can achieve this goal and promote the development of the entire industry." In addition, the company will also cooperate with the government and other companies in the industry to jointly promote the formulation of industry standards for robot brains.

Zhou Wenhui, Vice President of Dongfang Precision Machinery Group, said: "With the support of the profound academic background of Harbin Institute of Technology (Shenzhen), Ruoyu Technology focuses on the industrialization of multimodal large models in the field of robot embodied intelligence, which not only provides robots with a new generation of multimodal large models of intelligent brains, but also gives robots higher-dimensional autonomous reasoning capabilities and a wider range of task generalization capabilities. Faced with a new round of scientific and technological revolution and industrial transformation, Dongfang Precision Machinery Group realizes that it must seize opportunities, increase innovation efforts, cultivate and expand emerging industries, advance the layout of future industries, and accelerate the development of new quality productivity. Through in-depth cooperation with Ruoyu Technology, Dongfang Precision Machinery Group can not only provide more application scenarios for Ruoyu Technology's large models, but also look forward to leveraging Ruoyu Technology's artificial intelligence technology capabilities to help the group better achieve digital and intelligent breakthroughs and achieve high-quality development of the group."

Yao Haibo, founding partner of Kinzon Capital, said: "Kinzon Capital is committed to investing in the field of embodied intelligent robots, and has joined hands with research institutions of universities such as Harbin Institute of Technology, Hong Kong University of Science and Technology, and Southern University of Science and Technology to participate in the start-up of many robotics companies. Ruoyu Technology, as an innovative general robot brain based on multimodal large models, has world-class academic and technical reserves. The founding team led by Chairman Professor Nie Liqiang includes national high-level talents in the fields of multimodal large models, embodied intelligence, and large model quantization and compression. Relying on the joint development and achievement transformation of Harbin Institute of Technology (Shenzhen), combined with the characteristics of Harbin Institute of Technology's advantageous disciplines in robotics, Kinzon Capital is optimistic that Ruoyu Technology will become the leader of China's multimodal large model robot "brain" in the future."