Zhipu AI Launches Open Source GLM-Edge Models
date
Dec 1, 2024
damn
language
en
status
Published
type
News
image
https://www.ai-damn.com/1733020792102-202406051435016830_1.jpg
slug
zhipu-ai-launches-open-source-glm-edge-models-1733021825092
tags
Zhipu AI
GLM-Edge
Edge Computing
Open Source
Artificial Intelligence
summary
Zhipu Technology has announced the open-source release of its GLM-Edge series, a collection of large language and multimodal models designed for edge devices. The series includes four models optimized for mobile and desktop platforms, showcasing high performance and real-time inference capabilities. This move aims to enhance the development of edge AI applications for developers and researchers.
Zhipu AI Launches Open Source GLM-Edge Models
Zhipu Technology has recently unveiled its open-source GLM-Edge series, comprising large language and multimodal models optimized for edge devices. This initiative represents a significant effort by the company to facilitate the implementation of real-world applications on devices with limited computational resources.
Overview of GLM-Edge Series
The GLM-Edge series includes four models of varying sizes: GLM-Edge-1.5B-Chat, GLM-Edge-4B-Chat, GLM-Edge-V-2B, and GLM-Edge-V-5B. These models have been specifically designed for both mobile platforms—such as smartphones and automotive systems—and desktop environments like PCs. This versatility allows developers to integrate advanced AI capabilities across a wide range of devices.
Technological Enhancements
Building upon the technological framework established by the GLM-4 series, Zhipu’s research team has modified model structures and sizes to strike a balance between performance, real-time inference, and deployment ease. Collaborative efforts with industry partners have led to optimizations that significantly improve the operational speeds of the GLM-Edge models on various edge platforms.
Notably, on the Qualcomm Snapdragon 8 Elite platform, which utilizes NPU computing power alongside a mixed quantization scheme, the 1.5B chat model and the 2B multimodal model can achieve impressive decoding speeds exceeding 60 tokens per second. The application of speculative sampling techniques further enhances performance, allowing decoding rates to surpass 100 tokens per second.
Implications for Developers and Researchers
The open-source nature of the GLM-Edge series not only emphasizes Zhipu Technology's leadership in the field of artificial intelligence but also provides developers and researchers with robust tools and resources. This initiative aims to drive innovation in edge AI applications, enabling more efficient and effective use of AI in various sectors.
Access to GLM-Edge Models
Developers interested in exploring the GLM-Edge series can find the models available for download at the following link: GLM-Edge Collection. This accessibility is expected to foster the growth of innovative applications leveraging the capabilities of edge AI.
Conclusion
Zhipu Technology’s launch of the GLM-Edge series marks a pivotal moment in the integration of AI technology into edge computing. By offering these models as open source, the company not only enhances its technological standing but also empowers the developer community to create transformative applications.
Key Points
- Zhipu Technology has released the GLM-Edge series of models for edge devices.
- The series includes models optimized for both mobile and desktop platforms.
- The models demonstrate high decoding speeds, making them suitable for real-time applications.
- The open-source release aims to support innovation in edge AI applications.