-
Getting Started with TinyLlama-1.1B-Chat-v1.0
To quick start, you can run TinyLlama-1.1B-Chat-v1.0 with just one single command on your own device. The command tool automatically downloads and installs the WasmEdge runtime, the model files, and the portable Wasm apps for inference. The TinyLlama team recently launched TinyLlama-1.1B-Chat version 1.0. Compared with the previous versions, TinyLlama-1.1B-Chat-v1.0 model pretrains a 1.1B Llama model on 3 trillion tokens. This specific model is based on the Llama 2 architecture and tokenizer, and it has been fine-tuned for text generation tasks, making it suitable for generating conversational responses.…
-
Getting Started with ELYZA-japanese-Llama-2-7b
To quick start, you can run ELYZA-japanese-Llama-2-7b with just one single command on your own device. The command tool automatically downloads and installs the WasmEdge runtime, the model files, and the portable Wasm apps for inference. The ELYZA-japanese-Llama-2-7b model, developed by ELYZA, is a commercially available Japanese language model with 7 billion parameters. It is based on the Llama 2 architecture and is licensed under the LLAMA 2 Community License.…
-
Getting Started with OpenChat-3.5-0106
A new version of OpenChat 3.5 is released today. The OpenChat-3.5-0106 introduces two new modes: coding + generation and mathematical reasoning in this version. The model outperforms ChatGPT (March) and Grok-1 in some benchmarks like GSM8k and HumanEval. In this article, we will cover How to run OpenChat-3.5-0106 on your own device How to create an OpenAI-compatible API service for OpenChat-3.5-0106 We will use LlamaEdge (the Rust + Wasm stack) to develop and deploy applications for this model.…
-
Getting Started with SOLAR-10.7B-Instruct-v1.0
To quick start, you can run SOLAR-10.7B-Instruct-v1.0 with just one single command on your own device. The command tool automatically downloads and installs the WasmEdge runtime, the model files, and the portable Wasm apps for inference. SOLAR-10.7B-Instruct-v1.0 is a cutting-edge language model with 10.7 billion parameters, known for its exceptional performance in natural language processing tasks. This model stands out due to its depth up-scaling methodology, which includes architectural enhancements and additional pre-training.…
-
WasmEdge Provides a Better Way to Run LLMs on the Edge
Published on 2nd Jan 2024. The Rust + Wasm tech stack provides a portable, lightweight, and high-performance alternative to Python for AI/LLM inference workloads. The WasmEdge runtime supports open-source LLMs through its GGML (i.e., llama.cpp) plugin. Rust developers only need to call the WASI-NN API in their applications to perform AI inference. Once compiled to Wasm, the application can run on any CPU, GPU, and OS that supports WasmEdge. Recently, the WasmEdge team has updated its GGML plugin to llama.…
-
Getting Started with Mixtral-8x7B
Published on 1st Jan. To quick start, you can run Mixtral-8x7B with just one single command on your own device. The command tool automatically downloads and installs the WasmEdge runtime, the model files, and the portable Wasm apps for inference. When GPT4 first came out, the community speculated “how many billions of parameters” it had to achieve the amazing performance. But as it turned out, the innovation in GPT4 is not just “more parameters”.…
-
Getting Started with CALM2-7B-Chat
To quick start, you can run CALM2-7B-Chat with just one single command on your own device. The command tool automatically downloads and installs the WasmEdge runtime, the model files, and the portable Wasm apps for inference. CALM2-7B-Chat is an advanced language model fine-tuned for dialogue use cases from CyberAgentLM2, which is a decoder-only language model pre-trained on the 1.3T tokens of publicly available Japanese and English datasets. It is trained…
-
Getting Started with DeepSeek-LLM-7B-Chat
To quick start, you can run DeepSeek-LLM-7B-Chat with just one single command on your own device. The command tool automatically downloads and installs the WasmEdge runtime, the model files, and the portable Wasm apps for inference. DeepSeek-LLM-7B-Chat is an advanced language model trained by DeepSeek, a subsidiary company of High-flyer quant, comprising 7 billion parameters. It is trained on a dataset of 2 trillion tokens in English and Chinese.…
-
Getting Started with Mistral-7B-Instruct-v0.2
Or you can run this newest model with just one single command on your mac/ across devices. The Mistral-7B-Instruct-v0.2 model is a new model released by the Mistral AI team. It’s built upon the successful foundation of its predecessor, the Mistral-7B-v0.1. This model stands out for its improved abilities in understanding and following complex instructions, making it an even more powerful tool for a wide range of applications.This combination of advanced technology and user-friendly design makes Mistral-7B-Instruct-v0.…
-
Getting Started with Neural-Chat-7B-v3-1
Neural-Chat-7B-v3-1 is a fine-tuned model based on Mistral-7B-v0.1 and trained on the Open-Orca/SlimOrca open-source dataset. The model underwent training between September and October 2023. It incorporates a Direct Preference Optimization (DPO) algorithm, highlighting its advanced fine-tuning and optimization capabilities. In this article, we will cover How to run Neural-Chat-7B-v3-1 on your own device How to create an OpenAI-compatible API service for Neural-Chat-7B-v3-1 We will use the Rust + Wasm stack to develop and deploy applications for this model.…
