-
Getting Started with Dolphin-2.6-Phi-2
To quick start, you can run Dolphin-2.6-Phi-2 with just one single command on your own device. The command tool automatically downloads and installs the WasmEdge runtime, the model files, and the portable Wasm apps for inference. Dolphin 2.6 Phi-2, developed by Eric Hartford and Fernando Fernandes, is an advanced language model based on the Phi-2 architecture. Sponsored by Convai, this model has undergone significant improvements in its latest 2.…
-
Getting Started with Nous-Hermes-2-Mixtral-8x7B SFT
To quick start, you can run Nous-Hermes-2-Mixtral-8x7B-SFT with just one single command on your own device. The command tool automatically downloads and installs the WasmEdge runtime, the model files, and the portable Wasm apps for inference. The Nous Hermes 2 Mixtral 8x7B SFT is a supervised finetune-only version of the Nous Research model trained over the Mixtral 8x7B MoE LLM. It was trained on over 1,000,000 entries of primarily GPT-4 generated data, as well as other high-quality data from open datasets across the AI landscape, achieving state-of-the-art performance on a variety of tasks.…
-
Getting Started with TinyLlama-1.1B-Chat-v1.0
To quick start, you can run TinyLlama-1.1B-Chat-v1.0 with just one single command on your own device. The command tool automatically downloads and installs the WasmEdge runtime, the model files, and the portable Wasm apps for inference. The TinyLlama team recently launched TinyLlama-1.1B-Chat version 1.0. Compared with the previous versions, TinyLlama-1.1B-Chat-v1.0 model pretrains a 1.1B Llama model on 3 trillion tokens. This specific model is based on the Llama 2 architecture and tokenizer, and it has been fine-tuned for text generation tasks, making it suitable for generating conversational responses.…
-
Getting Started with ELYZA-japanese-Llama-2-7b
To quick start, you can run ELYZA-japanese-Llama-2-7b with just one single command on your own device. The command tool automatically downloads and installs the WasmEdge runtime, the model files, and the portable Wasm apps for inference. The ELYZA-japanese-Llama-2-7b model, developed by ELYZA, is a commercially available Japanese language model with 7 billion parameters. It is based on the Llama 2 architecture and is licensed under the LLAMA 2 Community License.…
-
Getting Started with OpenChat-3.5-0106
A new version of OpenChat 3.5 is released today. The OpenChat-3.5-0106 introduces two new modes: coding + generation and mathematical reasoning in this version. The model outperforms ChatGPT (March) and Grok-1 in some benchmarks like GSM8k and HumanEval. In this article, we will cover How to run OpenChat-3.5-0106 on your own device How to create an OpenAI-compatible API service for OpenChat-3.5-0106 We will use LlamaEdge (the Rust + Wasm stack) to develop and deploy applications for this model.…
-
Getting Started with SOLAR-10.7B-Instruct-v1.0
To quick start, you can run SOLAR-10.7B-Instruct-v1.0 with just one single command on your own device. The command tool automatically downloads and installs the WasmEdge runtime, the model files, and the portable Wasm apps for inference. SOLAR-10.7B-Instruct-v1.0 is a cutting-edge language model with 10.7 billion parameters, known for its exceptional performance in natural language processing tasks. This model stands out due to its depth up-scaling methodology, which includes architectural enhancements and additional pre-training.…
-
WasmEdge Provides a Better Way to Run LLMs on the Edge
Published on 2nd Jan 2024. The Rust + Wasm tech stack provides a portable, lightweight, and high-performance alternative to Python for AI/LLM inference workloads. The WasmEdge runtime supports open-source LLMs through its GGML (i.e., llama.cpp) plugin. Rust developers only need to call the WASI-NN API in their applications to perform AI inference. Once compiled to Wasm, the application can run on any CPU, GPU, and OS that supports WasmEdge. Recently, the WasmEdge team has updated its GGML plugin to llama.…
-
Getting Started with Mixtral-8x7B
Published on 1st Jan. To quick start, you can run Mixtral-8x7B with just one single command on your own device. The command tool automatically downloads and installs the WasmEdge runtime, the model files, and the portable Wasm apps for inference. When GPT4 first came out, the community speculated “how many billions of parameters” it had to achieve the amazing performance. But as it turned out, the innovation in GPT4 is not just “more parameters”.…
-
Getting Started with CALM2-7B-Chat
To quick start, you can run CALM2-7B-Chat with just one single command on your own device. The command tool automatically downloads and installs the WasmEdge runtime, the model files, and the portable Wasm apps for inference. CALM2-7B-Chat is an advanced language model fine-tuned for dialogue use cases from CyberAgentLM2, which is a decoder-only language model pre-trained on the 1.3T tokens of publicly available Japanese and English datasets. It is trained…
-
Getting Started with DeepSeek-LLM-7B-Chat
To quick start, you can run DeepSeek-LLM-7B-Chat with just one single command on your own device. The command tool automatically downloads and installs the WasmEdge runtime, the model files, and the portable Wasm apps for inference. DeepSeek-LLM-7B-Chat is an advanced language model trained by DeepSeek, a subsidiary company of High-flyer quant, comprising 7 billion parameters. It is trained on a dataset of 2 trillion tokens in English and Chinese.…
