-
Getting Started with Wizard-Vicuna-13B
Wizard-Vicuna-13B is an impressive creation based on the Llama 2 platform and developed by MelodysDreamj. This model represents a significant advancement in the field of large language models (LLMs). It effectively combines the principles of WizardLM and VicunaLM, which includes the dataset from WizardLM and the conversation extension from ChatGPT, along with Vicuna's unique tuning method. This innovative combination results in a robust model capable of a wide range of applications.…
-
Getting Started with CausalLM
The CausalLM 14B model is based on the popular llama2 architecture but with Qwen 14B model weights. The Qwen models are developed by Alibaba to be English / Chinese bilingual LLMs. They perform very well in benchmarks compared with other models of similar sizes. The CausalLM model is further SFT fine-tuned on an uncensored dataset with 1.3B tokens. So, it follows conversations and provides a solid basis for further fine-tuning with domain specific knowledge and styles.…
-
Getting Started with OpenChat 3.5
To quick start, you can run Orca or a list of other models with just one single command on your own device. The command tool automatically downloads and installs the WasmEdge runtime, the model files, and the portable Wasm apps for inference. The OpenChat 13B model is fine-tuned on llama2 13B base model for conversation / chat applications. It has a novel fine-tuning method that is more effective than SFT but less expensive than RLFT.…
-
Getting Started with Mistral-7b-Instruct-v0.1
The mistral-7b-instruct-v0.1 model is a 7B instruction-tuned LLM released by Mistral AI. It is a true open source model licensed under Apache 2.0. It has a context length of 8,000 tokens and performs on par with 13B llama2 models. It is great for generating prose, summarizing documents, and writing code. In this article, we will cover How to run mistral-7b-instruct-v0.1 on your own device How to create an OpenAI-compatible API service for mistral-7b-instruct-v0.…
-
Getting Started with Dolphin-2.2-yi-34b
The dolphin-2.2-yi-34b model is based on the 34B LLM, Yi, released by the 01.AI team. Yi is converted to the llama2 format by Charles Goddard and then further fine-tuned by Eric Hartford. In this article, we will cover How to run dolphin-2.2-yi-34b on your own device How to create an OpenAI-compatible API service for dolphin-2.2-yi-34b We will use the Rust + Wasm stack to develop and deploy applications for this model.…
-
Fast and Portable Llama2 Inference on the Heterogeneous Edge
The Rust+Wasm stack provides a strong alternative to Python in AI inference. Compared with Python, Rust+Wasm apps could be 1/100 of the size, 100x the speed, and most importantly securely run everywhere at full hardware acceleration without any change to the binary code. Rust is the language of AGI. We created a very simple Rust program to run inference on llama2 models at native speed. When compiled to Wasm, the binary application (only 2MB) is completely portable across devices with heterogeneous hardware accelerators.…
-
Wasm as the runtime for LLMs and AGI
Large Language Model (LLM) AI is the hottest thing in tech today. With the advancement of open source LLMs, new fine-tuned and domain-specific LLMs are emerging everyday in areas as diverse as coding, education, medical QA, content summarization, writing assistance, and role playing. Don't you want to try and chat with those LLMs on your computers and even IoT devices? But, Python / PyTorch, which are traditionally required to run those models, consists of 3+GB of fragile inter-dependent packages.…
-
Rust + WebAssembly: Building Infrastructure for Large Language Model Ecosystems
This is a talk at the track “The Programming Languages Shaping the Future of Software Development” at QCon 2023 Beijing on Sept 6th, 2023. The session aims to address the challenges faced by the current mainstream Python and Docker approach in building infrastructure for large language model(LLM) applications. It introduced the audience to the advantages of the Rust + WebAssembly approach, emphasizing its potential in addressing the performance, security, and efficiency concerns associated with the traditional approach.…
-
WasmEdge at WasmCon: Dive into Keynotes, Tech Talks, and a Hands-on Workshop
WasmCon, the highly anticipated tech conference dedicated to WebAssembly (Wasm), is just a week away from September 6th to 8th! The WasmEdge community is thrilled to be a part of it! Our maintainers and community contributors have several talks and a hands-on workshop covering high-performance networking, AI Inference, LLM, serverless functions, and microservices. If you are interested in Wasm technology or business use cases, do not miss these talks either in-person next week in Seattle or online later!…
-
🤖The Last Chance to Get in LFX Mentorship in 2023
The LFX Mentorship Term 3 (from September to November) is open to apply! Contributing to WasmEdge through a paid LFX / CNCF internship program is a sure way to future-proof your resume and skill sets! If you’re interested in making contributions to the CNCF-hosted projects, like WasmEdge, grab the last opportunities in 2023. You will gain the following benefits when you contribute to CNCF-hosted projects like WasmEdge through the LFX Mentorship program.…
