Run Your Own Local, Private, ChatGPT-like AI Experience with Ollama and OpenWebUI (DeepSeek, Llama3, Phi3, Gemma, Mistral, and more LLMs!)

Over the last couple years the emergence of Large Language Models (LLMs) has revolutionized the way we interact with Artificial Intelligence (AI) systems, enabling them to generate human-like text responses with remarkable accuracy. From powering chatbots and virtual assistants to aiding in natural language processing tasks, LLMs have found widespread applications across every industry. However, as their capabilities continue to grow, so do concerns surrounding data privacy and productivity.

Now with two innovative open source tools, Ollama and OpenWebUI, users can harness the power of LLMs directly on their local machines. Ollama, short for Offline Language Model Adapter, serves as the bridge between LLMs and local environments, facilitating seamless deployment and interaction without reliance on external servers or cloud services. Complementing Ollama is OpenWebUI, a user-friendly interface that simplifies communication with local LLMs, enabling users to engage in natural language conversations effortlessly.

In this article, we’ll look at running your own local ChatGPT-like app using both Ollama and OpenWebUI, enabling the use of multiple LLMs locally; including Microsoft’s Phi-3 SLM (small language model). We’ll also explore the benefits of bringing powerful AI capabilities to your local machine, including enhanced privacy and productivity. Additionally, we’ll look at the introductory steps to get started with setting up and using Ollama and OpenWebUI on your own local machines, paving the way for a seamless and personalized AI experience tailored to your needs and preferences.

Understanding Ollama and OpenWebUI

Before we get into running Ollama and OpenWebUI to run a local LLM AI experience, let’s look at exactly what these projects are.

What is Ollama?

Ollama is a very convenient, local AI deployment tool, functioning as an Offline Language Model Adapter. At its core, Ollama serves as a link between your local environment and large language models (LLMs), facilitating the local deployment of LLMs and local interactions with them. All without requiring any need for external servers or cloud-based infrastructure.

Ollama bridges the gap between the robust capabilities of LLMs and the desire for increased privacy and productivity that comes with running AI models locally. By eliminating the reliance on external servers, Ollama empowers users to leverage the full potential of LLMs while maintaining privacy, ownership, and control over their data and computations.

At it’s core, Ollama provides a terminal experience for interacting with the various supported LLMs. It also supports macOS, Linux and Windows, so regardless of your operating system, you’ll be able to easily get up and running with your own locally hosted LLMs!

What LLMs are Supported by Ollama?

One of Ollama’s most significant advantages lies in its compatibility with a wide array of LLMs, including but not limited to Llama3, Phi-3, Gemma, Mistral, and more. This versatility ensures that users have the flexibility to choose the LLM that best suits their needs and preferences, whether it’s for natural language understanding, text generation, or any other AI-related task. Whether you’re exploring the depths of conversational AI or delving into the intricacies of language processing, Ollama provides the framework to bring these powerful capabilities directly to your fingertips, right on your local machine.

The Ollama library contains a full list of openly available LLMs that are supported. This includes the following AI language models:

• deepseek-r1 – DeepSeek’s first-generation of reasoning models.
• llama3 – Meta Llama 3
• phi4 – Phi-4 14B parameter, state-of-the-art model from Microsoft.
• phi3 – Phi-3 Mini SLM is a 3.8B parameters, lightweight, state-of-the-art open model by Microsoft.
• wizardlm2 – LLM from Microsoft AI with improved performance and complex chat, multilingual, reasoning an dagent use cases
• mistral – The 7B model released by Mistral AI
• gemma – Gemma is a family of lightweight, state-of-the-art open models built by Google DeepMind
• and more!
• moondream – A small vision language model designed to run efficiently on edge devices

What is OpenWebUI?

OpenWebUI serves as the web gateway to effortless interaction with local LLMs, providing users with a user-friendly interface that streamlines the process of deploying and communicating with these powerful language models. OpenWebUI does this by providing a web interface for Ollama that is hosted on your machine using a Docker container.

At its core, OpenWebUI is designed to simplify the complexities associated with running LLMs locally, offering an intuitive platform for users to engage in natural language conversations and leverage the capabilities of AI without the need for specialized technical expertise.

One of the key features of OpenWebUI is its seamless communication with local LLMs. Through its intuitive interface, that bears similarities to OpenAI’s ChatGPT interface, you can effortlessly input queries, prompts, or commands and receive responses in real-time, enabling dynamic and interactive interactions with AI models running on your own machine. This direct communication pathway ensures low latency and high responsiveness, enhancing the user experience and enabling a fluid dialogue with the AI.

Moreover, OpenWebUI plays a crucial role in democratizing access to LLM capabilities by abstracting away the complexities of their setup and configuration. By providing a straightforward interface for deploying and interacting with LLMs, the combination of OpenWebUI with Ollama lowers the barrier to entry for individuals looking to harness the power of AI for various applications. Whether you’re a developer prototyping a new conversational agent, a researcher exploring natural language processing techniques, or any other type of user, OpenWebUI empowers you to focus on your goals without being bogged down by the technical details.

Get Started with Ollama and OpenWebUI

Let’s take a look at getting started running your own locally hosted LLMs using Ollama and OpenWebUI!

Get Started with Ollama

Step 1: Download and Install Ollama

Open a web browser and navigate over to https://ollama.com, then click the Download button and go through downloading and installing Ollama on your local machine.

Step 2: Run Ollama in the Terminal

Once you have Ollama installed, you can run Ollama using the ollama run command along with the name of the model that you want to run. Ollama will automatically download the specified model the first time you run this command.

# Run DeepSeek-R1 model locally
ollama run deepseek-r1:7b   # 7B parameter
ollama run deepseek-r1:14b  # 14B parameter

# Run llama3 LLM locally
ollama run llama3

# Run Microsoft's Phi-4 SLM locally
ollama run phi4

# Run Microsoft's Phi-3 Mini small language model locally
ollama run phi3:mini

# Run Microsoft's Phi-3 Medium small language model locally
ollama run phi3:medium

# Run Mistral LLM locally
ollama run mistral

# Run Google's Gemma LLM locally
ollama run gemma:2b   # 2B parameter model
ollama run gemma:7b   # 7B parameter model

Step 3: Enjoy asking the LLM questions

Once you have Ollama installed, have downloaded one or more LLMs with it, you can enjoy using your own locally hosted LLMs from the terminal / command-line of your local machine.

If you want a nicer web UI experience, that’s where the next steps come in to get setup with OpenWebUI.

Get Started with OpenWebUI

Step 1: Install Docker

OpenWebUI is hosted using a Docker container. Before you can download and run the OpenWebUI container image, you will need to first have Docker installed on your machine.

You can find help on installing Docker Desktop on your local machine here: https://docs.docker.com/desktop/

Step 2: Run OpenWebUI Docker Container

With OpenWebUI being hosted in a Docker container, it’s fairly easy to get it up and running. Once you have Docker installed, you can use the following command to run a local instance of the open-webui to host your own instance of OpenWebUI:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Step 3: Access OpenWebUI in the Web Browser

Looking at the Docker command to run the open-webui container, you can see that the app will be hosted on localhost port 3000. Open a web browser on your local machine, and navigate to http://localhost:3000 to access the local instance of OpenWebUI.

P.S. Keep in mind this is only a locally available instance of OpenWebUI. However, since it’s hosted in a Docker container, it’s possible to host this on a server in the cloud too, if you want to make it available from other machines too.

If you need additional help hosting or configuring the container, the OpenWebUI documentation includes more options.

Step 4: Login to OpenWebUI

The way that OpenWebUI works, is that it does have authentication using it’s own username/password store. The first time you access your local instance of OpenWebUI, you’ll need to click on signup and create an account. Then use that account to login to OpenWebUI after that.

Step 5: Enjoy Your Own Local ChatGPT-like Experience!

Now that you’re logged into your own locally hosted OpenWebUI, you can enjoy your own, personal, private ChatGPT-like experience!

Next Steps

There is so much that an be done with LLMs, and the combination of Ollama and OpenWebUI really empower you with a lot of local, offline, and disconnected AI power!

Once you have Ollama and OpenWebUI running locally, you can begin to play around with all the different supported LLMs. There are so many! You’ll be able to find your own combination of LLMs that work best for you, including image generation LLMs. Also, OpenWebUI has additional features, like the “Documents” option of the left of the UI that enables you to add your own documents to the AI for enabling the LLMs to answer questions about your won files.

Personally, I am intrigued by the ability to host LLMs locally and all the possible use cases this might bring to personal productivity. If you don’t mind sharing, please post in the comments, what about running your own local, private LLMs is the most interesting to you?

Original Article Source: Run Your Own Local, Private, ChatGPT-like AI Experience with Ollama and OpenWebUI (DeepSeek, Llama3, Phi3, Gemma, Mistral, and more LLMs!) written by Chris Pietschmann (If you're reading this somewhere other than Build5Nines.com, it was republished without permission.)