Experiment with an LLM: Unleash the Power of Large Language Models

This guide will walk you through setting up your own server and using Ollama and Open WebUI, interact with powerful Large Language Models (LLMs).

Once you have completed the steps below, you will be using a very powerful NVIDIA based server. This type of servers is expensive. My Dev Server makes it easy to manage your costs. Run the server when you are working. Stop the server when you are not.

YOU ONLY PAY FOR WHAT YOU USE.

Note: There is a very small storage cost when a server is stopped.

Get ready to explore the incredible capabilities of AI language processing!

1. Create Your Server

The foundation for your LLM playground! We recommend a server with sufficient GPU, memory, and storage to handle the demands of LLMs.

Click the link below to create your server on our platform:

Create Server

Note: This button will redirect you to the home page to create your server. After creating your server, return to this guide and continue with the next step.

2. Connect to Your Server

Now that your server is up and running, let's connect to it using SSH. You created an SSH key during server creation and saved it. You will use this key to connect to your server.

We have provided an example SSH command that you can use. IUt is under "Access" on the home page for your server.

Click on the command (it is automatically copied).

Open a Powershell window and paste in the command. If you stored the SSH key file in a folder, you will have to add that to the command.

You can also just type in the command as follows:

ssh -i [path_to_ssh_file] root@[ip_address]

Replace the following:

[path_to_ssh_file]: The complete path to your SSH key file.
[ip_address]: The IP address of your newly created server.

3. Install and Run Open Web UI and Ollama

Ollama allows you to easily run and experiment with different LLMs. Open Web UI provides a nice interface which allows you to do this.

Run the following command to install and start Open Web UI and Ollama in a Docker container:

docker run -d -p 8080:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama

4. Load LLM Models

Ready to unleash the power of LLMs? Head over to the Ollama library (ollama.com/library) to explore available models. Identify models that you are interested in. These will need to be loaded into Ollama.

Note: Some of these models are very large. Make sure that your server has sufficient space to load all the models.

Create a shell inside of Docker and load the LLM models you have identified:

docker exec -it open-webui bash
ollama pull [model_name]

When you are done, exit from the shell:

exit

5. Interact with Your LLM

You can interact with your LLM models remotely from a browser.

Start up Open Web UI

Enter:

http://[ip_address]:8080/

Replace the following:

[ip_address]: The IP address of your newly created server.

Creat an account and login.

The information that you enter remains on your server. It is not shared with anyone or stored anywhere else.

CONGRATULATIONS - ARE UP AND RUNNING!!

6. Time to Experiment!

There are lots of options for the kinds of experiments you might try. Here are two to get you started.

Compare the results of the same prompt on different LLM models
Compare the performance across different LLM models

These can be done with the following steps.

Create a prompt.
Select a LLM model from the upper left "Select a model".
Click the "+" from the upper left and select another LLM model.
Enter the prompt. It will be passed to both LLM models.
Compare the performance and result from the two LLM models.

Happy Experimenting!