Local LLM simplified

You didn’t come this far to stop

Easy mode LLM

A friend asked me about why one would need a local LLM and if they were useful and could I help simplify the setup so a non-technical person could use it?

I said "Sure!" ... And here we are.

We will call this Phase 1. First let's get the requirements out of the way.

• We need a decent computer. don't use grandma's hand-me-down it will be to slow and you will get annoyed.

• a secondary hard drive is preferred. once you start playing with llms, you'll want to try more advanced ones. First you'll download the 2gb mode, then by the end of the week you're trying a 50gb model to see what it can do.

• You need Windows if you want to use my scripts. I based my Scripts on PowerShell, because I didn't want people to need to install and figure out how to configure python, or next.js, or vscode. PowerShell is built-in already.

• For the more advance stuff you will need MS Office. My later scripts will be parsing word docs, pdf's, and excel sheets. Office helps with this.

What is in the Setup Script?

1. Run the script. - Double-click

2. select the number that correlates to a drive that will have enough space to store the llm's

3. wait for the rest to get installed and configured

What this actually does is:

• downloads and installs Ollama. a program for running local llms and creates a webservice to query the gguf model files.

• Creates a folder structure that will be used in later scripts

• creates a configuration file to be used in later scripts

• downloads 4 specific models

  • deepseekR1 - reasoning model

  • llama 3.2 - used for converting transcripts to Q/A docs

  • llama2 - used for querying local embedded docs

  • nomic-embed-text - used to embed local data into a format for querying

• validating office is installed - for later embedding files

Everything is installed. now what?

Let's get the bare minimum out of the way.

How do I ask this local LLM Questions?

open a command prompt (Start - type cmd)

type "Ollama run deepseek-r1:8b"

Ask a question or two or ten.

you can close the windows when you're done or type /bye

Let's get some data to play with

My friend is a teacher, tutor, engineer, Nasa scientist guy. He has 3700 videos uploaded to his channel give him some love. https://www.youtube.com/@MathAndScience

I was trying to find a way to help him understand LLM's and also benefit from all his channel data.

This one is to get-transcripts. but not only does it get transcripts, but it creates Questions and answers based on the data in the transcripts. Not only does it get transcripts from single Youtube videos, but we can plugin playlists and get all the videos from a playlist and Q/A from these.

I see you're not impressed. Not let's see if we can make it more exciting.

Or are you a business guy, do you want to be one? why not get study material from Alex Hormozi's podcast "The Game"?

https://www.youtube.com/playlist?list=PLjkaUn6QNTZTd2rKxXdy-_I0JDHlwdy6j

143 episodes of money making goodness.

Internet Learning made easy

https://www.youtube.com/playlist?list=PLhQjrBD2T381PopUTYtMSstgk-hsTGkVm

Here is the playlist for Harvards' CS50 intro to AI. 8 videos over 1hour long. Make yourself a study sheet based on all the lectures with the click of a button. Learning on easy mode. and since we're using AI to create the questions based on the material we can run it a few times to generate different varying questions based on the same material to further enhance your understanding. And not to spoil the punchline, but it's FREE. you don't have to buy tokens from ChatGPT, Claude, Perplexity, Deepseek, etc. because it is running the AI from your own computer to generate these questions.

https://www.youtube.com/playlist?list=PLnVYEpTNGNtVryZHBqJUqXprrQcx_BdhD

Do you have kids? are they taking a Math or Science class? Maybe a Chemistry class, but you don't remember chemistry. Why not have a refresher with some study questions to help your kid out? 12 videos right here. You get the transcripts and Questions and answer sheets.

Show me the Script already.

I know. So far this has been super complicated.

• Double click on a script and select a drive letter to setup the 'environment'

• open a command prompt and type "Ollama run <model>" to ask a question.

• Now this even more complicated script. Double click - paste in a youtube url to get some output.

I didn't do everything for you.. yet

As you can see we have the saved transcript and we have the Questions and answers based on the material. I didn't make it all pretty and formatted, but it's easy enough to use.

Some of you are going to ask about the image at the top of blog article. I see a little chatbox with some color and tabs and buttons and things. That's phase 2. Using transcripts or personal data to use with your local model. It's not quite perfected yet, but I'll have something to show in the next few days.

Until then.

How do i get these scripts?

Simple send me an email and I'll send you a link to the scripts.
feel free to reach out the same way if you have an issue with the scripts. maybe i can make some useful updates?