run AI models locally

well, I would want to learn how to set aceStep up, it broke when I tried to use dione to install it

Yeah,already wanted to open topic about Ace-Step. Basically what you have to do is to install Python 3.11 on your machine, then install uv via pip. After that clone the GitHub repo for Ace-Step 1.5. Don't use the classic Ace-Step, the new one sounds ay better. When git cloned, cd into it and run uv sync. When done uv run acestep. That's it at least on Windows.

Update: I am now able to run gpt-oss:20b so that's kinda cool I guess. You guys should really take a look at Anaconda and Miniconda prompts. It's pretty much Python with a bunch of AI and ML stuff already installed. I personally use Miniconda.
Also, take a look at Open-WebUI if you use Ollama. It has literally everything you need and lets you set about everything you would need to install:
conda create --name open-webui
conda activate open-webui
pip install open-webui
open-webui serve

then go to
localhost:8080 and have fun.

I don't like how it streams the response because it constantly interrupts NVDA, and I don't like the absense of headers, but hey it's open-sourced. Someone can fix it. Maybe even the AI model you run :d

16 gb ram. amd gpu and cpu. i use ollama and some others like openclaw. I also like useing LMstudio but some screenreaders have problems with it
if you want to use olllama. use ollama chat. github.com/blindsoftlgtm/ollama-chat-app. it is the best with nvda.

Anyone played with Vision Assistant Pro and using local models? I've switched over to LocalAI from Ollama because it has support for almost everything.
The addon supports text to speech, speech to text, captcha solving, system control, etc. I'm just playing around with it RN to see what I can get working with my 16gb card. Perhapse if it supported my NPU I could have the vision model run on that so nothing spills out of Vram.

What is LocalAI, do you mean you use Vision Assistant Pro with Ollama, or do you mean you switched from Ollama to another app called LocalAI?

The thing itself is called LocalAI. i should have been a little more specific. It does indeed support running Whisper and such which is pretty nice.

I tried running an ai model on this chip. Processor; Value: AMD Ryzen 3 7330U with Radeon Graphics, 2301 Mhz, 4 Core(s), 8 Logical Processor(s). And, it was kinder terrable

I have an rtx 3050 video card.
What music generators can I run on it that support file uploading?

Ace-Step 1.5, the normal models, not the XL ones. And of course MusicGen but this is very old.

I currently have a pretty nice use case. I use Gemma 4 e4b to verify and approve or deny Claude Code permissions.

Anyone running their own AI for NVDA AI addons? I do. I use this thing called Localai which is ran in a Docker container. The thing about it though is that it lets you run every type of model out there. You can run speech to text, vision, chatbot, etc. Ollama doesn't havethat flexibility, so you don't get full functionality with something like Vision Assistant Pro.

On another note, there is an optimized version of Gemma 4 26B that actually will fully fit in 16gb of vram

Yeah, I also could run it without optimisations and it would load into system RAM for some parts. But I've figured why do it if e4b is much faster in token generation and is accurate enough.