Mhanya Deepseek (Kana Chero Imwe AI Modhi) Mumusha Wako Uine Mitsetse Mishoma Yekodhi

Munguva pfupi yapfuura, pane kudiwa kweyakavanzika chatbot sevhisi seimwe nzira yakakwana kune OpenAI's ChatGPT. Saka, ini ndinosarudza kuita imwe kumba uye kuita kuti iwanikwe kune wese wemba yangu padivi nenetiweki printer yangu uye NAS (OpenMediaVault) .

Kare, ndaikurudzira vanhu kushandisa Llama akatevedzana echiRungu mabasa uye Qwen akatevedzana echiChinese mabasa. Pakanga pasina yakavhurika-sosi modhi yakasimba zvakakwana mumitauro yakawanda tichienzanisa neyevaridzi (GPT/Claude).

Zvisinei, sezvatinoziva tose—zvinhu zvakachinja munguva pfupi yapfuura. Ndanga ndichishandisa DeepSeek-V2 apo neapo pese pandakaneta neQwen2.5 uye ndanga ndichidzokera kumashure neDeepSeek V2.5 uye V3 nekuda kwekushaikwa kwehardware. Asi DeepSeek haina kundisiya pasi, R1 inoita zvinokatyamadza uye inopa diki se1.5B!

Izvi zvinoreva kuti tinogona kuimhanyisa kunyangwe paCPU neruzivo rwakakura rwemushandisi. Sezvo vanhu vazhinji vaine maGPU ekutamba, kumhanya haisi nyaya. Kugadzira maLLM emunharaunda maitiro akarodha magwaro nemifananidzo mukana wakakura sezvo OpenAI ichimisa kushandiswa kwemaakaundi emahara.

Kunyangwe Kuisa Vhura WebUI neBundled Ollama Tsigiro iri nyore kwazvo neiyo yepamutemo-mutsara wekuraira:

 docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama

Asi kuwana RAG (Webhu yekutsvaga) kushanda hakusi nyore kune vanhu vazhinji, saka ndinoda kuwana imwe kunze kwebhokisi mhinduro.

Sezvandakataura mune yangu yekupedzisira positi , chiteshi chengarava chiyedzo chikuru chekuyedza akasiyana LLM stack. Asi hazvina kunaka chete kune izvo, zvakare iri-mu-imwe mhinduro yekuzvigashira wega maLLM emunharaunda neRAG inoshanda kunze kwebhokisi. Saka, ngatitangei kuishandisa kubva kutanga uye tinzwe wakasununguka kusvetuka nhanho sezvo vanhu vazhinji vasingatangi kubva kuOS yekumisikidza.

Kugadzirira Sistimu (Inosarudzika)

Sezvakangoita sepakutanga , enda kuburikidza nekuisa maitiro uchishandisa debian-11.6.0-amd64-netinst.iso

Wedzera kune sudoer usermod -aG sudo username wobva watangazve

(Sarudzo) Wedzera imwe shanduko

 fallocate -l 64G /home/swapfile chmod 600 /home/swapfile mkswap /home/swapfile swapon /home/swapfile

uye ita kuti swapfile irambe iripo nano /etc/fstab

 UUID=xxxxx-xxx swap swap defaults,pri=100 0 0 /home/swapfile swap swap defaults,pri=10 0 0

Tarisa ne swapon --show kana free -h

Dzima mutyairi weNouveau

 bash -c "echo blacklist nouveau > /etc/modprobe.d/blacklist-nvidia-nouveau.conf" bash -c "echo options nouveau modeset=0 >> /etc/modprobe.d/blacklist-nvidia-nouveau.conf" update-initramfs -u update-grub reboot

Isa zvinoenderana

 apt install linux-headers-`uname -r` build-essential libglu1-mesa-dev libx11-dev libxi-dev libxmu-dev gcc software-properties-common sudo git python3 python3-venv pip libgl1 git-lfs -y

(Sarudzo) Ita uninstall kana zvichidikanwa

 apt-get purge nvidia* apt remove nvidia* apt-get purge cuda* apt remove cuda* rm /etc/apt/sources.list.d/cuda* apt-get autoremove && apt-get autoclean rm -rf /usr/local/cuda*

Isa cuda-tokit uye cuda

 wget https://developer.download.nvidia.com/compute/cuda/12.4.1/local_installers/cuda-repo-debian11-12-4-local_12.4.1-550.54.15-1_amd64.deb sudo dpkg -i cuda-repo-debian11-12-4-local_12.4.1-550.54.15-1_amd64.debsudo cp /var/cuda-repo-debian11-12-4-local/cuda-*-keyring.gpg /usr/share/keyrings/ sudo add-apt-repository contrib sudo apt-get update sudo apt-get -y install cuda-toolkit-12-4 sudo apt install libxnvctrl0=550.54.15-1 sudo apt-get install -y cuda-drivers

Isa iyo NVIDIA Container Toolkit sezvo chiteshi chiri docker-yakavakirwa

 curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \ && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \ sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \ sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

Zvadaro sudo apt-get update uye sudo apt-get install -y nvidia-container-toolkit

Ita cuda post-install chiito nano ~/.bashrc

 export PATH=/usr/local/cuda-12.4/bin${PATH:+:${PATH}} export LD_LIBRARY_PATH=/usr/local/cuda-12.4/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

Zvadaro sudo update-initramfs -u , ldconfig kana source ~/.bashrc kushandisa shanduko

mushure mekutangazve, simbisa ne nvidia-smi uye nvcc --version

Isa Miniconda (Sarudzo, kwete yechiteshi)

 wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh && sudo chmod +x Miniconda3-latest-Linux-x86_64.sh && bash Miniconda3-latest-Linux-x86_64.sh

Docker & Harbour

Isa docker

 # Add Docker's official GPG key: sudo apt-get update sudo apt-get install ca-certificates curl sudo install -m 0755 -d /etc/apt/keyrings sudo curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc sudo chmod a+r /etc/apt/keyrings/docker.asc # Add the repository to Apt sources: echo \ "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian \ $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \ sudo tee /etc/apt/sources.list.d/docker.list > /dev/null sudo apt-get update sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Ita post-install ye docker pasina sudo

 sudo groupadd docker sudo usermod -aG docker $USER newgrp docker docker run hello-world

Manual install Harbor

 git clone https://github.com/av/harbor.git && cd harbor ./harbor.sh ln

Simbisa harbor --version

Wedzera neRAG tsigiro kune defaults harbor defaults add searxng

Shandisa harbor defaults list kutarisa, ikozvino kune matatu masevhisi anoshanda: ollama , webui , searxng

Mhanya harbor up kuunza masevhisi aya mudocker

Shandisa harbor ps se docker ps , uye harbor logs kuti uone matanda anorema

Ikozvino iyo yakavhurika-webui yekumberi iri kushanda 0.0.0.0:33801 uye inogona kuwanikwa kubva http://localhost:33801 kana vatengi kubva kuLAN ine server's IP kero.

Tarisa kushandiswa kweVRAM watch -n 0.3 nvidia-smi

Monitor log harbor up ollama --tail kana harbor logs

Yese mirairo yeollama inoshandiswa senge harbor ollama list

Yave nguva yekuwana kubva kune mamwe maturusi (desktop/mobile) kunyoresa admin account uye kurodha modhi izvozvi.

Kushandisa Local LLM

Mushure mekupinda neakaundi ye admin, tinya kumusoro kurudyi avatar icon, vhura Admin Panel ipapo Settings , kana kungo pinda kuburikidza `http://ip:33801/admin/settings.

Dzvanya Models , uye kumusoro kurudyi tinya Manage Models inoita sebhatani rekurodha.

Isa deepseek-r1 kana chero imwe modhi mubhokisi rezvinyorwa pazasi Pull a model from Ollama.com uye tinya bhatani rekurodha padivi rekurudyi.

Mushure mekudhawunirodha modhi, inogona kuzorodzwa uye iyo ichangobva kudhawunirodha modhi ichashandiswa pasi peiyo inodonha menyu paNew New Chat (yekumba) peji.

Ikozvino, haisi kungomhanyisa chatbot imwe nzira yeChatGPT, asiwo inoshanda zvizere API imwe nzira yeOpenAI API, pamwe neyakavanzika yekutsvaga injini imwe nzira kuGoogle!

webui inowanikwa mukati meLAN kuburikidza ne: http://ip:33801

ollama inowanikwa mukati meLAN kuburikidza ne: http://ip:33821

searching inowanikwa mukati meLAN kuburikidza ne: http://ip:33811

Fonera Ollama API nechero application ine LLM API yekubatanidza:

 http://ip:33821/api/ps http://ip:33821/v1/models http://ip:33821/api/generate http://ip:33821/v1/chat/completionsb