Indlela yoKwakha iModeli yoLwimi oluNcinci (TLM) kwiRuby: ISikhokelo seNyathelo ngeNyathelo

Kweli nqaku, siza kuhamba ngendlela yokwenza imodeli yolwimi olula kakhulu usebenzisa iRuby. Ngelixa iiMifuziselo zoLwimi olukhulu lokwenyani (LLMs) zifuna izixa ezikhulu zedatha kunye nezixhobo zokubala, sinokuyila imodeli yokudlala ebonisa uninzi lweengqikelelo ezingundoqo emva kokwenziwa kolwimi. Kumzekelo wethu, siya kwakha imodeli ye-Markov Chain esisiseko "efunda" kumbhalo wokufakwayo kwaye emva koko ivelise isicatshulwa esitsha ngokusekwe kwiipateni eziyibonileyo.

Qaphela: Esi sifundo senzelwe iinjongo zemfundo kwaye sibonakalisa indlela elula yokwenziwa kolwimi. Ayikokuthatha indawo ye-LLMs zale mihla zokufunda nzulu njenge-GPT-4 kodwa yintshayelelo yezimvo ezisisiseko.

Isiqulatho

1. Ukuqonda iSiseko seMifuziselo yeeLwimi
2. Ukuseta indawo yakho yeRuby
3. Ukuqokelelwa kweeNkcukacha kunye nokuLungiswa kwangaphambili
4. Ukwakha iModeli yeChain yeMarkov
5. Ukuqeqesha uMzekelo
6. UkuVelisa kunye nokuVavanya isicatshulwa
7. Ukuqukumbela

Ukuqonda iSiseko seMifuziselo yeeLwimi

IModeli yoLwimi yinkqubo enika amathuba okulandelelana kwamagama. Embindini walo, iyilelwe ukubamba ukwakheka kweenkcukacha-manani zolwimi ngokufunda ukuba nokwenzeka kolandelelwano oluthile kwimeko ethile. Oku kuthetha ukuba imodeli ihlalutya imizimba emikhulu yesicatshulwa ukuqonda indlela amagama aqhele ukulandelelana ngayo, ngaloo ndlela ivumela ukuba iqikelele ukuba leliphi igama okanye ibinzana elinokuza ngokulandelayo ngokulandelelana. Ubunakho obunjalo bungundoqo kungekuphela nje kwimisebenzi efana nokuveliswa kokubhaliweyo kunye nokugqibezela ngokuzenzekelayo kodwa nakwiintlobo ezahlukeneyo zokusetyenzwa kolwimi lwendalo (NLP), kubandakanywa uguqulelo, ushwankathelo, kunye nohlalutyo lweemvakalelo.

Imifuziselo yolwimi olukhulu lwangoku (LLMs) olufana ne-GPT-4 isebenzisa ubuchule bokufunda obunzulu kunye neeseti zedatha ezinkulu ukuthabatha iipatheni ezinzima kulwimi. Basebenza ngokucubungula igalelo lesicatshulwa kusetyenziswa iileyironi ezininzi ezenziweyo, ezibenza ukuba baqonde kwaye bavelise isicatshulwa esifana nomntu ngobuchule obumangalisayo. Nangona kunjalo, emva kwezi nkqubo zintsonkothileyo kukho ingcamango efanayo esisiseko: ukuqonda kunye nokuqikelela ukulandelelana kwamagama ngokusekelwe kwizinto ezinokwenzeka ezifundiweyo.

Enye yeendlela ezilula zokulinganisa ulwimi nge- Markov Chain . I-Markov Chain yimodeli yezibalo esebenza kwingqikelelo yokuba ukubakho kwegama elinokwenzeka kuxhomekeke kuphela kwiseti elinganiselweyo yamagama andulelayo, kunembali yonke yesicatshulwa. Le ngcamango yaziwa ngokuba yipropati kaMarkov. Ngokwezinto eziphathekayo, imodeli ithatha ukuba igama elilandelayo ngokulandelelana linokuqikelelwa kuphela ngokujonga awona magama amvanje (amagama) - ukulula okwenza ingxaki ngekhompyutha ithatheke ngakumbi ngelixa isabamba iipateni eziluncedo kwidatha.

Kwimodeli yolwimi esekwe kuMarkov Chain:

• Imeko yexesha elizayo (igama elilandelayo) ixhomekeke kuphela kwimeko yangoku (amagama angaphambili): Oku kuthetha ukuba sakuba siwazi amagama ambalwa okugqibela (amiselwa ngolandelelwano lomfuziselo), sinomxholo owaneleyo wokuqikelela okuza kulandela. Imbali yonke yencoko okanye umbhalo ayifuni kuqwalaselwa, okunciphisa ubunzima.
• Sakha unikezelo olunokwenzeka lokuba leliphi igama eliza kulandela ngokunikwa igama (amagama) elandulelayo: Njengoko imodeli iqeqeshelwa ikhorasi yesicatshulwa, ifunda ukubakho kwamagama ahlukeneyo alandela ulandelelwano olunikiweyo. Olu nikezelo lokwenzeka lusetyenziswa ngexesha lesigaba sesizukulwana ukukhetha igama elilandelayo kulandelelwano, ngokuqhelekileyo kusetyenziswa inkqubo yesampulu engaqhelekanga ehlonipha okunokwenzeka okufundiweyo.

Ekuphumezeni kwethu, siza kusebenzisa "umyalelo" oqwalaselweyo ukujonga ukuba mangaphi amagama angaphambili ekufuneka aqwalaselwe xa kuqikelelwa. Umyalelo ophezulu ubonelela ngomxholo othe kratya, onokuthi ube nesiphumo sokuhambelana ngakumbi kunye nesicatshulwa esifanelekileyo ngokwemeko, njengoko imodeli inolwazi oluthe kratya malunga nento eza ngaphambili. Ngokuchaseneyo, ulandelelwano olusezantsi luzisa ukungakhethi okuthe kratya kwaye kunokukhokelela kubuchule obungakumbi, nangona bungenakucingeleka, ukulandelelana kwamagama. Oku kuthungelana phakathi kokuhambelana nokuyila yeyona nto iphambili kuqwalaselo kumzekelo wolwimi.

Ngokuqonda le migaqo isisiseko, sinokuxabisa zombini ukulula kweemodeli zeMarkov Chain kunye neengcinga ezisisiseko ezixhasa iimodeli zolwimi oluntsonkothileyo lwe-neural. Lo mbono wandisiweyo awuncedi nje kuphela ekubambeni ubuxhakaxhaka beenkcukacha-manani emva koqikelelo lolwimi kodwa ukwabeka isiseko sokuzama iindlela eziphucuke ngakumbi zokusetyenzwa kolwimi lwendalo.

Ukuseta indawo yakho yeRuby

Ngaphambi kokuba uqalise, qiniseka ukuba ufake iRuby kwinkqubo yakho. Unokujonga inguqulo yakho yeRuby ngokuqhuba:

 ruby -v

Ukuba iRuby ayifakwanga, ungayikhuphela kwi-ruby-lang.org .

Kwiprojekthi yethu, unokufuna ukwenza uluhlu olunikezelweyo kunye nefayile:

 mkdir tiny_llm cd tiny_llm touch llm.rb

Ngoku ulungele ukubhala ikhowudi yakho yeRuby.

Ukuqokelelwa kweeNkcukacha kunye nokuLungiswa kwangaphambili

Ukuqokelela Idatha yoQeqesho

Kwimodeli yolwimi, udinga i-text corpus. Ungasebenzisa nayiphi na ifayile yombhalo yoqeqesho. Kumzekelo wethu olula, ungasebenzisa isampulu encinci yesicatshulwa, umzekelo:

 sample_text = <<~TEXT Once upon a time in a land far, far away, there was a small village. In this village, everyone knew each other, and tales of wonder were told by the elders. The wind whispered secrets through the trees and carried the scent of adventure. TEXT

Ukulungiswa kweDatha

Ngaphambi koqeqesho, kuluncedo ukuqhubela phambili isicatshulwa:

• I-Tokenization: Yahlula isicatshulwa kumagama.
• Ukuqheleka: Ukhetho lokuguqula umbhalo ube ngoonobumba abancinci, susa iziphumlisi, njl.

Ngeenjongo zethu, indlela kaRuby's String#split isebenza kakuhle ngokwaneleyo kwithokheni.

Ukwakha iModeli yeChain yeMarkov

Siza kudala iklasi yeRuby egama MarkovChain ukuze ifake ukuziphatha komzekelo. Iklasi iya kubandakanya:

• Isiqalisi sokuseta ulandelelwano (inani lamagama andulelayo) kwitsheyini.
• Indlela train eyakha ikhonkco ukusuka kumbhalo wokufakwayo.
• Indlela generate evelisa isicatshulwa esitsha ngesampulu kwikhonkco.

Ngezantsi yikhowudi epheleleyo yemodeli:

 class MarkovChain def initialize(order = 2) @order = order # The chain is a hash that maps a sequence of words (key) to an array of possible next words. @chain = Hash.new { |hash, key| hash[key] = [] } end # Train the model using the provided text. def train(text) # Optionally normalize the text (eg, downcase) processed_text = text.downcase.strip words = processed_text.split # Iterate over the words using sliding window technique. words.each_cons(@order + 1) do |words_group| key = words_group[0...@order].join(" ") next_word = words_group.last @chain[key] << next_word end end # Generate new text using the Markov chain. def generate(max_words = 50, seed = nil) # Choose a random seed from the available keys if none is provided or if the seed is invalid. if seed.nil? || [email protected]?(seed) seed = @chain.keys.sample end generated = seed.split while generated.size < max_words # Form the key from the last 'order' words. key = generated.last(@order).join(" ") possible_next_words = @chain[key] break if possible_next_words.nil? || possible_next_words.empty? # Randomly choose the next word from the possibilities. next_word = possible_next_words.sample generated << next_word end generated.join(" ") end end

Ingcaciso yeKhowudi

• 
  

  **Ukuqaliswa:**Umakhi initialize umisela ulandelelwano (okuhlala kukho isi-2) kwaye kudala ihashi engenanto yekhonkco lethu. I-hash inikwe ibhloko engagqibekanga ukuze isitshixo esitsha ngasinye siqale njengoluhlu olungenanto.

• 
  

  **Uqeqesho loMzekelo:**Indlela train ithatha umtya wesicatshulwa, iwenze ibe yesiqhelo, kwaye yahlule ibe ngamagama. Ukusebenzisa each_cons , idala amaqela alandelelanayo amagama order + 1 . Amagama order lokuqala asebenza njengesitshixo, kwaye igama lokugqibela lihlonyelwa kuluhlu lokuqhubeka okunokwenzeka kweso sitshixo.

• 
  

  **Ukuvelisa okubhaliweyo:**Indlela generate iqala ngeqhosha lembewu. Ukuba akukho nanye enikiweyo, iqhosha elingakhethiyo liyakhethwa. Yakha ngokuphindaphinda ulandelelwano ngokujonga amagama order lokugqibela kunye nesampuli yegama elilandelayo de kufikelelwe kwelona nani liphezulu lamagama.

Ukuqeqesha uMzekelo

Ngoku ukuba sineklasi yethu MarkovChain , masiyiqeqeshe kwidatha yesicatshulwa.

 # Sample text data for training sample_text = <<~TEXT Once upon a time in a land far, far away, there was a small village. In this village, everyone knew each other, and tales of wonder were told by the elders. The wind whispered secrets through the trees and carried the scent of adventure. TEXT # Create a new MarkovChain instance with order 2 model = MarkovChain.new(2) model.train(sample_text) puts "Training complete!"

Xa uqhuba ikhowudi engentla (umzekelo, ngokuyigcina kwi llm.rb kunye nokusebenzisa ruby llm.rb ), imodeli iya kuqeqeshwa ngokusebenzisa isicatshulwa esinikiweyo.

UkuVelisa kunye nokuVavanya isicatshulwa

Nje ukuba imodeli iqeqeshwe, unokuvelisa isicatshulwa esitsha. Makhe songeze ikhowudi ukuvelisa kunye nokuprinta isampulu yesicatshulwa:

 # Generate new text using the trained model. generated_text = model.generate(50) puts "Generated Text:" puts generated_text

Ungazama nokubonelela ngembewu yokuvelisa umbhalo. Umzekelo, ukuba uyasazi esinye sezitshixo kumzekelo (njenge "once upon" ), ungenza:

 seed = "once upon" generated_text_with_seed = model.generate(50, seed) puts "\nGenerated Text with seed '#{seed}':" puts generated_text_with_seed

Ngokuzama ngeembewu ezahlukeneyo kunye neeparamitha (njengokulandelelana kunye nenani eliphezulu lamagama), unokubona ukuba isiphumo siyahluka njani.

Umzekelo opheleleyo: Uqeqesho kunye noVavanyo lweLLM encinci

Nasi iskripthi esipheleleyo seRuby esidibanisa onke la manyathelo angentla:

 #!/usr/bin/env ruby # llm.rb # Define the MarkovChain class class MarkovChain def initialize(order = 2) @order = order @chain = Hash.new { |hash, key| hash[key] = [] } end def train(text) processed_text = text.downcase.strip words = processed_text.split words.each_cons(@order + 1) do |words_group| key = words_group[0...@order].join(" ") next_word = words_group.last @chain[key] << next_word end end def generate(max_words = 50, seed = nil) if seed.nil? || [email protected]?(seed) seed = @chain.keys.sample end generated = seed.split while generated.size < max_words key = generated.last(@order).join(" ") possible_next_words = @chain[key] break if possible_next_words.nil? || possible_next_words.empty? next_word = possible_next_words.sample generated << next_word end generated.join(" ") end end # Sample text data for training sample_text = <<~TEXT Once upon a time in a land far, far away, there was a small village. In this village, everyone knew each other, and tales of wonder were told by the elders. The wind whispered secrets through the trees and carried the scent of adventure. TEXT # Create and train the model model = MarkovChain.new(2) model.train(sample_text) puts "Training complete!" # Generate text without a seed generated_text = model.generate(50) puts "\nGenerated Text:" puts generated_text # Generate text with a specific seed seed = "once upon" generated_text_with_seed = model.generate(50, seed) puts "\nGenerated Text with seed '#{seed}':" puts generated_text_with_seed

Ukuqhuba iSikripthi

1. Gcina iscript njenge llm.rb .
2. Vula i-terminal yakho kwaye ukhangele kulawulo oluqulathe llm.rb .
3. Qhuba iscript usebenzisa:

 ruby llm.rb

Kuya kufuneka ubone imveliso ebonisa ukuba imodeli iqeqeshiwe kwaye emva koko imizekelo emibini yombhalo ovelisiweyo.

Umlinganiselo

Le theyibhile ilandelayo ishwankathela ezinye iimetriki zombonakaliso kwiinguqulelo ezahlukeneyo zomiliselo lwethu oluNcinci lweLLM. I-metric nganye ichazwe ngezantsi:

• Umzekelo: Igama okanye isichongi soguqulelo lwemodeli yolwimi.
• Umyalelo: Inani lamagama angaphambili asetyenziswe kwi-Markov Chain ukuqikelela igama elilandelayo. Umyalelo ophezulu ngokubanzi uthetha ukuba kusetyenziswe umxholo ongaphezulu, okunokwandisa ukuhambelana.
• Ixesha loQeqesho (ms): Ixesha eliqikelelweyo elithathiweyo ukuqeqesha imodeli kwidatha yombhalo enikeziweyo, elinganiselwa kwii-milliseconds.
• Ixesha lesiZukulwana (ms): Ixesha elifunekayo ukuvelisa isampula yokuphuma kombhalo, kulinganiswe ngee-milliseconds.
• Ukusetyenziswa kweMemori (MB): Ubungakanani bememori esetyenziswe yimodeli ngexesha loqeqesho kunye nesizukulwana.
• Ukulinganisa ukuyondelelana: Ukulinganisa okuzimeleyo (ngaphandle kwesi-5) kubonisa indlela ehambelanayo okanye efanelekileyo ngayo umxholo wesicatshulwa.

Apha ngezantsi yitheyibhile yophawulo enedatha yebenchmark:

Ezi mpawu zibonelela ngombono okhawulezayo worhwebo phakathi kolungelelwaniso lweemodeli ezahlukeneyo. Njengoko iodolo isanda, imodeli ithatha ixesha elide ukuqeqesha kunye nokuvelisa umbhalo, kwaye isebenzisa imemori engaphezulu. Nangona kunjalo, oku kunyuka kokusetyenziswa kwemithombo kaninzi kuhamba kunye nokuphuculwa kokuhambelana kombhalo owenziweyo.

Ukuqukumbela

Kule tutorial, sibonise indlela yokwenza imodeli yolwimi olula kakhulu usebenzisa iRuby. Ngokusebenzisa ubuchule beMarkov Chain, sakhe inkqubo ethi:

• Uqeqesha kwisampulu yesicatshulwa ngokufunda utshintsho lwamagama.
• Uvelisa isicatshulwa esitsha ngokusekelwe kwiipateni ezifundiweyo.

Ngelixa le modeli yezinto zokudlala ikude lee kwii-LLM zomgangatho wemveliso, isebenza njengenyathelo lokuqonda indlela iimodeli zolwimi ezisebenza ngayo kwinqanaba elisisiseko. Unokwandisa kule ngcamango ngokubandakanya ubuchule obuphambili, ukuphatha iziphumlisi ngcono, okanye ukudibanisa iRuby kunye namathala eencwadi okufunda ngomatshini kwiimodeli ezinobunkunkqele.

Ikhowudi yolonwabo!