Xa i-OpenAI yazisa i-ChatGPT ngasekupheleni kuka-2022, yabangela ulonwabo kunye nenkxalabo. I-Generative AI ibonise amandla amangalisayo-ukwenza izincoko, ukusombulula iingxaki zekhowudi, kunye nokudala ubugcisa. Kodwa iphinde yaphakamisa ii-alam phakathi kweengcali zokusingqongileyo, abaphandi, kunye neengcali zetekhnoloji. Eyona nto ixhalabisayo? Usetyenziso olukhulu lwamandla olufunekayo ekuqeqesheni nasekuqhubeni iiModeli zoLwimi olukhulu (LLMs), oko kubuza imibuzo malunga nokuzinza kwazo kwexesha elide.
Njengoko ii-LLM ziqhubeka nokuseka amashishini afana nezemfundo kunye nokhathalelo lwempilo, impembelelo yazo ayinakungahoywa. Eli phepha liphakamisa umbuzo obalulekileyo: Ngaba ezi nkqubo zikrelekrele ziyakwazi ukuziphucula ngokwazo ukunciphisa ukusetyenziswa kwamandla kunye nokunciphisa unyawo lwazo lokusingqongileyo? Kwaye ukuba kunjalo, oku kungawuguqula njani ubume be-AI?
Siza kucazulula imiceli mngeni yamandla e-LLMs, ukusuka kuqeqesho ukuya kutsho, kwaye siphonononge iindlela ezintsha zokuzilungisa ezinokwenza i-AI izinze ngakumbi.
Ukuqonda i-AI Energy Challenge
Uqeqesho vs
Uqeqesho lukaGoogle kwiimodeli zeelwimi ezinkulu ezifana neGPT-4 okanye iPaLM ifuna isixa esikhulu sezixhobo zokubala. Umzekelo, uqeqesho lwe-GPT-3 luthathe amawaka ee-GPU ezibaleka iiveki, zisebenzisa amandla amaninzi njengamakhulu emizi yase-US ngonyaka. I-carbon footprint ixhomekeke kumxube wamandla wokunika amandla amaziko edatha. Nasemva koqeqesho, inqanaba lokujonga-apho iimodeli zibamba imisebenzi yehlabathi yokwenyani-yongeza ekusebenziseni amandla. Nangona amandla afunekayo kumbuzo omnye emancinci, xa sicinga ukuba kukho iibhiliyoni zonxibelelwano olunjalo ezenzeka kumaqonga ahlukeneyo yonke imihla, iba yingxaki enkulu.
Kutheni iiLLMs Zisebenzisa Amandla Kangaka?
Ubungakanani boMzekelo: IiLLM zanamhlanje zinovakalelo lweparamitha; banamabhiliyoni okanye iitriliyoni zeeparamitha ezifuna ubutyebi obuninzi ukuze ziqwalaselwe, zihlaziywe kwaye zigcinwe.
Izithintelo zeHardware: Ukusetyenziswa kweetshiphusi ezisekwe kwisilicon kukhawulelwe ngamandla azo okusebenza kwaye ke imfuno yamaqela e-GPUs okanye ii-TPUs ukwandisa ukusetyenziswa kwamandla ngokubonakalayo.
- Iimfuno zokupholisa: Amaziko eenkcukacha axhasa imithwalo yemisebenzi ephezulu yokubala afudumele kwaye iinkqubo zokupholisa zinokutya ukuya kuthi ga kwi-40% yamandla ukuba ayenzi amandla.
Umrhumo wokusiNgqongileyo noQoqosho
Iindleko ngokwemeko yokusingqongileyo ziquka ukukhutshwa kwekhabhoni ngokunjalo nokusetyenziswa kwamanzi ekubaleni ngelixa iindleko zokusebenza ziyingxaki kwiinkampani ezincinci ze-AI. Iindleko zonyaka zinokufikelela kwiibhiliyoni, nto leyo eyenza ukuzinza kube yinto ebalulekileyo kungekuphela nje kokusingqongileyo kodwa nomba wezoqoqosho.
Imodeli ye-AI yokuSetyenziswa kwamandla
Ukuqonda indlela ii-LLMs eziwasebenzisa ngayo amandla, masiyicazulule:
Ukusebenza kwe-AI | Ukusetyenziswa kwamandla (%) |
|---|---|
Isigaba soQeqesho | 60% |
Ukungeniswa (Imibuzo Eqhutywayo) | 25% |
Ukupholisa iziko leDatha | 10% |
Ukusebenza kweHardware | 5% |
I-Key Takeaway: Isigaba soqeqesho sihlala sinegalelo elikhulu ekusebenziseni amandla.
Ubuchule bokuZiphuhlisa
Abaphandi bajonge indlela ii-LLMs ezinokwandisa ngayo ukusetyenziswa kwamandla abo, ukudibanisa umsebenzi wesoftware kunye notshintsho lwehardware.
Umzekelo wokuPruna kunye nobungakanani
- Ukuthena: Iiparamitha ezingafunekiyo ezichaphazela ukuchaneka kwinqanaba elilinganiselweyo ziyasuswa, okukhokelela ekunciphiseni ubungakanani bemodeli ngaphandle kokuphazamisa ukuchaneka.
- Ubungakanani: Oku kunciphisa ukuchaneka (umzekelo, ukusuka kwi-32-bit ukuya kwi-8-bit) yedatha, enciphisa imemori kunye neemfuno zokubala.
Ubungakanani kunye nokuPruning ziluncedo kodwa xa zisetyenziswa kunye ne-feedback looops apho imodeli ikwaziyo ukumisela ukuba ngawaphi amalungu abalulekileyo kwaye ngawaphi amalungu anokwahlulwa ngokomlinganiselo emva koko iyasebenza kakhulu. Le yindawo entsha, kodwa amandla akhona kuthungelwano oluzenzelayo.
iNgqungquthela enamandla (Conditional Computation)
Ingcamango yokubala ngokwemiqathango yenza ukuba iimodeli zisebenzise kuphela ezo neurons okanye iileya ezihambelana nomsebenzi onikiweyo. Umzekelo, indlela kaGoogle yoMxube-weNgcali (i-MoE) yahlula inethiwekhi ibe yi-subnetworks ezikhethekileyo eziphucula uqeqesho kunye nokunciphisa ukusetyenziswa kwamandla ngokunciphisa inani leeparamitha ezisebenzayo.
Ukomelezwa kokuFunda kuThungelwano
Ukufunda okomeleza kunokwandisa ii-hyperparameters ezifana nezinga lokufunda kunye nobukhulu bebhetshi, ukuchaneka kokulinganisa kunye nokusetyenziswa kwamandla ukuqinisekisa ukuba iimodeli zisebenza ngokufanelekileyo.
UPhuculo lweNjongo ezininzi
Ukongeza ekulungiseleleni ukuchaneka, ii-LLMs zinokuphucula ezinye iinjongo: ukuchaneka, ukulinda, kunye nokusetyenziswa kwamandla, usebenzisa izixhobo ezifana neGoogle Vizier okanye iRay Tune. Kutsha nje, ukusebenza kakuhle kwamandla kuye kwaba yinjongo ebalulekileyo kwezi zikhokelo.
Izinto ezintsha zeHardware kunye ne-AI Co-Design
- IiSekethe eziDityanisiweyo eziKhethekileyo zesicelo (ii-ASICs): Iichips ezikhethekileyo zenjongo yokuphucula ukusebenza kakuhle ekuqhutyweni kwemisebenzi ye-AI.
- I-Neuromorphic Computing: Iichips eziphefumlelweyo zobuchopho, ezisekuphuhlisweni ukunciphisa ukusetyenziswa kwamandla xa kusenziwa i-neural network computations ziphantsi kophuhliso.
- IKhompyutha ye-Optical: Ukubala usebenzisa ukukhanya kunokoyisa imida yenkqubo ye-elektroniki ukuthoba usetyenziso lwamandla enkqubo.
Iinkqubo ze-AI ezidalwe ngokubambisana kwe-hardware kunye nesofthiwe zivumela ukulungiswa kwangaxeshanye kwe-algorithms ye-software kunye nezixhobo ze-hardware.
Ukuthelekisa iiTekhnoloji zokuSebenzisa amandla e-AI
Ubuchwephesha | Ukunciphisa amandla (%) | INzuzo yokuqala |
|---|---|---|
Umzekelo wokuPruna | 30% | Ukunciphisa iiparamitha zemodeli ezingafunekiyo |
Ubungakanani | 40% | Yehlisa ukuchaneka kokubala |
Ubalo oluneMiqathango (MoE) | 25% | Ivula imodeli eyimfuneko kuphela |
UkuFunda okomeleza | 15% | Itshintsha ngamandla ukusetyenziswa kwamandla |
I-Neuromorphic Computing | 50% | Ilinganisa ukusebenza kakuhle kwengqondo |
I-Hardware Co-Design (ASICs, i-Optical Chips) | 35% | Uphuhlisa i-AI-specific hardware ukuze usebenze kakhulu |
Iimodeli ze-AI zexesha elizayo ziya kudibanisa iindlela ezininzi zokufezekisa i-60-70% yokunciphisa amandla ngokubanzi.
Imicelimngeni yokuZiphucula i-AI
- Ukuchaneka koRhwebelwano : Ezinye iimpawu, ezifana nokuthenwa kunye nobungakanani, zinokubeka esichengeni ukuchaneka kancinane.
- IMida yeZiseko zeZiko leDatha: Sisasebenza phantsi kwentelekelelo yokuxhomekeka kwiitshiphusi zesilicon ezingasebenziyo.
- Izikhewu zeMilinganiso yokuSebenza kwamandla: Okwangoku akukho mgangatho wehlabathi wokulandelela ukusebenza kakuhle kwamandla.
- Ulawulo lukaRhulumente: Imithetho engqongqo yozinzo inokunyanzela ukwamkelwa kweemodeli ezisebenzayo.
Iimpembelelo zexesha elizayo
Ii-LLMs ezizisebenzelayo zinokucutha ukusetyenziswa kwamandla nge-20% okanye ngaphezulu kwiibhiliyoni zemibuzo, nto leyo eya kukhokelela kwindleko enkulu kunye nokongiwa kwe-emission. Oku kuhambelana nethagethi enguziro yehlabathi jikelele kwaye kuchaphazela amacandelo amaninzi:
- Ishishini : Ii-LLMs ezonga amandla zinokunyusa ukuthathwa kwenkonzo yabathengi kunye nohlalutyo.
- UPhando : Amanyathelo omthombo ovulekileyo afana noBuso obukwangayo anokukhawulezisa ukuvela kwezinto ezintsha.
- Umgaqo-nkqubo : Imigangatho yokungafihli kwamandla inokutyhala ukuziphucula njengesiqhelo.
Ukuqukumbela
Ii-LLMs ziye zazisa inqanaba elitsha lobuchule ekusetyenzweni kolwimi kodwa ingxaki yokusetyenziswa kwamandla azo yeyona nto ixhalabisayo. Nangona kunjalo, ubukrelekrele obunye obuye bavelisa ezi modeli bubonelela ngesisombululo. Ubuchwephesha obunjengokuthena, ubungakanani, ukubala ngokwemiqathango, kunye noyilo oludibeneyo lwehardware lubonisa ukuba kuyenzeka ukuyila iiLLM ezilawula ukusetyenziswa kwamandla azo. Njengoko uphando luqhubela phambili, umba uba ngaphantsi wokuba i-AI ezinzileyo inokwenzeka kwaye ngakumbi ukuba imboni yezobuchwephesha inokuhlangana ngokukhawuleza kangakanani ukuyifezekisa-ngaphandle kokuncama ukutsha kokusingqongileyo.
Iimbekiselo
- UBrown, T., et al. (2020). "IiModeli zoLwimi ngaBafundi abaDutywayo abambalwa." Ukuqhubela phambili kwiiNkqubo zoLwazi lwe-Neural , i-33, i-1877-1901. (Umthombo ocingelwayo wedatha yoqeqesho ye-GPT-3.)
- Strubell, E., Ganesh, A., & McCallum, A. (2019). "Iingqwalasela zaMandla kunye nePolisi yokuFunda ngokuNzulu kwi-NLP." Iinkqubo zeNtlanganiso yoNyaka ye-57 ye-ACL , 3645-3650. (Umthombo obonisa i-AI kwiindleko zamandla.)
- Fedus, W., et al. (2021). "Tshintshela iiTransformers: Ukukala kwiiModeli zeParameter yeTriyoni kunye neSparity eLula kwaye eSebenzayo." arXiv preprint arXiv:2101.03961 . (Isiseko sengxoxo yoMxube weeNgcali.)
- Patterson, D., et al. (2021). "Ukukhutshwa kweCarbon kunye noQeqesho oluKhulu lweNeural Network." arXiv preprint arXiv:2104.10350 . (Umthombo woqikelelo lwamandla oqeqesho.)
- UPhando lukaGoogle. (2023). "IVizier: Inkonzo yokuPhucula iBhokisi eMnyama." Ibhlog yeGoogle AI . (Isixhobo esibonisa isalathiso.)
