I-AI Ifaka Imephu Ukuxhumana Okufihliwe-Futhi Iqala Ukuqalisa

Kulokhu okuthunyelwe kwesihlanu nokokugcina kochungechunge lwethu lokubikezela isixhumanisi sisebenzisa i-Neptune ML, singena enqubweni yokucabanga: simisa isiphetho sokusebenzisa imodeli yethu ye-GNN eqeqeshiwe yokubikezela isixhumanisi. Ukuze uthole indawo yokugcina, sizosebenzisa i-API yeqoqo le-Neptune kanye nama-artifact emodeli agcinwe ku-S3. Ngephoyinti lokugcina libukhoma, sizokubuza ukubikezela kwesixhumanisi, sisebenzisa umbuzo we-Gremlin ukuze sihlonze ukuxhumana okungaba nethemba eliphezulu kugrafu yethu.

Njengamanje, sesivele silayishe idatha yenethiwekhi yokuxhumana nomphakathi ye-Twitch kuqoqo le-Neptune (njengoba kuchazwe esigabeni 1 salolu chungechunge), sikhiphe idatha sisebenzisa iphrofayela ye-ML (hlola Ingxenye 2 ukuze uthole imininingwane), sicubungule idatha (njengoba kuchazwe esigabeni 3), saqeqesha imodeli (bona Ingxenye 4), futhi manje sesilungele ukusebenzisa imodeli eqeqeshiwe ukuze sikhiqize izibikezelo.

Funda ingxenye 1 lapha ; ingxenye 2 lapha ; ingxenye 3 lapha; futhi ingxenye yesi-4 lapha .

UKWENZA ISIPHETHO

Masidale isiphetho se-inference sisebenzisa i-API yeqoqo kanye nama-artifact emodeli esinawo ku-S3. Njengokuvamile, sidinga indima ye-IAM enokufinyelela ku-S3 naku-SageMaker kuqala. Indima kufanele futhi ibe nenqubomgomo yokwethenjwa esivumela ukuthi siyengeze kuqoqo le-Neptune (inqubomgomo yokwethenjwa ingatholakala Engxenyeni 3 yalo mhlahlandlela). Kudingeka futhi sinikeze ukufinyelela ku-SageMaker ne-CloudWatch API kusuka ngaphakathi kwe-VPC, ngakho sidinga amaphoyinti okugcina e-VPC njengoba kuchazwe Engxenyeni 1 yalolu chungechunge.

Masisebenzise i-API ye-cluster ukuze senze isiphetho se-inference ngalo myalo we-curl:

 curl -XPOST https://(YOUR_NEPTUNE_ENDPOINT):8182/ml/endpoints \ -H 'Content-Type: application/json' \ -d '{ "mlModelTrainingJobId": YOUR_MODEL_TRAINING_JOB_ID, "neptuneIamRoleArn": "arn:aws:iam::123456789012:role/NeptuneMLNeptuneRole" }'

Singasebenzisa amapharamitha we-' exampleType ' kanye ' ne- exampleCount ' ukuze sikhethe uhlobo lwesibonelo lwe-EC2 oluzosetshenziselwa ukuqagela isixhumanisi, futhi sikhiphe isenzakalo esingaphezu kwesisodwa. Uhlu oluphelele lwamapharamitha lungatholakala lapha . Sizosebenzisa isibonelo esizenzakalelayo se-' ml.m5(d).xlarge ' njengoba ine-CPU ne-RAM eyanele kugrafu yethu encane, futhi lolu hlobo lwesibonelo lunconywe kokuthi infer_instance_recommendation.json ekhiqizwe ngemva kokuqeqeshwa kwemodeli esigabeni sethu sangaphambilini:

 { "disk_size": 12023356, "instance": "ml.m5d.xlarge", "mem_size": 13847612 }

I-API iphendula nge-ID yephoyinti lokugcina:

 {"id":"b217165b-7780-4e73-9d8a-5b6f7cfef9f6"}

Ngemuva kwalokho singabheka isimo se-inference endpoint ngalo myalo:

 curl https://YOUR_NEPTUNE_ENDPOINT:8182/ml/endpoints/INFERENCE_ENDPOINT_ID?neptuneIamRole='arn:aws:iam::123456789012:role/NeptuneMLNeptuneRole'

futhi uma iphendula ngokuthi ' isimo: InService ' kanje,

 { "endpoint": { "name": "YOUR_INFERENCE_ENDPOINT_NAME-endpoint", "arn": "...", "status": "InService" }, "endpointConfig": {...}, "id": "YOUR_INFERENCE_ENDPOINT_ID", "status": "InService" }

kusho ukuthi sesilungele ukuqala ukuyisebenzisa ngemibuzo egciniwe.

Iphoyinti lokugcina lingabuye libukwe futhi liphathwe kusukela kukhonsoli ye-AWS, ngaphansi kwe-SageMaker -> Inference -> Endpoints.

UKUBUZA NGESIPHELO

Masisebenzise indawo yokugcina ukuze sibikezele izixhumanisi ezintsha 'zokulandela' kugrafu. Ukuze senze lokho, sidinga ukukhetha i-vertex yomthombo wezixhumanisi ezintsha ezingaba khona.

Kuyasiza uma i-vertex yomthombo isivele inezixhumanisi, ngakho-ke sizosebenzisa lo mbuzo ukuze sithole i-vertex enenani eliphakeme kakhulu lezixhumanisi ezikhona (uxhumano lwe-outE ne-inE):

 gV() .group() .by() .by(bothE().count()) .order(local) .by(values, Order.desc) .limit(local, 1) .next()

Umphumela uba

 {v[1773]: 1440}

okusho ukuthi i-vertex ene-ID = 1773 inokuxhumana okungu-1440 (720 inE kanye ne-720 outE).

Ukuthola inani lamaphethelo aqala ku-node 1773 kanye nenani lamaphethelo aphela kuleyo nodi singasebenzisa le mibuzo:

 gV('1773').outE().count() gV('1773').inE().count()

Inani elifanayo loxhumo lwe-inE ne-outE lilindelekile ngenxa yokuthi idathasethi yokuqala iqukethe ubungane obufanayo, futhi sengeze idatha ngamaphethelo angemuva ukuze siyenze isebenze ngegrafu eqondisiwe ye-Neptune.

Manje ake sithole ukuxhumana okubikezelwe. Ukwenza lokho, sizosebenzisa umbuzo we-Gremlin ngezilandiso ze-Neptune ML . Sizosebenzisa i-inference endpoint kanye nendima ye-SageMaker eyengezwe kuqoqo le-DB ukuze sithole abasebenzisi okungenzeka umsebenzisi 1773 abalandele ngokuqiniseka (amathuba amancane okuba khona kwesixhumanisi ngokuya ngemodeli) okungenani engu-0.1 (10%) , kuyilapho singabandakanyi abasebenzisi lowo umsebenzisi 1773 asevele elandela:

 %%gremlin g.with('Neptune#ml.endpoint', 'YOUR_INFERENCE_ENDPOINT_NAME') .with('Neptune#ml.iamRoleArn', 'arn:aws:iam::123456789012:role/NeptuneMLSagemakerRole') .with('Neptune#ml.limit', 10000) .with('Neptune#ml.threshold', 0.1D) .V('1773') .out('follows') .with('Neptune#ml.prediction') .hasLabel('user') .not( __.in('follows').hasId('1773') )

Lokho kubuyisela imiphumela emi-4:

 "Result" "v[755]" "v[6086]" "v[6382]" "v[7005]"

Ngokwemodeli yethu, okungenani kunethuba elingu-10% lokuthi umsebenzisi 1773 uzolandela ngamunye walaba basebenzisi abangu-4. Mhlawumbe singayithuthukisa imodeli ngokwandisa inani lemisebenzi yokuqeqesha ifinyelele okungenani ku-10 njengoba kunconywe i-AWS, bese siqhathanisa ukusebenza kwemodeli ewumphumela kanye nezixhumanisi ezibikezelwe. Kuzinhlelo zokusebenza zangempela, ukwengeza iphrofayela yomsebenzisi nedatha yomsebenzi kugrafu nakho kuthuthukisa ukunemba kokuqagela.

Ukuxhumana okubikezelwe kumadathasethi enethiwekhi yokuxhumana nomphakathi kungasetshenziswa ukunikeza izincomo ezenziwe zaba ngezakho ezifana neziphakamiso zomngane nokuqukethwe, ukuthuthukisa ukuzibandakanya komsebenzisi nokukhula komphakathi, ukunciphisa ukuxokozela, nokuhlinzeka ngedatha eyengeziwe yokukhangisa okuqondiwe.

Nakuba sisebenzise ukubikezela kwesixhumanisi kudathasethi yenethiwekhi yokuxhumana nomphakathi, kungenye yezinhlelo eziningi ezingasetshenziswa zalobu buchwepheshe. Kusukela ezinjinini zokuncoma ukuya ekuthuthukiseni inethiwekhi, ukuqagela kwesixhumanisi kuyithuluzi elishintshashintshayo eliletha inani ngokudalula ubudlelwano obufihliwe ngaphakathi kwedatha. Njengoba izinhlelo zokusebenza ezisekelwe kumagrafu ziqhubeka nokukhula, amandla okubikezela isixhumanisi kuzo zonke izimboni athembisa imininingwane emisha, ukusebenza kahle, kanye nolwazi oluthuthukisiwe lwabasebenzisi.