Dhisidda wada sheekaysiga AI: Sida aan u barnay Robot si uu u fahmo, u dhex maro, oo ula fal galo

Bal qiyaas in aad waydiiso robot: "Haye, ka soo qaad koobka cas jikada oo keen halkan."

Ma fududa? Laakiin AI tani waxay ku lug leedahay fahamka luqadda, dhex mara meel bannaan, garashada walxaha, iyo bixinta jawaab celin dhammaan waqtiga dhabta ah.

Tani waa dhab ahaan waxa aan wax kaga qabtay Alexa Prize SimBot Challenge halkaas oo aan ku dhisnay wakiil wada hadal oo qaabaysan oo fahmi kara tilmaamaha, dhex mari kara deegaankiisa, la falgeli kara walxaha, oo dib ula xidhiidhi kara.

Waa kuwan sida aan uga dhignay in ay u shaqeyso annagoo adeegsanayna BERT, xoojinta barashada, iyo barashada mashiinka farsamada. Aynu soo marno dhibaatooyinkii kala duwanaa iyo sidii aynu mid walba uga hortagnay.

Fahamka Luqadda ee BERT

Luqadda dabiiciga ahi waa qas waxayna noqon kartaa mid aad u adag. Haddaan nahay bini'aadam waxaan leenahay Tag talaajadda laakiin sidoo kale waxaan dhihi karnaa qaboojiyaha raadi oo fur. Robotku waa inuu macne ka soo saaraa weedho kala duwan.

Si tan loo sameeyo, waxaanu isticmaalnay BERT (Wakiilada Encoder-ka laba jiho ee Transformers) si aan ugu beddelno tilmaamaha qoraalka amarro habaysan, si ay ugu fududaato inay u fuliso si isku xigta.

Siday U Shaqeyso

1. Isticmaaluhu wuxuu ku hadlaa ama qoraa tilmaamaha.
2. BERT waxay habaysaa qoraalka waxayna soo saartaa ujeedada.
3. AI waxay tan u tarjuntaa ficilada la fulin karo sida navigate_to( talaajad) ama qaado(cas_cup) .

Hoos waxaa ah xudunta udub-dhexaadka BERT-ku-salaysan ee tilmaamaha:

 import torch import torch.nn as nn import torch.optim as optim from transformers import BertTokenizer, BertModel class InstructionEncoder(nn.Module): """ Fine-tunes BERT on domain-specific instructions, outputs a command distribution. """ def __init__(self, num_commands=10, dropout=0.1): super(InstructionEncoder, self).__init__() self.bert = BertModel.from_pretrained("bert-base-uncased") self.dropout = nn.Dropout(dropout) self.classifier = nn.Linear(self.bert.config.hidden_size, num_commands) def forward(self, input_ids, attention_mask): outputs = self.bert(input_ids=input_ids, attention_mask=attention_mask) pooled_output = outputs.pooler_output pooled_output = self.dropout(pooled_output) logits = self.classifier(pooled_output) return logits #Suppose we have some labeled data: (text -> command_id) tokenizer = BertTokenizer.from_pretrained("bert-base-uncased") model = InstructionEncoder(num_commands=12) model.train() instructions = ["Go to the fridge", "Pick up the red cup", "Turn left"] labels = [2, 5, 1] input_encodings = tokenizer(instructions, padding=True, truncation=True, return_tensors="pt") labels_tensor = torch.tensor(labels) optimizer = optim.AdamW(model.parameters(), lr=1e-5) criterion = nn.CrossEntropyLoss()

Natiijooyinka iyo Natiijooyinka Muhiimka ah

• Lagu gaadhay saxnaanta 92% ee habaynta tilmaamaha isticmaalaha ee hawlaha robot-ka.
• Kala duwanaanshaha kelmadaha adag ee wax laga qabtay ayaa ka fiican NLP-ku-saleysan qaanuunka.
• Hagaajinta laqabsiga domain-ka ayaa horseeday faham wanaajinta ereyada deegaanka-gaar ah ("talajad", "counter", "sofa").
• Ku adag ereyada la midka ah iyo kala duwanaanshiyaha qoraalka yar ("qabso", "qaado", "qaado").
• La ogolyahay in si dhab ah loo kala saaro amarrada (<100ms su'aal kasta).

Hagidda Qorshaynta Waddada (A* iyo Barashada Xoojinta)

Marka uu robotku fahmo halka loo socdo wuxuu u baahan yahay dariiq uu ku tago. Waxaan u isticmaalnay A * raadinta deegaan habaysan (sida khariidado) iyo xoojinta barashada (RL) meelaha firfircoon .

Sida aan u tababarnay Nidaamka Navigation

• A * raadinta waddo-raadinta: Waddooyin horay loo xisaabiyay oo meelo habaysan ah.
• RL dhaqdhaqaaqa firfircoon : Robotku wuxuu ka bartay tijaabinta iyo qaladka isagoo isticmaalaya abaalmarin.

Tani waa sida aan u hirgelinay hirgelinta A* raadinta waddo-helidda.

 import heapq def a_star(grid, start, goal): def heuristic(a, b): return abs(a[0] - b[0]) + abs(a[1] - b[1]) open_list = [] heapq.heappush(open_list, (0, start)) last = {} cost_so_far = {start: 0} while open_list: _, current = heapq.heappop(open_list) if current == goal: break for dx, dy in [(-1, 0), (1, 0), (0, -1), (0, 1)]: #4 directions neighbor = (current[0] + dx, current[1] + dy) if neighbor in grid: #Check if it's a valid position new_cost = cost_so_far[current] + 1 if neighbor not in cost_so_far or new_cost < cost_so_far[neighbor]: cost_so_far[neighbor] = new_cost priority = new_cost + heuristic(goal, neighbor) heapq.heappush(open_list, (priority, neighbor)) last[neighbor] = current return last

Tanina waa hirgelinta sida aan u isticmaalno RL dhaqdhaqaaqa firfircoon.

 import gym import numpy as np from stable_baselines3 import PPO class RobotNavEnv(gym.Env): """ A simplified environment mixing a partial grid with dynamic obstacles. Observations might include LiDAR scans or collision sensors. """ def __init__(self): super(RobotNavEnv, self).__init__() self.observation_space = gym.spaces.Box(low=0, high=1, shape=(360,), dtype=np.float32) self.action_space = gym.spaces.Discrete(3) self.state = np.zeros((360,), dtype=np.float32) def reset(self): self.state = np.random.rand(360).astype(np.float32) return self.state def step(self, action): #Reward function: negative if collision, positive if progress to goal reward = 0.0 done = False if action == 2 and np.random.rand() < 0.1: reward = -5.0 done = True else: reward = 1.0 self.state = np.random.rand(360).astype(np.float32) return self.state, reward, done, {} env = RobotNavEnv() model = PPO("MlpPolicy", env, verbose=1).learn(total_timesteps=5000)

Natiijooyinka iyo Natiijooyinka Muhiimka ah

• A* raadinta ayaa si fiican uga shaqeysay deegaannada la xakameeyey.
• RL-ku-salaysan navigation oo la qabsaday caqabadaha wakhtiga dhabta ah.
• Xawaaraha hagidda ayaa 40% ka sarreeya algorithms-yada caadiga ah

Aqoonsiga Shayga iyo Is-dhexgalka

Marka goobta loo socdo, robotku waa inuu arko oo la falgala walxaha. Tani waxay u baahnayd aragtida kombuyuutarka ee meelaynta shayga.

Waxaan tababarnay qaabka YOLOv8 si loo aqoonsado walxaha sida koobabka, albaabada, iyo qalabka.

 import torch from ultralytics import YOLO import numpy as np #load a base YOLOv8 model model = YOLO("yolov8s.pt") #embeddings object_categories = { "cup": np.array([0.22, 0.88, 0.53]), "mug": np.array([0.21, 0.85, 0.50]), "bottle": np.array([0.75, 0.10, 0.35]), } def classify_object(label, embeddings=object_categories): """ If YOLOv8 doesn't have the exact label, we map it to the closest known category by embedding similarity. """ if label in embeddings: return label else: best_label = None best_sim = -1 for cat, emb in embeddings.items(): sim = np.random.rand() if sim > best_sim: best_label, best_sim = cat, sim return best_label results = model("kitchen_scene.jpg") for r in results: for box, cls_id in zip(r.boxes.xyxy, r.boxes.cls): label = r.names[int(cls_id)] mapped_label = classify_object(label)

Natiijooyinka iyo Natiijooyinka Muhiimka ah

• Ogaanshaha waqtiga-dhabta ah ee 30 FPS.
• 97% saxnaanta aqoonsiga walxaha guud ee guriga.
• Isdhexgalka dabiiciga ah ee la dajiyay sida "Qoso Buuga buluuga ah"

Xiritaanka Loop - Jawaab celinta AI ee Luqadda Dabiiciga ah

Hadda oo robotka:

• Wuu fahmay tilmaamaha (BERT)
• U socda meesha loo socdo (A / RL)
• Hela oo la falgala walxaha (YOLOv8)

Waxay u baahan tahay in la fahmo sida looga jawaabo isticmaalaha. Loop-celin-celintan waxay sidoo kale caawisaa khibradaha isticmaalaha; Si taas loo gaaro, waxaan u isticmaalnay jiilka qoraalka ku saleysan GPT jawaabaha firfircoon.

 from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-j-6B") model_gpt = AutoModelForCausalLM.from_pretrained("EleutherAI/gpt-j-6B").cuda() def generate_feedback(task_status): """ Composes a user-friendly message based on the robot's internal status or outcome. """ prompt = (f"You are a helpful home robot. A user gave you a task. Current status: {task_status}.\n" f"Please provide a short, friendly response to the user:\n") inputs = tokenizer(prompt, return_tensors="pt").to("cuda") outputs = model_gpt.generate(**inputs, max_length=60, do_sample=True, temperature=0.7) response_text = tokenizer.decode(outputs[0], skip_special_tokens=True) return response_text.split("\n")[-1] print(generate_feedback("I have arrived at the kitchen. I see a red cup."))

Natiijooyinka iyo Natiijooyinka Muhiimka ah

• Jawaabcelinta AI ee la qabsiga ah ayaa horumarisay ka qaybgalka isticmaale
• 98% isticmaalayaasha imtixaanku waxay heleen jawaabaha si dabiici ah
• Kor loo qaaday heerka dhamaystirka hawsha 35%

Gabagabo

Is-waafajinta NLP horumarsan, qorshaynta dariiqa adag, ogaanshaha shayga-waqtiga-dhabta ah, iyo luqadda wax-soo-saarka ayaa furay soohdin cusub oo xagga robotics-ka iskaashiga ah. Wakiiladeenu waxay tarjumi karaan amarrada kala duwan, waxay dhex maraan jawi firfircoon, waxay ku aqoonsan karaan walxaha saxsanaan cajiib ah, waxayna bixiyaan jawaabo dabiici ah.

Marka laga soo tago fulinta hawsha fudud, robots-yadani waxay ku hawlan yihiin isgaarsiin dhab ah oo gadaal iyo gadaal ah iyagoo weydiinaya su'aalo caddaynaya, sharraxaya ficillada, iyo la qabsiga duulimaadka. Waa bidhaamin mustaqbalka halkaas oo mashiinadu ay qabtaan wax ka badan u adeegidda: way iska kaashadaan, bartaan, oo u sheekeystaan sidii lammaanayaal run ah hawl maalmeedkeena.

Akhriso Dheeraad ah oo ku saabsan Qaar ka mid ah Farsamooyinka

• BERT ee Fahamka Luuqadda
• A* Raadi Qorshaynta Waddada
• YOLOv8 ee Ogaanshaha Shayga-Waqtiga dhabta ah
• Xoojinta Barashada Robotics-ka