Mercor, the AI recruiting startup founded by three 21-year-old Thiel Fellows, has raised $100 million in a Series B round, the company confirmed to TechCrunch.

Mercor, an AI recruiting startup founded by 21-year-olds, raises $100M at $2B valuation

Nvidia’s ($NVDA) foray into investing in artificial intelligence startups seems to be showing yet more signs of success as one company it’s backed is about to targe...

Nvidia-Backed Robotics Startup Field AI in $2B Funding Drive - TipRanks.com

Nvidia’s (NVDA) foray into investing in artificial intelligence startups seems to be showing yet more signs of success as one company it’s backe...

Nvidia-Backed Robotics Startup Field AI in $2B Funding Drive | Markets Insider

Zeta, a provider of banking software to banks and fintech startups, has raised $50 million from a strategic investor at a $2 billion valuation. The new Zeta, a provider of banking software to banks and fintech startups, has raised $50 million from a strategic investor at a $2 billion valuation.

Zeta valued at $2B in new funding | TechCrunch

As Chief Marketing Officer of 2B Living, the phrase “sell me this pen” is part of Rishi’s daily morning mantra. After running a leasing company in Chicago for 5 years and then renovating or brokering dozens of Bay Area properties for another 5 years, Rishi comes to 2B Living perfectly ...

2B Living – Property Management. Reimagined.

2A Consulting: Offers brand strategy, content and design, and project management, with a focus on crafting the client&#x27;s story.

2A Consulting | Inc.com

Revolutionize your air quality monitoring with the 2B Technologies AQSync Air Quality Monitoring Station. ​

2B TECHNOLOGIES - Home

At 2B Studio our collaborative approach to design creates lasting relationships with clients and stronger solutions for the marketplace.

Design Driven Product Development | Grand Rapids, Michigan | 2B Studio

2A Armament is the original lightweight AR manufacturer. High Quality, lightweight AR-15, AR22LR, and AR-10 rifles.

2A Armament | Lightest AR15 and AR10 rifles

Table of Links Abstract and 1. Introduction Abstract and 1. Introduction 2 Related works 2 Related works 3 Methodology and 3.1 Causal language model as a classification model 3 Methodology and 3.1 Causal language model as a classification model 3.2 Functional token 3.2 Functional token 3.3 Dataset collection 3.3 Dataset collection 3.4 Model development and training 3.4 Model development and training 4 Experiments and 4.1 Android function calls 4 Experiments and 4.1 Android function calls 4.2 Extension to Vehicle, Yelp, and DoorDash function sets 4.2 Extension to Vehicle, Yelp, and DoorDash function sets 4.3 Full and partial training datasets and 4.4 Full training and LoRA training 4.3 Full and partial training datasets and 4.4 Full training and LoRA training 4.5 Parallel and nested function call and 4.6 Weighted loss function for special tokens 4.5 Parallel and nested function call and 4.6 Weighted loss function for special tokens 5 Discussion and future works and References 5 Discussion and future works and References Appendix Appendix A.1 Android function examples A.1 Android function examples A.2 Vehicle function examples A.2 Vehicle function examples 3.4 Model development and training We employ the Google Gemma-2B model as the pretrained model in our framework. Our approach incorporates two distinct training methodologies: full model training and LoRA model training. For full model training, we utilize an AdamW optimizer with a learning rate set at 5e-5, a warm-up step of 10, and a linear learning rate scheduler. The same optimizer and learning rate configuration are applied to LoRA training. We specify the LoRA rank as 16 and apply LoRA to the following modules: q_proj, k_proj, v_proj, o_proj, up_proj, down_proj. The LoRA alpha parameter is set to 32. For both training methods—full model and LoRA—we set the number of epochs to 3. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. available on arxiv Authors:
(1) Wei Chen, Stanford University, with equal contribution and a corresponding author {weichen6}@stanford.edu;
(2) Zhiyuan Li, Stanford University and a corresponding author {zhiyuan8}@stanford.edu. Authors: Authors: (1) Wei Chen, Stanford University, with equal contribution and a corresponding author {weichen6}@stanford.edu; (2) Zhiyuan Li, Stanford University and a corresponding author {zhiyuan8}@stanford.edu.

Developing Function Calling Models: Comparing Full Training and LoRA on Gemma-2B

Singapore, March 21, 2025 — Mira, the pioneer in decentralized AI verification technology, announces unprecedented growth with 2.5 million users and two billion tokens processed daily across its ecosystem applications. The milestone demonstrates growing market demand for AI that can operate autonomously without human oversight. Processing two billion tokens daily is equivalent to approximately half of Wikipedia's content, generating 7.9 million images, or processing over 2,100 hours of video content per day. "This growth confirms we're addressing the critical barrier to AI's transformative potential," said Karan Sirdesai, Co-founder and CEO of Mira. "Today's AI remains constrained by the need for human verification—we're removing that bottleneck to enable truly autonomous intelligence capable of operating independently in high-stakes scenarios." "This growth confirms we're addressing the critical barrier to AI's transformative potential," said Karan Sirdesai, Co-founder and CEO of Mira. "Today's AI remains constrained by the need for human verification—we're removing that bottleneck to enable truly autonomous intelligence capable of operating independently in high-stakes scenarios." Enabling the Shift from 'Baby AI' to Autonomous Intelligence Current AI systems excel at generating content and solving problems, but their probabilistic nature makes them fundamentally unreliable. This creates a paradox: the more powerful AI becomes, the more human hours are required to verify its output, severely limiting AI's potential. Mira is addressing this challenge through decentralized verification that enables AI systems to operate autonomously in consequential domains. By transforming the verification process from a human bottleneck to a scalable, trustless system, Mira is paving the way for AI's next evolutionary leap. Applications Demonstrating the Vision This vision is coming to life through several applications built on Mira's verification infrastructure: Klok — A multi-LLM chat application providing access to leading AI models including DeepSeek-R1, GPT-4o mini, and Llama 3.3 70B Instruct within a unified interface. Klok treats each AI model as a trustless independent node, with verification features set to roll out in coming weeks.


WikiSentry — An AI agent that autonomously fact-checks Wikipedia articles against verified sources, identifying hallucinations, biases, and misinformation—a task previously requiring extensive human oversight.


Astro — A guidance platform helping users navigate life's important decisions through personalized, AI-powered insights that leverage verified information for reliable advice.


Amor — A supportive AI companion offering conversation and emotional connection for those seeking meaningful interaction, with verification ensuring consistent and trustworthy responses.
Public Testnet Enables Developer Innovation Klok — A multi-LLM chat application providing access to leading AI models including DeepSeek-R1, GPT-4o mini, and Llama 3.3 70B Instruct within a unified interface. Klok treats each AI model as a trustless independent node, with verification features set to roll out in coming weeks. Klok — A multi-LLM chat application providing access to leading AI models including DeepSeek-R1, GPT-4o mini, and Llama 3.3 70B Instruct within a unified interface. Klok treats each AI model as a trustless independent node, with verification features set to roll out in coming weeks. WikiSentry — An AI agent that autonomously fact-checks Wikipedia articles against verified sources, identifying hallucinations, biases, and misinformation—a task previously requiring extensive human oversight. WikiSentry — An AI agent that autonomously fact-checks Wikipedia articles against verified sources, identifying hallucinations, biases, and misinformation—a task previously requiring extensive human oversight. Astro — A guidance platform helping users navigate life's important decisions through personalized, AI-powered insights that leverage verified information for reliable advice. Astro — A guidance platform helping users navigate life's important decisions through personalized, AI-powered insights that leverage verified information for reliable advice. Amor — A supportive AI companion offering conversation and emotional connection for those seeking meaningful interaction, with verification ensuring consistent and trustworthy responses.
Public Testnet Enables Developer Innovation Amor — A supportive AI companion offering conversation and emotional connection for those seeking meaningful interaction, with verification ensuring consistent and trustworthy responses. Public Testnet Enables Developer Innovation Mira has today launched its Public Testnet, enabling developers, enterprises, and users to explore its blockchain-based verification network at explorer.mira.network, where all AI inferences are recorded as transparent, verifiable events. Read more here. explorer.mira.network explorer.mira.network here here Strategic Partnerships Expanding the Ecosystem Mira has integrated with leading agent frameworks including Eliza, SendAI, Arc, ZerePy, and FereAI, while establishing partnerships with blockchain networks like Monad and Plume Network. Infrastructure collaborations with io.net, Aethir, Hyperbolic, Exabits, and Spheron, along with industry partnerships with KernelDAO and Delphi Digital, further strengthen Mira's ecosystem. Monad Monad Plume Network Plume Network Backed by $9 million in funding from Framework, Accel, Mechanism, and Bitkraft, with angel investors including Balaji Srinivasan and Sandeep Nailwal, Mira is positioned to transform how AI systems operate at scale. About Mira Mira is building the foundational verification layer that enables trustless AI systems through advanced consensus mechanisms. The network combines sophisticated claim transformation and distributed verification protocols to achieve reliable AI execution at scale. By solving the fundamental challenge of error rates in AI, Mira is establishing new standards for reliability—paving the way for autonomous intelligence that can operate without human oversight, unlocking AI's transformative potential across society. Follow Mira on X and join their Discord for real-time updates. X X Discord Discord For more information, visit mira.network. mira.network mira.network This press release is brought to you by EAK Wire, the leading Web3 PR Newswire via HackerNoon Business Blogging Program. Web3 PR Newswire Web3 PR Newswire

Mira Reaches Major Milestone: 2.5M Users and 2B Daily Tokens Processed

Mira дасягнула важнай вехі: 2,5 мільёна карыстальнікаў і 2 мільярды штодзённых апрацаваных токенаў

Mira alcanza un hito importante: 2,5 millones de usuarios y 2 mil millones de tokens procesados diariamente

Mira dosegla veliku prekretnicu: 2,5 milijuna korisnika i 2 milijarde obrađenih dnevnih tokena

Mira が大きなマイルストーンを達成: ユーザー数 250 万人、1 日あたり 20 億トークンの処理

Мира негизги этапка жетти: 2,5 миллион колдонуучу жана 2B күнүмдүк Токендер иштетилди

Mira Akomi na Milestone ya munene: 2.5M ya ba usagers na 2B ya ba jetons ya mokolo na mokolo oyo esalemi

Mira sasniedz galveno pavērsienu: 2,5 miljoni lietotāju un 2 miljoni dienā apstrādāti marķieri

Mira atinge marco importante: 2,5 milhões de usuários e 2 bilhões de tokens diários processados

Mira Yafikia Hatua Kubwa: Watumiaji 2.5M na Tokeni 2B za Kila Siku Zimechakatwa

ሚራ ኣብ ዓቢ ምዕራፍ በጺሓ: 2.5M ተጠቀምቲን 2B መዓልታዊ ቶከናት ተሰሪሖም

Mira досягла важливого рубежу: 2,5 млн користувачів і 2 млрд щоденних токенів оброблено

UMira Ufinyelela Ingqophamlando Eyinhloko: Abasebenzisi abangu-2.5M kanye namathokheni ansuku zonke angu-2B Acutshunguliwe

Playing Simon Says with Gemma-2b and MediaPipe 

Gemma-2b এবং MediaPipe-এর সাথে সাইমন সেস খেলছে

Simon Says mit Gemma-2b und MediaPipe spielen

Jugando a Simon Says con Gemma-2b y MediaPipe

Jouer Simon Says avec Gemma-2b et MediaPipe

जेम्मा-2बी और मीडियापाइप के साथ साइमन सेज़ बजाना

Gemma-2b と MediaPipe を使用して Simon Says を再生する

Gemma-2b 및 MediaPipe를 사용하여 Simon Says 재생

Jogando Simon Says com Gemma-2b e MediaPipe

Игра в Simon Says с Gemma-2b и MediaPipe

Simon Says'ı Gemma-2b ve MediaPipe ile oynamak

Chơi Simon Says với Gemma-2b và MediaPipe

使用 Gemma-2b 和 MediaPipe 玩“西蒙说”游戏

By Robert Bell, CEO of 2B3D.com "Compassion" is not a word you see in a lot of business plans among top technology and financial firms. It may even be considered something of a liability for most businesses, but I don't see it that way. As the 2B3D team prepares for our upcoming Veterans Day NFT drop, we believe the compassionate aspect of our mission bears repeating: Compassion is our greatest strength, and underpins everything we do. was founded by me as well as my close friend, Andrew Bjarnsen, an Afghan War veteran and combat medic who himself struggles with PTSD. Andrew set out on this mission to help his fellow veterans through the use of virtual reality technology. 2B3D What we are creating at 2B3D (along with our partners at BiocoRx, Mason Labs, and Metabillionaire) is centered around a deep concern for the pain and plight of others. In fact, when we talk about the massive utility and gamification of our NFTs, it is purposely designed in response to the growing need for mental health treatment in a world that needs more empathy all-around- whether in daily 1-on-1 with our neighbors, business partners and fellow citizens, or when we’re thinking about others far removed from our own location and daily experiences. 2B3D is deep in the creation of a first-of-its-kind virtual reality medical environment (VRME) in the metaverse, architected specifically to to deliver free mental health services to military veterans in real-time- and largely to address just about everything that is inadequate about the current status quo for vets seeking these services. We’ll launch this in the first quarter of the new year. Our firm is owned and operated by military veterans, and we intend to head off the growing mental health crisis in America, starting with our at-risk military veterans. We are  deploying new treatments through virtual reality (VRx) that will radically improve the physical and mental health of veterans throughout the nation regardless of their location or distance from big cities. In the last several years, suicide rates have increased, and this burden of self-harm is disproportionately carried by veterans. and we feel something must be done. Veteran suicides rates are vastly outpacing those of the general US population It stands to reason, since resources are strained, the response is too slow. 2B3D’s mission is to build new technologies that can offer solutions. It is our passion. Director Christopher Jones of Public Health Service at the Center for Disease Control and Prevention testified in June that between 2001 to 2019, the rate of suicide In the general population rose nearly 30%, while among veterans the rate increased nearly 36%. What 2B3D is building is a realistic VR medical environment available online 24 hours a day through VR headsets, and in 2D through phones and desktop PCs. Veterans experiencing suicidal ideation (often exacerbated by PTSD symptoms and substance use disorder) will be able to access the 2B3D metaverse for immediate access to suicide prevention counselors, group therapy, meditation spaces, and / treatment. EMDR DBT Our realistic environments will not be just a ‘second-best’ option for suffering veterans, but a cutting-edge solution in an increasingly online society. Veterans who have seen combat or feel isolated are the most vulnerable to thoughts of suicide. 20-30% of veterans who commit suicide which affects divergent thinking, making it nearly impossible to break out of a mental spiral. At those times, seconds literally count. also screen positive for service-related injuries like Post Traumatic Stress Disorder Imagine it's 3 a.m. somewhere in the world where a veteran is suffering a mental health crisis forty-five minutes from the nearest emergency room. They may be considering suicide as an option to ending their pain, and they begin to spiral. Help through 2B3D will be available, always on standby. Rather than leave their home at 3 a.m. to get in a car or find public transportation in search of help, veterans will simply use their VR headset, phone, or computer to access our virtual medical environment equipped with mental health resources. In a safe, virtual space, they also have the option to talk with a live person. Even in the form of avatars, research has shown that this human connection can be enough to prevent a tragedy from occurring. Our funding will be as unique as our service. To that end, we've announced the launch of our Supporter Badge, an NFT collection of 3D PTSD support ribbons that will fund our virtual reality mental health solutions for military veterans. This collection will be available for purchase at this Veterans Day, Nov. 11, 2022. https://mint.2b3d.com By raising funds through the sale of our Supporter Badge NFT collection, we'll be able to provide military veterans with access to proven cognitive therapy modalities in safe and effective VR environments. The public sale price of each 3D ribbon will be $10 dollars. All proceeds go to helping us in our mission to help veterans, and 40% percent of the proceeds will be donated directly to veteran nonprofits and charities also fighting to save veteran lives. Those who are interested in helping veterans do not need experience with NFTs or cryptocurrency to purchase one of our Supporter Badges. They are available for purchase with a credit card on our website. There is no limit to the number of NFTs we can sell to help veterans. For those who love NFTs and helping veterans, the public release of an additional 2,880 unique Collector’s Pass NFTs will also be live on Nov. 11, 2022, with a floor price of .7 ETH each, or approximately $100 until sold out. Every Collector’s Pass is represented as a yet-to-be-revealed 3D object with unique properties like special graphics and rare traits, and future in-game applications. The number of available Collector’s Passes is set at exactly 2,880 - a number representing the 2,880 minutes in the span of two days in which, statistically, 44 veterans will take their own lives. Every minute counts. Join us. Launching Veteran’s Day at . https://mint.2b3d.com ### About Robert Bell Robert Bell is an entrepreneur, investor, and philanthropist who founded 2B3D Inc in 2020 as an all-in-one gaming studio, nonfungible token mint, auction house, and storage system. Bell’s mission was to push the bounds of virtual, augmented and mixed reality technologies as a way to help military veterans suffering from post-traumatic stress disorder. His mission to bring peace and healing to veterans is fueled by play-to-earn games, utility NFTs, and access to exclusive in-game cryptocurrency within a wider decentralized metaverse. Bell is also the founder and CEO of Bell Medical solutions. About 2B3D 2B3D is a military veteran-operated tech company specializing in web3 gaming, virtual reality, NFTs, and mental health solutions in the metaverse. 2B3D is currently developing a metaverse ecosystem using blockchain technology with play-to-earn games, an NFT marketplace and VR showrooms, and VR medical environments for therapy and meeting with healthcare professionals in real-time. The 2880 Genesis Collection NFT will drop on the eleventh day of the eleventh month, in honor of military veterans on Veterans Day. The 2B3D metaverse includes several core projects: VRx | Virtual healthcare with NFT prescriptions and real life professionals. NFTy150 | NFT marketplace, minting option and showroom. Topher's Inferno | Connecting enthusiastic gamers with ambitious developers. RestXP | B2B meeting rooms with a resting crypto reward. So Many Gods | A Play-to Earn, space-themed sci fi looter shooter. Sci-fi To learn more about 2B3D, visit or follow us on Twitter at @2B3Dinc. www.2B3D.com

The Ultimate Utility: 2B3D is Using NFTs to Save Military Veteran Lives

Blake 2B and Blake3 are slightly different algorithms. A while ago, I came up with the question of whether or not Siacoin should migrate to Blake3. This question was posted on . Reddit This question has intrigued me, but what intrigued me more or less was the answer. It would need to be a hardfork, and although Blake3 is considered multiple times faster by others, there are debates in regards to HAIFA construction vs. traditional Merkle trees and so forth. That said, a hardfork of Siacoin would likely be easy given the code is standardized, but according to devs on Reddit, a Blake3 upgrade is the wrong thing to be worried about. I do agree. Is Blake a good choice? However, I still had a curiosity in exploring Blake3. In one of my own projects, I have decided to let the founder integrate Blake3 as its core hashing mechanism. That project is (2nd version), and I think Blake3 would work as the hashing algo of choice for BitBadges. The whitepaper can be seen .  It is worth a read. BitBadges here Another fact I found interesting, is that besides Siacoin, another Blake2b adaptor was . Nano Nano utilizes Blake2b in regards to the digested seed hashing. In regards to Blake3 adaption, Nano and most of these other cryptos would likely need a hard fork over soft fork. Interestingly enough about Blake3, is there have been many adaptations. This includes a PHP Hash Blake3 , and Blake3 itself has a Rust and C implementation. class Blake3 is said to be a merkle tree on the inside, and the speed itself is substantially higher then Blake2b in regards to the they gave. benchmark test Conclusion Blake3 is utilized more towards CPU multithreading, and likely if a cryptocurrency wants the benefit of hashing, but aren't too focused on multithreading or parallelization, then Blake2b is still a decent choice. For me, as a person liking grid and verifiable forms of computing, I had the preference of Blake3. That said, it is nice that it was already available at the time starting development.

2B or to B3? That Is The Question... Blake 2B vs 3 for Siacoin

Over the past 6 months, we benchmarked multiple tracking technologies for mobile, unpowered assets: construction gear, medical rentals, tools, shipping containers, and more. The goal: build a system that reliably tells you: Where is this item now?
When was it last seen?
What’s the cost per asset to do this at scale? Where is this item now? Where is this item now? When was it last seen? When was it last seen? What’s the cost per asset to do this at scale? What’s the cost per asset to do this at scale? We tested: Cellular GPS (LandAirSea, Samsara)
Passive/Active RFID (Zebra, Alien)
BLE Trackers (Tile, One-Key, etc.)
Apple AirTags + programmatic layer Cellular GPS (LandAirSea, Samsara) Passive/Active RFID (Zebra, Alien) BLE Trackers (Tile, One-Key, etc.) Apple AirTags + programmatic layer Only one approach gave us: High-fidelity passive tracking
Minimal labor
Predictable location updates in the wild
Sub-$4/month total cost High-fidelity passive tracking Minimal labor Predictable location updates in the wild Sub-$4/month total cost 🔴 GPS Trackers — Power-Hungry & Operationally Fragile lessCopyEdit[ Asset ]
 |
[ GPS Module ]
 |
[ LTE Modem ]---[ Carrier Network ]---[ Cloud Platform ]
 |
[ Battery ] lessCopyEdit[ Asset ]
 |
[ GPS Module ]
 |
[ LTE Modem ]---[ Carrier Network ]---[ Cloud Platform ]
 |
[ Battery ] What it’s good at: What it’s good at: Real-time tracking when you have cell signal
Fine-grained updates (every 10s–60s) Real-time tracking when you have cell signal Fine-grained updates (every 10s–60s) Failure modes in the field: Failure modes in the field: Battery depletion → dead device in 7–30 days
Indoor/urban canyons → GPS lock fails
SIM provisioning → ~3% failure rate during setup
Techs forget to charge or miswire installs Battery depletion → dead device in 7–30 days Indoor/urban canyons → GPS lock fails SIM provisioning → ~3% failure rate during setup Techs forget to charge or miswire installs Measured in field: Measured in field: Avg install time (non-hardwired): 18 min
Avg labor revisit per year: 5.7x
Location latency indoors: >1hr or failed ping Avg install time (non-hardwired): 18 min 18 min Avg labor revisit per year: 5.7x 5.7x Location latency indoors: >1hr or failed ping >1hr or failed ping GPS works for powered fleets. It breaks down hard for passive assets. GPS works for powered fleets. It breaks down hard for passive assets. Annual TCO per device: ~$400 \ ($150 hardware + $240 data + $50 labor) 🟠 RFID — Great for Warehouses, Not the Field lessCopyEdit[ RFID Tag ]---(Passive)---[ RFID Reader ]
 |
 [ Backend DB ] lessCopyEdit[ RFID Tag ]---(Passive)---[ RFID Reader ]
 |
 [ Backend DB ] Useful for: Useful for: Inventory control at fixed chokepoints
Bulk scanning in highly structured environments Inventory control at fixed chokepoints Bulk scanning in highly structured environments Limitations: Limitations: No real tracking — just “seen at gate X”
Requires infrastructure at every decision point
0 coverage outdoors, in transit, on job sites
Readers are expensive and fragile (EM interference prone) No real tracking — just “seen at gate X” Requires infrastructure at every decision point 0 coverage outdoors, in transit, on job sites Readers are expensive and fragile (EM interference prone) Cost Breakdown: Cost Breakdown: Passive tags: $0.12–$0.80


Reader infrastructure/site: $2,500–$10,000


Integration overhead: high Passive tags: $0.12–$0.80 Passive tags: $0.12–$0.80 Reader infrastructure/site: $2,500–$10,000 Reader infrastructure/site: $2,500–$10,000 Integration overhead: high Integration overhead: high Unless you control the entire environment, RFID creates visibility gaps you can't close. Unless you control the entire environment, RFID creates visibility gaps you can't close. Annual TCO (real deployments): $8k–$40k/site Annual TCO (real deployments): $8k–$40k/site 🟡 BLE Trackers — “Tracking” That Usually Isn’t cssCopyEdit[ BLE Beacon ] --x--> [ Needs Someone Running App ] cssCopyEdit[ BLE Beacon ] --x--> [ Needs Someone Running App ] Examples tested: Examples tested: Tile Pro
Milwaukee One-Key
Chipolo Business Beta Tile Pro Milwaukee One-Key Chipolo Business Beta Core issue: They rely on proprietary apps running in the background — no real mesh, just marketing. Core issue: no real mesh Mesh Density (urban tests): Mesh Density (urban tests): Tile: 2 updates/week/asset in SF
One-Key: 0 updates outside Milwaukee job sites
Battery life: ~12 months (replaceable CR2032) Tile: 2 updates/week/asset in SF One-Key: 0 updates outside Milwaukee job sites Battery life: ~12 months (replaceable CR2032) Key limitation: Key limitation: If no one’s running the app, the device is invisible. In practice, that means ~99%+ dead zones. If no one’s running the app, the device is invisible. In practice, that means ~99%+ dead zones. No way to manage at scale. No geofences. No asset dashboard. Just a few stale pings, if you're lucky. TCO: ~ $30–$60/year (still too high for something that barely works) TCO: ~ $30–$60/year (still too high for something that barely works) 🟢 AirTags — When Paired with the Right Stack, They Win lessCopyEdit[ AirTag ]---(BLE)---[ Any iPhone Nearby ]
 |
 [ Apple Find My Mesh ]
 |
 [ Programmatic Layer ]
 |
 [ AirPinpoint Dashboard ] lessCopyEdit[ AirTag ]---(BLE)---[ Any iPhone Nearby ]
 |
 [ Apple Find My Mesh ]
 |
 [ Programmatic Layer ]
 |
 [ AirPinpoint Dashboard ] Key technical advantages: Key technical advantages: BLE beaconing interval: ~800ms (adaptive)
U1 chip + BLE triangulation improves spatial accuracy
Global passive mesh: 2.2B+ iOS devices relay encrypted updates
Battery: 2–4 years on CR2032
Range: We observed ~0.2–2.3 miles real-world handoff distance in cities BLE beaconing interval: ~800ms (adaptive) U1 chip + BLE triangulation improves spatial accuracy Global passive mesh: 2.2B+ iOS devices relay encrypted updates 2.2B+ iOS devices Battery: 2–4 years on CR2032 Range: We observed ~0.2–2.3 miles real-world handoff distance in cities Test results: Test results: Urban ping latency: <8 minutes
Suburban latency: ~12–45 minutes
Dense indoor location accuracy: ~10–15m radius
Deployment time: <60 seconds per asset
No SIM, no charge cycles, no field config Urban ping latency: <8 minutes <8 minutes Suburban latency: ~12–45 minutes ~12–45 minutes Dense indoor location accuracy: ~10–15m radius Deployment time: <60 seconds per asset <60 seconds No SIM, no charge cycles, no field config This is the only BLE-based system with actual nationwide mesh reliability. This is the only BLE-based system with actual nationwide mesh reliability. AirTag Comprehensive CR2032 Battery Brand Comparison Based on extensive empirical testing of CR2032 batteries under conditions similar to AirTag operation, here's how different brands perform: Brand

Runtime (Hours)

Runtime (Approx. Months)*

Cost per Cell

Value Rating (Hours/$)



Panasonic

1500

12.5

$1.04

1442



Voniko

1448

12.1

$0.83

1745



Maxell

1444

12.0

$0.58

2490



Toshiba

1325

11.0

$0.64

2070



Harbor Freight

1307

10.9

$0.75

1743



Energizer

1307

10.9

$1.28

1021



Duracell

1298

10.8

$1.81

717



AC Delco

1223

10.2

$0.79

1548



Nightkonic

1186

9.9

$0.24

4942



Murata

1175

9.8

$0.45

2611



Hua Dao

642

5.4

$0.14

4586 Brand

Runtime (Hours)

Runtime (Approx. Months)*

Cost per Cell

Value Rating (Hours/$)



Panasonic

1500

12.5

$1.04

1442



Voniko

1448

12.1

$0.83

1745



Maxell

1444

12.0

$0.58

2490



Toshiba

1325

11.0

$0.64

2070



Harbor Freight

1307

10.9

$0.75

1743



Energizer

1307

10.9

$1.28

1021



Duracell

1298

10.8

$1.81

717



AC Delco

1223

10.2

$0.79

1548



Nightkonic

1186

9.9

$0.24

4942



Murata

1175

9.8

$0.45

2611



Hua Dao

642

5.4

$0.14

4586 Brand

Runtime (Hours)

Runtime (Approx. Months)*

Cost per Cell

Value Rating (Hours/$) Brand Brand Runtime (Hours) Runtime (Hours) Runtime (Approx. Months)* Runtime (Approx. Months)* Cost per Cell Cost per Cell Value Rating (Hours/$) Value Rating (Hours/$) Panasonic

1500

12.5

$1.04

1442 Panasonic Panasonic 1500 1500 12.5 12.5 $1.04 $1.04 1442 1442 Voniko

1448

12.1

$0.83

1745 Voniko Voniko 1448 1448 12.1 12.1 $0.83 $0.83 1745 1745 Maxell

1444

12.0

$0.58

2490 Maxell Maxell 1444 1444 12.0 12.0 $0.58 $0.58 2490 2490 Toshiba

1325

11.0

$0.64

2070 Toshiba Toshiba 1325 1325 11.0 11.0 $0.64 $0.64 2070 2070 Harbor Freight

1307

10.9

$0.75

1743 Harbor Freight Harbor Freight 1307 1307 10.9 10.9 $0.75 $0.75 1743 1743 Energizer

1307

10.9

$1.28

1021 Energizer Energizer 1307 1307 10.9 10.9 $1.28 $1.28 1021 1021 Duracell

1298

10.8

$1.81

717 Duracell Duracell 1298 1298 10.8 10.8 $1.81 $1.81 717 717 AC Delco

1223

10.2

$0.79

1548 AC Delco AC Delco 1223 1223 10.2 10.2 $0.79 $0.79 1548 1548 Nightkonic

1186

9.9

$0.24

4942 Nightkonic Nightkonic 1186 1186 9.9 9.9 $0.24 $0.24 4942 4942 Murata

1175

9.8

$0.45

2611 Murata Murata 1175 1175 9.8 9.8 $0.45 $0.45 2611 2611 Hua Dao

642

5.4

$0.14

4586 Hua Dao Hua Dao 642 642 5.4 5.4 $0.14 $0.14 4586 4586 *Converted from accelerated test conditions to approximate real-world months based on typical AirTag usage patterns Converted from accelerated test conditions to approximate real-world months based on typical AirTag usage patterns Test Methodology: All batteries were tested with a consistent 80μA base load (approximately 16x the typical AirTag sleep current) with periodic higher current pulse loads similar to Bluetooth transmissions. Tests were normalized to account for minor variations in test fixtures. Test Methodology 🧮 Real Cost Comparison (Per 100 Assets, Per Year) Tech

Hardware

Labor

Subscriptions

TCO (Year 1)



GPS

$15,000

$5,000

$36,000

$56,000



RFID

$2,000

$4,000

$0

$6,000+



BLE

$3,000

$3,000

$0

$6,000



AirTag + Platform

$2,000

$500

$47.88 (flat)

~$2,548 Tech

Hardware

Labor

Subscriptions

TCO (Year 1)



GPS

$15,000

$5,000

$36,000

$56,000



RFID

$2,000

$4,000

$0

$6,000+



BLE

$3,000

$3,000

$0

$6,000



AirTag + Platform

$2,000

$500

$47.88 (flat)

~$2,548 Tech

Hardware

Labor

Subscriptions

TCO (Year 1) Tech Tech Hardware Hardware Labor Labor Subscriptions Subscriptions TCO (Year 1) TCO (Year 1) GPS

$15,000

$5,000

$36,000

$56,000 GPS GPS $15,000 $15,000 $5,000 $5,000 $36,000 $36,000 $56,000 $56,000 $56,000 RFID

$2,000

$4,000

$0

$6,000+ RFID RFID $2,000 $2,000 $4,000 $4,000 $0 $0 $6,000+ $6,000+ $6,000+ BLE

$3,000

$3,000

$0

$6,000 BLE BLE $3,000 $3,000 $3,000 $3,000 $0 $0 $6,000 $6,000 $6,000 AirTag + Platform

$2,000

$500

$47.88 (flat)

~$2,548 AirTag + Platform AirTag + Platform AirTag + Platform $2,000 $2,000 $2,000 $500 $500 $500 $47.88 (flat) $47.88 (flat) $47.88 (flat) ~$2,548 ~$2,548 ~$2,548 ✅ Final Notes AirTags on an ios device alone are not sufficient for business use. They lack: Asset naming / grouping


Geofencing


Bulk management


Alerting / integrations


Silent mode (you don’t want customers getting “you’re being tracked” alerts) Asset naming / grouping Asset naming / grouping Geofencing Geofencing Bulk management Bulk management Alerting / integrations Alerting / integrations Silent mode (you don’t want customers getting “you’re being tracked” alerts) Silent mode (you don’t want customers getting “you’re being tracked” alerts) But when paired with a 3rd party platform that solves those (like AirPinpoint) they become arguably the highest-leverage, lowest-friction tracking layer available for unpowered asset fleets. highest-leverage, lowest-friction tracking layer available If you’re trying to track equipment, rentals, forklifts, pumps, dollies, carts, containers, or anything that moves but doesn’t have power — this is the stack to beat. this is the stack to beat

We Tested GPS, RFID, BLE, and AirTags for Asset Tracking — Only One Actually Worked

Table of Links Abstract and 1. Introduction Abstract and 1. Introduction 2 Related works 2 Related works 3 Methodology and 3.1 Causal language model as a classification model 3 Methodology and 3.1 Causal language model as a classification model 3.2 Functional token 3.2 Functional token 3.3 Dataset collection 3.3 Dataset collection 3.4 Model development and training 3.4 Model development and training 4 Experiments and 4.1 Android function calls 4 Experiments and 4.1 Android function calls 4.2 Extension to Vehicle, Yelp, and DoorDash function sets 4.2 Extension to Vehicle, Yelp, and DoorDash function sets 4.3 Full and partial training datasets and 4.4 Full training and LoRA training 4.3 Full and partial training datasets and 4.4 Full training and LoRA training 4.5 Parallel and nested function call and 4.6 Weighted loss function for special tokens 4.5 Parallel and nested function call and 4.6 Weighted loss function for special tokens 5 Discussion and future works and References 5 Discussion and future works and References Appendix Appendix A.1 Android function examples A.1 Android function examples A.2 Vehicle function examples A.2 Vehicle function examples 4 Experiments Our study conducts a comprehensive evaluation of language model capabilities via an extensive benchmarking approach, aimed at assessing their effectiveness in generating accurate function calls. Initially, we compare our model’s accuracy and response time against premier models in the field, namely GPT-4 (checkpoint: gpt-4-0125-preview) and GPT-3.5 (checkpoint: gpt-3.5-turbo-0125). In the next phase, we explore the efficacy of the RAG technique, renowned for its ability to reduce incorrect outputs (hallucinations) and latency by equipping language models with a concise selection of potential functions. Through the integration of Meta’s FAISS for semantic search, we enhance the function call description retrieval process, opting for the top 5 descriptions to navigate context length constraints seen in models like Meta’s Llama-7B and OpenAI’s GPT-3.5. Subsequently, we analyze the impact of training dataset size and model training methods on performance metrics. 4.1 Android function calls To illustrate our model’s application, we select Android system function calls as a case study, focusing on accuracy and latency in function call generation. Initially, we chose 20 Android APIs as our dataset foundation. We adopted two distinct methods for generating function call commands. For details on API design, see Appendix. The first method involves a RAG approach to identify top similar function descriptions based on user queries, which the language model then uses, along with the user query, to generate the expected function call commands. We detail the various models employed during this evaluation. Utilizing Google Gemini, we sample relevant queries for the selected Android function calls and manually label the ground truth as the evaluation dataset. We document our benchmark results, focusing on two critical metrics: accuracy and latency, as illustrated in Figure (4) and Figure (5) respectively. Llama-7B RAG Evaluation Initially, the pretrained Llama-7B model showed limited ability in generating the expected outcomes, leading us to employ a Llama-7B variant fine-tuned for function call generation [48]. For the Llama-7B assessment, we applied the RAG method without strict output format requirements, considering responses with missing parentheses as correct. This evaluation was conducted on a single NVIDIA A100 machine, with all results from Llama-7B compared against the ground truth. The primary errors were incorrect function name selection and erroneous parameter generation. Despite employing few-shot learning to guide the model towards accurate function generation, the performance was modest, with an accuracy rate of 68.095% when overlooking format requirements and a latency of 13.46 seconds, excluding model loading time. To improve latency, we implemented optimizations such as flash attention and a fast tokenizer. Llama-7B RAG Evaluation GPT-3.5 RAG Evaluation Similar to the approach with Llama-7B, we utilized GPT-3.5 for response generation, employing the same semantic search strategy for context acquisition. To enhance GPT-3.5’s performance, we designed a specific prompt style, incorporating one-shot learning to improve accuracy further. GPT-3.5 RAG Evaluation In this benchmark test, an impressive accuracy of 98.095% was achieved, leveraging the gpt-3.5-turbo0125 checkpoint known for its optimization in function calling tasks. The latency was significantly improved to 1.97 seconds for generating a single function call, a notable enhancement over the Llama-7B model’s performance. This improvement in speed is primarily attributed to the efficiency of the language model inference, as the RAG component remained consistent. GPT-3.5’s quicker response may be due to OpenAI’s use of multiple GPUs or a more advanced inference infrastructure. Further analysis revealed that a significant portion of the time was spent on content retrieval, despite only needing to fetch 5 function descriptions from a pool of 20. To optimize latency, all function descriptions’ embeddings were precomputed using OpenAI’s text-embedding-3-large endpoint, with IndexFlatL2 employed for search indexing and parallel computation on multicore CPUs to enhance speed. GPT-3.5 and GPT-4 Evaluations In efforts to further reduce latency for GPT-3.5 and GPT-4, we included all 20 function descriptions directly in the context, bypassing the RAG method to avoid microservices interactions and their associated IO-bound overheads. This adjustment yielded a latency reduction to 1.18 seconds for GPT-3.5. The prompt template mirrored that of the GPT-3.5 RAG, with the addition of more candidate functions. However, accuracy slightly declined to 97.143%, possibly due to diminished language model effectiveness with longer text inputs. Conversely, GPT-4 exhibited superior accuracy at 98.571% and even lower latency than GPT-3.5, despite expectations of GPT-4 being a larger model. This performance, evaluated on March 18 at 2 PM PDT, might reflect variances in API traffic or hardware configurations between the two models. GPT-4’s enhanced performance suggests OpenAI could be allocating more GPU resources to it or that it experiences less demand compared to GPT-3.5. GPT-3.5 and GPT-4 Evaluations Octopus model Now, we present the octopus model with 1000 data points sampled for each API. And we observe a 99.524% accuracy in our evaluation dataset. Moreover, the prompt used for this method is as simple as: Octopus model In our approach, incorporating function information directly into the context is unnecessary, as the Octopus model has already learned to mapping functional tokens to corresponding function descriptions, thereby conserving a significant number of tokens for processing. Given its compact size and the brevity of the context required, the Octopus model demonstrates a reduced latency of 0.38 seconds. To maintain an equitable comparison, we adhered to the same benchmark settings used for the Llama7B evaluation, such as incorporating flash attention and not using quantization. Furthermore, we explored the deployment of our Octopus 2B model on mobile devices through quantization. By precomputing the state for the fixed prefix—"Below is the query from the users, please call the correct function and generate the parameters to call the function. Query:"—our on-device model achieves remarkable performance, completing a function call within 1.1 to 1.7 seconds for typical queries of 20 to 30 tokens using a standard Android phone. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. available on arxiv Authors:
(1) Wei Chen, Stanford University, with equal contribution and a corresponding author {weichen6}@stanford.edu;
(2) Zhiyuan Li, Stanford University and a corresponding author {zhiyuan8}@stanford.edu. Authors: Authors: (1) Wei Chen, Stanford University, with equal contribution and a corresponding author {weichen6}@stanford.edu; (2) Zhiyuan Li, Stanford University and a corresponding author {zhiyuan8}@stanford.edu.

Benchmarking Large Language Models for Function Calling: GPT-4, GPT-3.5, Llama, and Octopus

Authors:
(1) Clemencia Siro, University of Amsterdam, Amsterdam, The Netherlands;
(2) Mohammad Aliannejadi, University of Amsterdam, Amsterdam, The Netherlands;
(3) Maarten de Rijke, University of Amsterdam, Amsterdam, The Netherlands. Authors: Authors: (1) Clemencia Siro, University of Amsterdam, Amsterdam, The Netherlands; (2) Mohammad Aliannejadi, University of Amsterdam, Amsterdam, The Netherlands; (3) Maarten de Rijke, University of Amsterdam, Amsterdam, The Netherlands. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Methodology and 2.1 Experimental data and tasks 2 Methodology and 2.1 Experimental data and tasks 2.2 Automatic generation of diverse dialogue contexts 2.2 Automatic generation of diverse dialogue contexts 2.3 Crowdsource experiments 2.3 Crowdsource experiments 2.4 Experimental conditions 2.4 Experimental conditions 2.5 Participants 2.5 Participants 3 Results and Analysis and 3.1 Data statistics 3 Results and Analysis and 3.1 Data statistics 3.2 RQ1: Effect of varying amount of dialogue context 3.2 RQ1: Effect of varying amount of dialogue context 3.3 RQ2: Effect of automatically generated dialogue context 3.3 RQ2: Effect of automatically generated dialogue context 4 Discussion and Implications 4 Discussion and Implications 5 Related Work 5 Related Work 6 Conclusion, Limitations, and Ethical Considerations 6 Conclusion, Limitations, and Ethical Considerations 7 Acknowledgements and References 7 Acknowledgements and References A. Appendix A. Appendix 3 Results and Analysis We address (RQ1) and (RQ2) by providing an overview of the results and in-depth analysis of our crowdsourcing experiments. We first describe the key data statistics. RQ1 RQ2 3.1 Data statistics Phase 1. Figure 1 presents the distributions of relevance and usefulness ratings across the three variations, C0, C3, and C7. Figure 1a indicates a larger number of dialogues rated as relevant when annotators had no prior context (C0), compared to instances of C3 and C7, where a lower number of dialogues received such ratings. This suggests that in the absence of prior context, annotators are more inclined to perceive the system’s response as relevant, as they lack evidence to assert otherwise. This trend is particularly prevalent when user utterances lean towards casual conversations, such as inquiring about a previously mentioned movie or requesting a similar recommendation to their initial query, aspects to which the annotators have no access. Consequently, this suggests that annotators rely on assumptions regarding the user’s previous inquiries, leading to higher ratings for system response relevance. We observe a similar trend for usefulness (Figure 1b), compared to C3 and C7, C0 has more dialogues rated as useful. The introduction of the user’s next utterance introduced some level of ambiguity to annotators. Evident in instances where the user introduced a new item not mentioned in the system’s response and expressed an intention to watch it, the usefulness of the system’s response became uncertain. This ambiguity arises particularly when annotators lack access to prior context, making it challenging to tell if the movie was mentioned before in the preceding context. These observations highlight the impact of the amount of dialogue context on the annotators’ perceptions of relevance and usefulness in Phase 1. This emphasizes the significance of taking contextual factors into account when evaluating TDSs. Phase 2. In Phase 2, we present findings on how different types of dialogue contexts influence the annotation of relevance and usefulness labels. When the dialogue summary is included as supplementary information for the turn under evaluation (C0-sum), a higher proportion of dialogues are annotated as relevant compared to C0-llm for relevance (60% vs. 52.5%, respectively); see Figure 2a. Phase 2. In contrast to the observations made for relevance, we see in Figure 2b that a higher percentage of dialogues are predominantly labeled as not useful when additional information is provided to the annotators. This accounts for 60% in C0-heu, 47.5% in C0-llm, and 45% in C0-sum. This trend is consistent with our observations from Phase 1, highlighting that while system responses may be relevant, they do not always align with the user’s actual information need. We find that C0-sum exhibits the highest number of dialogues rated as useful, indicating its effectiveness in providing pertinent information to aid annotators in making informed judgments regarding usefulness. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Study Finds AI Responses Rated Higher When Context is Limited

Alibaba's Claude Killer Enters the Ring

Authors:
(1) Frank Palma Gomez, from Boston University and the work done by him and Ramon during their internship in Google Research and Google DeepMind respectively;
(2) Ramon Sanabria, The University of Edinburgh and the work done by him and Frank during their internship in Google Research and Google DeepMind respectively;
(3) Yun-hsuan Sung, Google Research;
(4) Daniel Cer, Google Research;
(5) Siddharth Dalmia, Google DeepMind and Equal Advising Contributions;
(6) Gustavo Hernandez Abrego, Google Research and Equal Advising Contributions. Authors: Authors: (1) Frank Palma Gomez, from Boston University and the work done by him and Ramon during their internship in Google Research and Google DeepMind respectively; (2) Ramon Sanabria, The University of Edinburgh and the work done by him and Frank during their internship in Google Research and Google DeepMind respectively; (3) Yun-hsuan Sung, Google Research; (4) Daniel Cer, Google Research; (5) Siddharth Dalmia, Google DeepMind and Equal Advising Contributions; (6) Gustavo Hernandez Abrego, Google Research and Equal Advising Contributions. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Method 2 Method 3 Data and Tasks 3 Data and Tasks 4 Model 4 Model 5 Experiments 5 Experiments 6 Related Work 6 Related Work 7 Conclusion 7 Conclusion 8 Acknowledgements and References 8 Acknowledgements and References A Appendix A Appendix Abstract Large language models (LLMs) are trained on text-only data that go far beyond the languages with paired speech and text data. At the same time, Dual Encoder (DE) based retrieval systems project queries and documents into the same embedding space and have demonstrated their success in retrieval and bi-text mining. To match speech and text in many languages, we propose using LLMs to initialize multi-modal DE retrieval systems. Unlike traditional methods, our system doesn’t require speech data during LLM pre-training and can exploit LLM’s multilingual text understanding capabilities to match speech and text in languages unseen during retrieval training. Our multi-modal LLMbased retrieval system is capable of matching speech and text in 102 languages despite only training on 21 languages. Our system outperforms previous systems trained explicitly on all 102 languages. We achieve a 10% absolute improvement in Recall@1 averaged across these languages. Additionally, our model demonstrates cross-lingual speech and text matching, which is further enhanced by readily available machine translation data. 1 Introduction LLMs have demonstrated their effectiveness in modelling textual sequences to tackle various downstream tasks (Brown et al., 2020; Hoffmann et al., 2022; Chowdhery et al., 2023). This effectiveness has led to the development of powerful LLMs capable of modelling text in a wide range of languages. The abundance of textual data in different languages across the internet has fueled the progress of multi-lingual models (Johnson et al., 2017; Xue et al., 2020; Siddhant et al., 2022). On the other hand, speech technologies are prevalent in smartphones and personal assistants, but their language availability is relatively limited compared to the languages that LLMs support (Baevski et al., 2020; Radford et al., 2023). Various efforts have explored solutions to the speech-text data scarcity problem (Duquenne et al., 2021; Ardila et al., 2019; Wang et al., 2020). Works such as SpeechMatrix (Duquenne et al., 2022) use separate speech and text encoders to mine semantically similar utterances that are neighbors in an embedding space. However, these approaches are limiting because they require speech and text encoders that have aligned representation spaces. We posit that we can retrieve speech and text utterances by aligning both modalities within the embedding space built from a single pre-trained LLM. We take inspiration from previous works that use pre-trained LLMs to perform automatic speech recognition (ASR) and automatic speech translation (AST) (Rubenstein et al., 2023; Wang et al., 2023; Hassid et al., 2023). Our intuition is that we can perform the speech and text alignment leveraging the capabilities of text-only LLMs without requiring two separate models. In this paper, we propose converting LLMs into speech and text DE retrieval systems without requiring speech pre-training and outperform previous methods with significantly less data. By discretizing speech into acoustic units (Hsu et al., 2021), we extend our LLMs embedding layer and treat the acoustic units as ordinary text tokens. Consequently, we transform our LLM into a retrieval system via a contrastive loss allowing us to match speech and text utterances in various languages. Our contributions are the following: We build a speech-to-text symmetric DE from a pre-trained LLM. We show that our retrieval system is effective matching speech and text in 102 languages of FLEURS (Conneau et al., 2023 despite only training on 21 languages.


We show that our model exhibits cross-lingual speech and text matching without training on this type of data. At the same time, we find that cross-lingual speech and text matching is furthered improved by training on readily available machine translation data. We build a speech-to-text symmetric DE from a pre-trained LLM. We show that our retrieval system is effective matching speech and text in 102 languages of FLEURS (Conneau et al., 2023 despite only training on 21 languages. We build a speech-to-text symmetric DE from a pre-trained LLM. We show that our retrieval system is effective matching speech and text in 102 languages of FLEURS (Conneau et al., 2023 despite only training on 21 languages. We show that our model exhibits cross-lingual speech and text matching without training on this type of data. At the same time, we find that cross-lingual speech and text matching is furthered improved by training on readily available machine translation data. We show that our model exhibits cross-lingual speech and text matching without training on this type of data. At the same time, we find that cross-lingual speech and text matching is furthered improved by training on readily available machine translation data. 2 Method We train a transformer-based DE model that encodes speech and text given a dataset D = {(xi , yi)}, where xi is a speech utterance and yi is its transcription. We denote the speech and text embeddings as xi = E(xi) and yi = E(yi), respectively, where E is a transformer-based DE that encodes speech and text. 2.1 Generating Audio Tokens We convert raw speech into discrete tokens using the process in Lakhotia et al. (2021); Borsos et al. (2023). The process converts a speech query xi into an embedding using a pre-trained speech encoder. The output embedding is then discretized into a set of tokens using k-means clustering. We refer to the resulting tokens as audio tokens. We use the 2B variant of the Universal Speech Model (USM) encoder (Zhang et al., 2023) as the speech encoder and take the middle layer as the embedding for xi . Additionally, we generate audio tokens at 25Hz using k-means clustering, resulting in a set of 1024 possible audio tokens. We will refer to this as our audio token vocabulary. 2.2 Supporting Text and Audio Tokens To support text and audio tokens in our LLM, we follow the formulation of Rubenstein et al. (2023). We extend the embedding layer of a transformer decoder by a tokens, where a represents the size of our audio token vocabulary. This modification leads to an embedding layer with size (t + a) × m, where t is the number of tokens in the text vocabulary and m is the dimensions of the embedding vectors. In our implementation, the first t tokens represent text and the remaining a tokens are reserved for audio. We initialize the embeddings layer from scratch when training our model. 3 Data and Tasks Appendix A.3 details our training and evaluation datasets along with the number of languages in each dataset, the split we used, and the size of each dataset. We focus on the following retrieval tasks: Speech-to-Text Retrieval (S2T) involves retrieving the corresponding transcription from a database given a speech sample. In S2T, we train on CoVoST-2 (Wang et al., 2021) speech utterances and their transcriptions. CoVoST-2 is a large multilingual speech corpus derived from Wikipedia expanding over 21 languages and provides translation to and from English. We use FLEURS (Conneau et al., 2023) to evaluate S2T performance on 102 languages. FLEURS is an n-way parallel dataset containing speech utterances from FLoRES-101 (Goyal et al., 2021) human translations. To evaluate S2T, we report recall at 1 (R@1) rates for retrieving the correct transcription for every speech sample and word error rate (WER). Speech-to-Text Retrieval (S2T) Speech-to-Text Translation Retrieval (S2TT) attempts to retrieve the corresponding text translation of a speech sample. We use S2TT to measure the cross-lingual capabilities of our multi-modal DE retrieval system. We evaluate this capability zero-shot on X → En S2TT data of FLUERS and explore if we can further improve this capability by training on readily-available machine translation data from WikiMatrix (Schwenk et al., 2019). We pick French, German, Dutch, and Polish to English that are common across WikiMatrix and FLEURS and further discuss the amount of machine translation data used in Appendix A.3. For S2TT, we report 4-gram corpusBLEU (Post, 2018). Speech-to-Text Translation Retrieval (S2TT) 4 Model Figure 1 shows an illustration of our model. We initialize our dual encoder from PaLM 2 XXS (Google et al., 2023) and append a linear projection layer after pooling the outputs along the sequence length dimension. The embedding and linear projection layers are initialized randomly. After initializing our model from PaLM 2, we use a contrastive loss (Hadsell et al., 2006). Appendix A.1 includes more details on our training setup. We will refer to our proposed model as PaLM 2 DE. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Stories, News & Mentions about 2B

Categories

Trending Topics

Stories, News & Mentions about 2B

2b’s Stories on HackerNoon

2b's News & Mentions

Categories

Trending Topics