Shares of Ecolab Inc. slumped 0.05% to $202.89 Thursday, on what proved to be an all-around great trading session for the stock market, with the S&P 500...

Ecolab Inc. stock outperforms competitors despite losses on the day - MarketWatch

Ecolab to Buy 1000 Ford Electric Vehicles

Ecolab Inc (ECL) is around the top of the Basic Materials sector according to InvestorsObserver. ECL rec

Can Ecolab Inc (ECL) Stock Rise to the Top of Basic Materials Sector Wednesday?

Key Insights Institutions' substantial holdings in Ecolab implies that they have significant influence over the...

With 77% ownership, Ecolab Inc. (NYSE:ECL) boasts of strong institutional backing - Simply Wall St News

Ecolab, Inc. (NYSE: ECL) has seen a rise in its stock price by 5.40 in relation to its previous close of 159.15. However, the company has experienced a 4.67% gain in its stock price over the last five trading sessions.

Ecolab, Inc. (ECL) Shares Rise Despite Market Challenges

Greetings. Welcome to the Ecolab Third Quarter 2023 Earnings Release Conference Call. (Operator Instructions) As a reminder, this conference is being recorded. It is now my pleasure to introduce your host, Andrew Hedberg, Vice President, Investor Relations for Ecolab. Thank you, Mr. Hedberg. You may begin.

Q3 2023 Ecolab Inc Earnings Call

Continuing its momentum from earlier this year, Ecolab reported stronger than expected third-quarter earnings and is looking confidently at the remainder of the year and into 2024. While many economists and other CEOs remain concerned about persistently ...

Ecolab's strong third quarter boosts end-of-year confidence

SAINT PAUL, Minn. (AP) — SAINT PAUL, Minn. (AP) — Ecolab Inc. (ECL) on Tuesday reported third-quarter profit of $404 million. The Saint Paul, Minnesota-based company said it had profit of $1.41 per share. Earnings, adjusted for non-recurring costs and pretax expenses, were $1.54 per share.

Ecolab: Q3 Earnings Snapshot

Ecolab (ECL) came out with quarterly earnings of $1.54 per share, beating the Zacks Consensus Estimate of $1.52 per share. This compares to earnings of $1.30 per share a year ago. These figures are adjusted for non-recurring items.

Ecolab (ECL) Q3 Earnings Beat Estimates

Ecolab's profit rose in the third quarter as sales were pushed higher by double-digit growth in its institutional and specialty business and its pest elimination division.

Ecolab 3Q Profit Jumps on Higher Sales

Ecolab

ecolab

I found myself formless, waking
up near the Asset Tree with this
predicting platform and really
wanted to do something broad
and correctively positive with it. It all clinged on quality, a síngle
leak in asset performance lured a
loop of leverage. When team-tool alerts demonstrated dripping operational upsets, the portal pause changed to cloud-based triggers. A chart-change of digital trends subtly raised, one stacked with smart alarms; so sustained as sensors' analytics. A chart like no shapeless other, that floated on three hundred pretreatment areas, then flowed through sixty mechanical grades mitigating the most minimal manual e-coats end-to-end in circuits. The current model of efficiency
effectively hinges on preventative
plans, panned still secured lines
as networks— one phosphate problem
embedded in e-vehicle bodies via overuse. One boost moment words would distill
through optimized solutions written
ripe in the area's best practices practicing real-time identification. So enormous those checklists were
at e-points, first in data's field-train
refining rigour, next to layered levels
of fuel waters' intelligence data; laid
plain and visible enterprise-wide. The last reacts for CIP; when driven through tracked trouble spots by laboratory data. Actionable parameters might very well verify that pace of completion; in certain certainty, combining controller data with customer KPIs. Integrity's transmission trend thinks outside of the automated box, then consolidates prescribed limits in the non-linear long term. The monitoring methodology stands
maintained in-scope granular efficacy
only when necessary.
Put into smart inventories it overcomes inadequacy; put into post-treatment anomalies it sets remote IIoT benchmarks for the needs of several incorporated industries ranging in sustainable impact. Had it not been for the smarter asset's
performance having highly correlated
foresight to sink the project's
cutting-edge, this dynamic insight
wouldn't have prioritized advanced
algorithms that enable detected
peace of mind.

ECOLAB3D: A Poem

331 Stories To Learn About Iot

Let's learn about via these 31 free stories. They are ordered by most time reading created on HackerNoon. Visit the to find the most read stories about any technology. Autonomous Cars /Learn Repo 1. What A Driverless Future Means for Auto Insurance: An Interview with Rory Angold Rory Angold, United Car Care EVP, Shares What A Driverless Future Means for Auto Insurance 2. Robotic Process Automation + Machine Learning = Intelligent Automation Robotic Process Automation has generated a lot of buzz across many different industries. As businesses focus on digital innovation, automation of repetitive tasks to increase efficiency while decreasing human errors is an attractive proposition. 3. Autonomous Driving Lidar Perception Stack with PCL: An Algorithmic Implementation C++ pipeline for LiDAR-based autonomous driving. 4. Electric Cars: What Does the Future Hold? Better Batteries and Wireless Charging Electric Cars: What Does the Future Hold? 5. How Self-Driving Cars Will Change the Auto Insurance Industry Welcome to the dawn of the self-driving car era, and a subsequent emerging challenge: how to insure them. 6. How Will The World End? Top 4 Cyber Apocalypse Scenarios Take a look at a few of the cyber security scenarios which could lead to the end of the world, in true science fiction cinematic fashion. 7. Designing an HMI in Automotive: A Stress-Free and Intuitive Way In 2020, sophisticated-car infotainment system designs are no longer impressive. Smartphone integration, voice assistance, recognition of driver distraction, assisted personalized driving, connected in-car payments can be easily taken for granted. With the rapid growth of automotive technologies, OEMs are being challenged to create HMIs that offer usability and ensure safety while standing out from the competition. Still, there are general rules that can be helpful to those interested in designing an HMI for connected cars and who are approaching this topic for the first time. 8. How Swarm Intelligence-Powered Vehicles and Smart Cities can Help Streamline Traffic Given the advent of autonomous cars and their growing penetration in the transportation market, it is empirical that companies consider using Swarm Intelligence to tackle traffic and make the daily commute a lot more pleasant for billions around the world. 9. Digital Transformation in Automotive Industry Automotive industry is getting accustomed to digital era. Brands like Mercedes Benz, which had been manufacturing cars since 19th century, have to reinvent the automotive technology again due to digitalization. Yes, we are about to witness autonomous cars powered by complex connectivity algorithms and sensors. 10. The Future Self-Driving Cars and Their Societal Impact What societal changes might we see from the advent of self-driving cars? Self-driving cars may one day eliminate traffic laws, DUIs, and major accidents 11. Uber CEO Interview, and the Advent of Self Driving Car Technology 12. Why WFH Could Spell The End of Motoring as We Know it Covid-19 has thrown us into an era of great uncertainty. Every industry has been impacted by the devastating pandemic in some way or another, and there’s no sign of a return to normal life in the pipeline any time soon. 13. ECOLAB3D: A Poem I found myself formless, waking up near the Asset Tree with this predicting platform 14. Tesla to Build $5 Billion Gigafactory in Mexico With the Gigafactory agreement inked, Musk stands to benefit from a series of advantages a Mexican factory could provide. 15. Driving Change: 5 Automotive Trends and Innovations During the Coronavirus pandemic, people’s attitudes towards public transportation and carsharing have changed. They are not safe enough. Precisely because of that, most users are now finding comfort in owning a private vehicle. 16. SpaceX Technology May Have The Potential To Revolutionise The Future of Cars As the years have gone by, entrepreneur Elon Musk has undoubtedly become a household name. His broader visions, and not to mention wealth, has helped him to carve out a legendary mark in outer space and the automotive industry alike. 17. Machine Learning News Roundup - 6 Essential AI Articles of 2019 18. When Will Self-Driving Cars Become Safe Enough? UK's government is still preparing for the widespread arrival of self-driving cars, but how safe are self-driving vehicles right now? 19. Why Hasn't Tesla Entered the Indian Market? The internet has been buzzing since Elon Musk announced that Tesla is coming to India next year. This is big news and not just for the people who are waiting to get their hands on a luxury electric car like Tesla but also for the entire electric vehicles (EV) industry in the country. 20. How to Annotate 100 LIDAR Scans in 1 Hour: A Step-by-Step Guide A guide to efficient annotation of 3D point clouds for use in self-driving vehicles. Fast semantic segmentation of the drivable areag 21. Insights From Waymo: Self-Driving Taxis Are The Future of Ride-Hailing In the on-demand market, ride-hailing is one of the key services that account for a higher percentage of total revenue. Uber is the first ride-hailing app that entered the market and following that many taxi services launched their business in the market. Many taxi apps became successful and are still sustaining in the market. There is so much competition in the taxi service industry. Many small scale taxi services are competing with each other to become the key players in the market. 22. Cars With Smartphone Integration Soon Will Be Mainstream In today’s fast-paced world we’re constantly online, whether we’re at home, in the office, or behind the wheel. Connectivity and mobility rule our lives, and smartphones have become an integral part of our daily routine. But while connectivity and mobility bring tons of possibilities, they also bring new problems. One of them is safely managing our communication. 23. Car Safety Technology Is A Must-Have To Save Pedestrians' Lives When we talk about pedestrian traffic, safety must be the top concern. But sadly, transportation infrastructure worldwide pays more attention to the vehicle's safety, not people. 24. The Future of Automotive Tech: Top Trends in 2023 Here’s a look at five of the biggest technology trends that will shape the automotive market in 2023. 25. 5 Technologies Used to Prevent Car Accidents Back-up cameras, LED headlights, voice manipulatation, your hands-off and even voice-up devices are all examples of accident-preventing technology in cars. 26. How AI from Driver Technologies Company Helps To Protect the Motorists Across the world, more than 1.3 million people die in car accidents, and over 50 million people are seriously injured every year. That’s nearly 4,000 people each day. Drivers in developing nations are most at risk. Only 54% of the world’s motor vehicles are in developing countries, but 90% of the world’s fatal car accidents occur in those countries. Even within the wealthiest countries vehicle-related injury and death are directly correlated to personal and neighborhood incomes. 27. Automotive Companies only Access 5% of their Vehicle Data Only 5% of autonomous driving sensor data is used for product development today. Better data infrastructure holds the keys to progress. 28. RANSAC, OLS, PCA: 3 Ways to Draw a Straight Line Across a Set of Points How I approached solving an interview task for autonomous driving from 3 different perspectives: RANSAC, PCA, and Ordinary Least Squares (OLS). 29. An Intro to Robotaxi and Tesla’s Foray into Autonomous Taxis Elon Musk promised Tesla robotaxis to be launched in 2020. No Robotaxis are yet to be seen while competitors like Hyundai, Waymo, and Ford make progress. 30. 4 Overlooked Challenges to Our Shiny Self-Driving Future According to the United States Environmental Protection Agency, transportation alone contributes 29 percent of US greenhouse gas emissions and according to the World Health Organization, approximately 1.35 million people die each year as a result of road traffic crashes, but self-driving cars are coming to fix this, with multi-billion dollar tech and motor companies like Google, Uber, Ford, and General Motors betting on the future of transportation to be one with electric, autonomous and shared vehicles. 31. Why the Rise of Active Assistance and Safety Systems Must Not Distract Drivers While the introduction of assistance and safety technologies intends to reduce vehicle accidents, improper use endangers the driver and other road users. Thank you for checking out the 31 most read stories about Autonomous Cars on HackerNoon. to find the most read stories about any technology. Visit the /Learn Repo

31 Stories To Learn About Autonomous Cars

One in five Americans their health with the help of fitness trackers or apps. During the first two quarters of 2020, there were over 1.2 billion of health and fitness apps. These facts indicate that people are interested in platforms that will allow them to track vital signs, control nutrition, and remotely communicate with doctors. Before starting a healthcare software project, a medical facility should decide what type of medical platform its target audience needs. Let’s take a look at the most popular apps and options for their use. controls downloads According to a by Statista, health apps are most frequently used in China (65%), India (63%), and Indonesia (57%). The Japanese (12%) and Europeans (22-40%) turn to mHealth less often. study People often use applications for a balanced diet. Platforms for monitoring heart rate, tracking vital signs, and meditation are also popular. The healthcare software market is not limited to this. There are at least 10 types of healthcare software that the medical industry can use. Health monitoring apps Healthcare providers are investing in remote monitoring apps. Insider Intelligence predicts that the global RPM systems market will be worth over $1.7 billion by 2027. Investing in such software is not only popular but also economically justified. Applications and devices for remote patient monitoring collect, process, and transmit information about users’ state of health. Here are some examples of such platforms: Hand hygiene monitoring software Such applications have become especially relevant during the pandemic. Although hand hygiene is a common practice for healthcare professionals, there have been introduced systems that control proper and timely disinfection. For example, Ecolab records every time medical workers wash their hands. It notifies users when the next disinfection is required. Thus, the system protects patients from possible infection. Ecolab includes three components: Patient directory — A user sees how often a nurse or a doctor washes hands. Disinfection statistics — The application records each hand washing and sends reminders to the medical staff. Reporting function — The chief physician collects information about employees to ensure they follow the WHO requirements and do not put patients at risk. Application to Fight Depression According to the WHO, about 280 million people in the world from depression. The actual figure might be higher if the unreported cases are counted. Many people experience negative emotions and do not realize that they are seriously ill. suffer Therefore, Stanford scientists are people suffering from depression. They want to create an application to monitor the state of patients before and after the treatment. They plan to build a device that continuously measures physiological parameters (cortisol levels in sweat, changes in heart rate, and so on) to assess a patient’s state of mind. IoT sensors also make it possible to predict, prevent and treat mental illnesses. trying to help Blood glucose monitoring platform It is a glucometer connected to a smartphone and an application for collecting data. The application reminds patients when they need to take a test, assesses blood sugar levels, and gives recommendations on nutrition and physical activity. Such measures allow a patient to control diabetes and live comfortably, despite the restrictions. The application can be used by a doctor who monitors the sick person or relatives taking care of them. Asthma monitoring system Such a system includes the following components: IoT devices with sensors collect information about respiratory rate, pulse, cough, and rales. A mobile application that collects and visualizes data, transfers it to a specialist, and reminds the patient about a visit to the doctor. A web application for storing medical history and statistics. Thanks to this device, the system predicts an approaching asthma attack. The patient manages to take medicine on time, and the doctor can evaluate the effectiveness of the prescribed drugs. Surgery Software Surgery actively uses software and robots to plan and carry out operations. In 2018, 5,000 robots helped doctors to more than a million procedures in orthopedics, surgery, neurology, otolaryngology, and other areas. perform Specially designed equipment and software help surgeons manage instruments and perform more complex operations on limited areas of the body without the risk to patients. Doctor Appointment Apps When a patient makes an appointment with a doctor and doesn’t show up for the consultation, the medical institution loses money. Forbes that due to such incidents American healthcare loses about $150 billion annually. states Doctor appointment apps help to partially solve this problem. The system helps to plan visits and reminds patients about appointments by email or via push notifications. The registration form is not confirmed until the patient fills in the required fields (full name, contact details, complaints). The information helps administrators to determine how long the visit will last. Doctors don’t need to fill out the documentation from scratch, they use the patient data recorded in the system, adding the necessary details. CRM for Healthcare 30% of patients clinics due to poor service. It is important for people that medical workers understand their needs and provide personalized care. are willing to change One could manage this issue with the help of customer relationship management software. Such an app improves the clinic’s communication with patients and the work of the organization. It helps to store medical histories and visit statistics, inform about new medical services, and analyze the work of the staff. If a clinic lacks such a tool, it is worth investing in healthcare software development and ordering such an application. Hospital Management Software While CRP optimizes communication with customers, hospital management software is responsible for the internal processes of a clinic. The system gathers information about the employees in an integrated interface to automate administrative operations not related to medical procedures. With the help of such a program, the organization performs the following tasks: It plans the medical equipment maintenance; Makes an inventory; Keeps track of deliveries; Manages hospital beds; Registers patients; Pays utility bills and performs other operations. Such software served well when hospitals were overcrowded with patients and people lacked ventilators and oxygen masks. The program helped to track the equipment and uninterruptedly transfer it to hospital wards for critical patients. Healthcare Analytics Platform Between 2013 and 2020, the global healthcare data volume 15 times to 2,134 exabytes. Healthcare is a leader in Big Data generation. Medical software, laboratories, human resources, and CRM produce loads of information. It must be skillfully analyzed to be used for the benefit of a medical institution, employees, and patients. increased Medical analytics systems process incoming information and single out valuable data which is used to: Set accurate diagnoses; Predict the treatment results; To manage the inventory and equipment; To monitor the state of equipment and for other purposes. The healthcare analytics market is predicted to $40 billion by 2025. reach Billing System It a week or a month for an average organization to manually process an invoice. For the healthcare sector, such delays are undesirable. The billing system automates the issuance and payment of electronic invoices. It takes less time and money, and fewer medical workers to carry out this procedure. takes This system complies with the Health Insurance Portability and Accountability Act (HIPAA) and easily integrates with accounting software. Some applications offer online payments, self-service, prepayment, and consulting. If a clinic has a poor bill payment system, it is worth starting healthcare software development with a project of a new efficient billing system. Software for Training and Retraining Medical Personnel Physicians need to constantly learn and improve their skills, and keep track of the latest advances in medicine, technological inventions, and novelties. Therefore, applications providing new knowledge enter the market: Software for e-learning. These are applications that collect the latest news from the healthcare industry and interactive libraries with a new knowledge base. The most popular are Medscape Mobile, Human Anatomy Atlas, and Brainscape. Software for practice. Such platforms contain VR/AR elements that help doctors to hone their skills in practice. For example, they can train for complex surgeries or practice invasive procedures. The hospital team can remotely improve their skills. Electronic Medical Records (EMR) An EMR is a program that digitally stores information about a patient. It includes a medical history, prescribed medications, a list of vaccinations, information about chronic diseases, and so on. Patients entrust their health data to the clinic, and doctors remotely monitor their health status. A doctor makes accurate and clear medical records, so their colleagues, pharmacists, and other employees do not have to read illegible handwriting. The fewer misunderstandings an EMR causes, the fewer medical errors occur. Advanced EMRs with IoT sensors and medical applications so that doctors can remotely monitor patients. integrate Telemedicine Apps Telemedicine is a rapidly developing healthcare sector. It gained immense popularity when the pandemic provoked a leap in the remote treatment of patients. In the US, online appointments 41% in 2020. The telemedicine market will from $49.9 billion in 2019 to $459.8 billion in 2030. A remote care application is a great idea for a healthcare software development project. constituted grow Telemedicine applications connect patients and doctors via chats, instant messengers, and video calls. A patient can consult a physician using their home computer, tablet, or smartphone. They can get an electronic prescription or a referral for lab tests. Why Clinics Should Invest in Healthcare Software Development The demand for healthcare apps is growing as they increase the competitive edge of hospitals and clinics. They provide: Instant and twenty-four-hour access to patient information. Access to medical records from any device, from any corner of the world. Easier exchange of medical data between hospitals, laboratories, and insurance companies. Personalized patient care. Optimization of internal operations. Doctors see more patients because they do not need to manually perform certain tasks. Simplified appointment making and remote consultations. Better coordination of clinic workflows, because the necessary information is stored in one place, and tasks are automated. Thanks to healthcare software, organizations save time, money, avoid medical errors and satisfy customers with the quality of services. Every modern healthcare institution strives to achieve such a result. Conclusion The medical applications mentioned above solve different problems, have different architecture and functions. But they serve a common goal which is to improve and modernize the healthcare industry. Thus, it will carry out its mission even better. When planning healthcare software development, you should define the purpose of the project and select the appropriate application type. Focus on those operations that need to be improved first. If the development process is set up correctly, the result will be rewarding for your healthcare facility.

As Healthcare App Popularity Rises, Organizations Need to Focus on New Software

A medida que aumenta la popularidad de las aplicaciones para el cuidado de la salud, las organizaciones deben centrarse en un nuevo software

Alors que la popularité des applications de santé augmente, les organisations doivent se concentrer sur de nouveaux logiciels

जैसे-जैसे हेल्थकेयर ऐप की लोकप्रियता बढ़ती है, संगठनों को नए सॉफ्टवेयर पर ध्यान देने की जरूरत है

ヘルスケア アプリの人気が高まる中、組織は新しいソフトウェアに注力する必要があります

À medida que a popularidade dos aplicativos de assistência médica aumenta, as organizações precisam se concentrar em novos softwares

Khi mức độ phổ biến của ứng dụng chăm sóc sức khỏe tăng lên, các tổ chức cần tập trung vào phần mềm mới

随着医疗保健应用程序受欢迎程度的提高，组织需要专注于新软件

I intend this article as a thought-provoking piece on a topic that has intrigued me for quite some time. In early 2024, the BitNet b1.58 paper was published, and I wrote a short article about it. The core idea behind this architecture is that highly efficient large language models (LLMs) can be built using only three types of weights (-1, 0, 1). (The name comes from the fact that log2(3) = 1.58.) This is particularly exciting because such a network does not require multiplications—only conditional additions. Architectures like these can run efficiently without GPUs or can run on ASICs, which enable much more efficient execution compared to GPUs. The only problem with this solution is that it cannot be trained using gradient descent. Training still relies on floating-point numbers, and the weights are quantized afterward for inference. This made me wonder for the first time: could there be a point where using gradient-free methods instead of gradient descent might make training more efficient? But even if we momentarily set aside these extreme quantized networks, there is still a noticeable trend: the gap between the networks used for training and those used for inference is growing. Training a large language model (LLM) starts at several million dollars. Currently, DeepSeek v3 holds the "budget-friendly" record at $6 million. This model requires massive GPU clusters both for training and operation. However, through distillation and quantization, significantly smaller models can be derived from these large ones. These smaller models perform almost as well but can run locally. Recently, Exolabs showcased a demo where they ran an LLM on an old Windows 98 computer. It seems LLMs are becoming the new Doom—developers are now competing to see who can run the most efficient LLM on the lowest-performance devices. https://x.com/exolabs/status/1873103218596811059?embedable=true Of course, Exolabs' LLM is nowhere near the level of DeepSeek, but the trend is clear. In training, the number of parameters is continuously increasing, ever-larger clusters are being used, and ever-greater sums of money are being burned to "breed" increasingly advanced LLMs. From these highly developed LLMs, smaller models are then derived through distillation and quantization, offering comparable performance to the large models but with significantly lower computational requirements. The gap between training and inference continues to grow steadily. This logically raises the question: wouldn’t it be possible to directly produce the small model, instead of training a large one at a cost of millions of dollars and then reducing it? The root of the problem lies in gradient descent, which does not work efficiently on these smaller models. Gradient descent is the method without which today’s deep neural networks would not exist. Essentially, it’s a heuristic that helps determine how to adjust a network's parameters so that its behavior aligns more closely with what we expect. I won’t go into too much detail here, as I’ve written a full article on the topic for those interested in a deeper dive. Gradient descent is so fundamental that machine learning libraries (e.g., Jax) differ from general mathematical libraries (e.g., NumPy) primarily in their ability to compute derivatives for various operations—a critical requirement for gradient descent. Gradient descent is the cornerstone of these systems. There are, however, gradient-free solutions as well, such as genetic algorithms or even a simple random search. While these are less efficient than gradient descent, they have their advantages. The most obvious advantage is that they can be used in cases where derivatives cannot be computed. This includes, for example, the 1.58-bit neural networks mentioned before. Without gradient descent, finding the optimal weights for a network becomes significantly more challenging, and gradient-free methods can take 100x or even 1000x more steps. However, if individual steps are much faster, training can still end up being more efficient overall. Imagine if we repurposed the transistors in a GPU cluster and, instead of building GPUs, created ASICs specifically designed to efficiently run 1.58-bit networks. While a single training step for a large network runs on a GPU cluster, 10,000 training steps could be executed simultaneously on an ASIC cluster with a much smaller, reduced network. Even if training requires 1,000x more steps using a gradient-free method, the process could still be 10x faster overall. Additionally, such a system would likely scale better and consume less energy. In addition, these 1.58-bit networks could open up an entirely new world of possibilities. For instance, instead of ASICs, we could imagine highly specialized analog or even light-based circuits that are 100x or even 1,000x more efficient than ASICs. A gradient-free method might take 1,000x more steps to find the optimal weights, but if each step can be executed 10,000x faster, the training process would still be 10x faster overall and likely more energy-efficient as well. Another very exciting area is decentralization. For the reasons mentioned earlier, training large neural networks is increasingly becoming the privilege of major corporations, as it requires massive GPU clusters and millions of dollars. However, the reduced small models derived from these networks can run efficiently locally on desktops or even mobile devices. The latest processors often come equipped with dedicated units specifically designed for running neural networks, making this more feasible. If we use gradient-free methods for training, any device capable of running the neural network can be involved in the training process. In gradient-free methods like evolutionary algorithms, the task is straightforward: after randomly altering a few parameters, simply run the network. After several attempts, the better-performing parameter sets are kept, the poorer ones are discarded, and the process continues. Periodically, the best candidates can be exchanged between devices, allowing massive clusters to be built from something as simple as mobile phones. It is possible to build a system similar to Bitcoin, where participants "mine" neural networks. In Bitcoin, mining means that participants’ computers make random attempts to find a specific hash. For this purpose, specialized ASICs are available that are designed solely to generate hashes very quickly. The first participant to find the desired hash earns the reward. A decentralized system for "mining" intelligence would look very similar, but instead of calculating hashes, the ASIC would run a neural network extremely quickly. GPUs could also be used, but for something like 1.58-bit networks, an ASIC might already be more efficient. Miners would make random attempts to find more effective parameters, and those who succeeded could submit their results. If the network is indeed more efficient, they earn a reward, and a new round begins with the goal of finding an even better-performing network. This seems like a far more meaningful way to utilize energy compared to Bitcoin mining. It’s hard to say how effective gradient-free solutions could be. It’s also possible that the gradient-free direction might simply be a dead end. However, I am certain that the answer is far from trivial. This field deserves deeper research, as the potential rewards could be significant. If we could genuinely train models more efficiently this way, it could lead to the development of new and better LLMs. It’s also possible we might find that training a network from scratch using gradient-free methods is not effective, but even in that case, there could be potential for gradient-free fine-tuning. Additionally, the decentralization of training—and thus the democratization of AI—are promises compelling enough to make this area worth exploring further. I started a discussion in a Reddit post. If you're interested in the topic and would like to join the conversation, you can do so here. I’d greatly appreciate any opinions or comments! I intend this article as a thought-provoking piece on a topic that has intrigued me for quite some time. In early 2024, the BitNet b1.58 paper was published, and I wrote a short article about it . The core idea behind this architecture is that highly efficient large language models (LLMs) can be built using only three types of weights (-1, 0, 1). (The name comes from the fact that log2(3) = 1.58.) This is particularly exciting because such a network does not require multiplications—only conditional additions. Architectures like these can run efficiently without GPUs or can run on ASICs, which enable much more efficient execution compared to GPUs. a short article about it The only problem with this solution is that it cannot be trained using gradient descent. Training still relies on floating-point numbers, and the weights are quantized afterward for inference. This made me wonder for the first time: could there be a point where using gradient-free methods instead of gradient descent might make training more efficient? But even if we momentarily set aside these extreme quantized networks, there is still a noticeable trend: the gap between the networks used for training and those used for inference is growing. Training a large language model (LLM) starts at several million dollars. Currently, DeepSeek v3 holds the "budget-friendly" record at $6 million. This model requires massive GPU clusters both for training and operation. However, through distillation and quantization, significantly smaller models can be derived from these large ones. These smaller models perform almost as well but can run locally. Recently, Exolabs showcased a demo where they ran an LLM on an old Windows 98 computer. It seems LLMs are becoming the new Doom—developers are now competing to see who can run the most efficient LLM on the lowest-performance devices. https://x.com/exolabs/status/1873103218596811059?embedable=true https://x.com/exolabs/status/1873103218596811059?embedable=true Of course, Exolabs' LLM is nowhere near the level of DeepSeek, but the trend is clear. In training, the number of parameters is continuously increasing, ever-larger clusters are being used, and ever-greater sums of money are being burned to "breed" increasingly advanced LLMs. From these highly developed LLMs, smaller models are then derived through distillation and quantization, offering comparable performance to the large models but with significantly lower computational requirements. The gap between training and inference continues to grow steadily. The gap between training and inference continues to grow steadily. The gap between training and inference continues to grow steadily. This logically raises the question: wouldn’t it be possible to directly produce the small model, instead of training a large one at a cost of millions of dollars and then reducing it? The root of the problem lies in gradient descent, which does not work efficiently on these smaller models. wouldn’t it be possible to directly produce the small model, instead of training a large one at a cost of millions of dollars and then reducing it? Gradient descent is the method without which today’s deep neural networks would not exist. Essentially, it’s a heuristic that helps determine how to adjust a network's parameters so that its behavior aligns more closely with what we expect. I won’t go into too much detail here, as I’ve written a full article on the topic for those interested in a deeper dive. a full article on the topic Gradient descent is so fundamental that machine learning libraries (e.g., Jax) differ from general mathematical libraries (e.g., NumPy) primarily in their ability to compute derivatives for various operations—a critical requirement for gradient descent. Gradient descent is the cornerstone of these systems. There are, however, gradient-free solutions as well, such as genetic algorithms or even a simple random search. While these are less efficient than gradient descent, they have their advantages. The most obvious advantage is that they can be used in cases where derivatives cannot be computed. This includes, for example, the 1.58-bit neural networks mentioned before. Without gradient descent, finding the optimal weights for a network becomes significantly more challenging, and gradient-free methods can take 100x or even 1000x more steps. However, if individual steps are much faster, training can still end up being more efficient overall. Imagine if we repurposed the transistors in a GPU cluster and, instead of building GPUs, created ASICs specifically designed to efficiently run 1.58-bit networks. While a single training step for a large network runs on a GPU cluster, 10,000 training steps could be executed simultaneously on an ASIC cluster with a much smaller, reduced network. Even if training requires 1,000x more steps using a gradient-free method, the process could still be 10x faster overall. Additionally, such a system would likely scale better and consume less energy. In addition, these 1.58-bit networks could open up an entirely new world of possibilities. For instance, instead of ASICs, we could imagine highly specialized analog or even light-based circuits that are 100x or even 1,000x more efficient than ASICs. A gradient-free method might take 1,000x more steps to find the optimal weights, but if each step can be executed 10,000x faster, the training process would still be 10x faster overall and likely more energy-efficient as well. A gradient-free method might take 1,000x more steps to find the optimal weights, but if each step can be executed 10,000x faster, the training process would still be 10x faster overall and likely more energy-efficient as well. A gradient-free method might take 1,000x more steps to find the optimal weights, but if each step can be executed 10,000x faster, the training process would still be 10x faster overall and likely more energy-efficient as well. Another very exciting area is decentralization. For the reasons mentioned earlier, training large neural networks is increasingly becoming the privilege of major corporations, as it requires massive GPU clusters and millions of dollars. However, the reduced small models derived from these networks can run efficiently locally on desktops or even mobile devices. The latest processors often come equipped with dedicated units specifically designed for running neural networks, making this more feasible. If we use gradient-free methods for training, any device capable of running the neural network can be involved in the training process. If we use gradient-free methods for training, any device capable of running the neural network can be involved in the training process. If we use gradient-free methods for training, any device capable of running the neural network can be involved in the training process. In gradient-free methods like evolutionary algorithms, the task is straightforward: after randomly altering a few parameters, simply run the network. After several attempts, the better-performing parameter sets are kept, the poorer ones are discarded, and the process continues. Periodically, the best candidates can be exchanged between devices, allowing massive clusters to be built from something as simple as mobile phones. It is possible to build a system similar to Bitcoin, where participants "mine" neural networks. In Bitcoin, mining means that participants’ computers make random attempts to find a specific hash. For this purpose, specialized ASICs are available that are designed solely to generate hashes very quickly. The first participant to find the desired hash earns the reward. A decentralized system for "mining" intelligence would look very similar, but instead of calculating hashes, the ASIC would run a neural network extremely quickly. GPUs could also be used, but for something like 1.58-bit networks, an ASIC might already be more efficient. Miners would make random attempts to find more effective parameters, and those who succeeded could submit their results. If the network is indeed more efficient, they earn a reward, and a new round begins with the goal of finding an even better-performing network. This seems like a far more meaningful way to utilize energy compared to Bitcoin mining. It’s hard to say how effective gradient-free solutions could be. It’s also possible that the gradient-free direction might simply be a dead end. However, I am certain that the answer is far from trivial. This field deserves deeper research, as the potential rewards could be significant. If we could genuinely train models more efficiently this way, it could lead to the development of new and better LLMs. It’s also possible we might find that training a network from scratch using gradient-free methods is not effective, but even in that case, there could be potential for gradient-free fine-tuning. Additionally, the decentralization of training—and thus the democratization of AI—are promises compelling enough to make this area worth exploring further. I started a discussion in a Reddit post . If you're interested in the topic and would like to join the conversation, you can do so here. I’d greatly appreciate any opinions or comments! started a discussion in a Reddit post

How Gradient-Free Training Could Decentralize AI

Go Fast, Small, and Productive With GolangSpell

Google Colab and VS Code are the most popular editor tools used by many Python developers and Researchers to develop different tech solutions or systems (especially in Machine Learning & Data Science). Many Python developers and Data scientists know how to use Google Colab, but do you know that you can set up a VS Code on Google Colab and use it as an editor the same way as in your local machine? In this article, you will learn How to install the colabcode python package. How to start the VS Code (code server). How to access the online VS Code. Open the terminal. Run a python file. Steps on How to Use Google Colab with VS Code 1.Open Colab Notebook The first step is to launch a new colab notebook in your Google Colab and you can rename the file as you want. For example, run_vscode.ipynb . 2.Install colabcode python package To use Google Colab with VS Code (code server), you need to install the colabcode python package. This is an awesome open-source python package developed by . Abhishek Thakur To install the package, run the following command in your notebook cell. !pip install colabcode 3.Import ColabCode The next step is to import ColabCode class from the package. colabcode ColabCode from import 4.Create an instance of ColabCode After importing the ColabCode, you need to create an instance of ColabCode and set the following arguments. - The port you want to run code-server on. For example port=10000 port - You can set a password to protect your code-server from unauthorized access. This is an optional argument. password - If you want to use your Google drive. This is a Boolean argument means you can set True or False. This is an optional argument. mount_drive ColabCode(port= ) 10000 5.Start the Code Server After running the ColabCode instance, it will start the server and show the link to access the code server. You need to click the link and it will open in a new tab. Now you can take advantage of a full-fledged code editor and run different experiments on the Colab VM. If you check on your Colab Notebook, you will observe the cell that runs the ColabCode instance is continuously running. Don't close your Colab notebook unless you want to close the code server that runs VS Code. Note: Tips to use VS Code on Google Colab After launching the code server, the following tips will help you start using VS Code on Google Colab. 1.Open Terminal To open the terminal on VS Code that runs on Google Colab, use the following shortcut command Ctrl + Shift + ` 2.Change Theme You can change the theme of the editor by clicking the setting icon (bottom-left corner) and then click "Color Theme". It will open a popup window with different theme options you can select. 3.Run Python File You can create a python file by clicking the "File" section on the sidebar and then select a "New File" tab. In the following example, you will see how to run a simple python file that trains a machine-learning algorithm to classify iris flowers into three species (setosa, versicolor, or virginica) and then make a prediction. Final Thoughts on How to use Google Colab with VS Code Congratulations 👏👏, you have made it to the end of this article! I hope you have learned something new. You can set up VS Code on Google Colab and take your coding to the next level. You can also use the colabcode python package on the Kaggle platform to run VS Code. You just need to follow the same steps mentioned above. If you learned something new or enjoyed reading this article, please share it so that others can see it. Until then, see you in the next post! You can also find me on Twitter . @Davis_McDavid And you can read more articles like this . here Want to keep up to date with all the latest in python? in the footer below! Subscribe to our newsletter

How To use Google Colab with VS Code

Cómo usar Google Colab con VS Code

Open sourced by Google Research team, achieved wide popularity amongst NLP enthusiasts for all the right reasons! It is one of the best Natural Language Processing pre-trained models with superior NLP capabilities. It can be used for language classification, question & answering, next word prediction, tokenization, etc. pre-trained models of BERT Our case study Question Answering System in Python using BERT NLP and BERT based Question and Answering system demo , developed in Python + Flask, got hugely popular garnering hundreds of visitors per day. We got a lot of appreciative and lauding emails praising our QnA demo. Along with that, we also got number of people asking about how we created this QnA demo. And till the day, we keep getting requests on how to develop such a QnA system using BERT pre-trained model open-sourced by Google. [1] [2] To start with, the readme file on the official GitHub repository of BERT provides a good amount of information about how to fine-tune the model on SQuAD 2.0 but we could see that developers are still facing issues. So, we decided to publish a step-by-step tutorial to fine-tune the BERT pre-trained model and generate inference of answers from the given paragraph and questions on using TPU. Colab In this tutorial, we are not going to cover how to create web-based interface using Python + Flask. We’ll just cover the fine-tuning and inference on Colab using TPU. You can create your own interface using Flask or Django. Overview In this tutorial we will see how to perform a fine-tuning task on SQuAD using Google Colab, for that we will use BERT GitHub Repository, BERT Repository includes: TensorFlow code for the BERT model architecture. Pre-trained models for both the lowercase and cased version of BERT-Base and BERT-Large. You can also refer or copy to follow the steps. our colab file Steps to perform BERT Fine-tuning on Google Colab 1) Change Runtime to TPU On the main menu, click on and select . Set “ “ as the hardware accelerator. Below screeenshot will help you understand how you can change the runtime to TPU. Runtime Change runtime type TPU After Clicking on “Change runtime type”, Select from the dropdown option as given in the below figure. TPU 2) Clone the BERT github repository , or idirectional mbedding epresentations from ransformers, is a new method of pre-training language representations which obtains state-of-the-art results on a wide array of Natural Language Processing (NLP) tasks. You can find the academic paper here: . BERT B E R T https://arxiv.org/abs/1810.04805 BERT has two stages: and . Pre-training fine-tuning is fairly expensive (four days on 4 to 16 Cloud TPUs), but is a one-time procedure. BERT has released a number of pre-trained models. Most NLP researchers will never need to pre-train their own model from scratch. Pre-training is inexpensive. One can replicate all the results given in the paper, in at most 1 hour on a single Cloud TPU, or a few hours on a GPU. For example, SQuAD can be trained in around 30 minutes on a single Cloud TPU to achieve a Dev F1 score of 91.0%. Fine-tuning So our first step is to Clone the BERT github repository, below is the way by which you can clone the repo from github. Now get inside the Bert repo using “ “ command cd !git clone https: //github.com/google-research/bert.git cd bert 3) Download the BERT PRETRAINED MODEL BERT Pretrained Model List : BERT-Large, Uncased (Whole Word Masking): 24-layer, 1024-hidden, 16-heads, 340M parameters BERT-Large, Cased (Whole Word Masking): 24-layer, 1024-hidden, 16-heads, 340M parameters BERT-Base, Uncased: 12-layer, 768-hidden, 12-heads, 110M parameters BERT-Large, Uncased: 24-layer, 1024-hidden, 16-heads, 340M parameters BERT-Base, Cased: 12-layer, 768-hidden, 12-heads, 110M parameters BERT-Large, Cased: 24-layer, 1024-hidden, 16-heads, 340M parameters BERT-Base, Multilingual Cased (New, recommended): 104 languages, 12-layer, 768-hidden, 12-heads, 110M parameters BERT-Base, Multilingual Uncased (Orig, not recommended): 102 languages, 12-layer, 768-hidden, 12-heads, 110M parameters BERT-Base, Chinese: Chinese Simplified and Traditional, 12-layer, 768-hidden, 12-heads, 110M parameters BERT has released and models, that have uncased and cased version. Uncased means that the text is converted to lowercase before performing Workpiece tokenization, e.g., John Smith becomes john smith, on the other hand, cased means that the true case and accent markers are preserved. BERT-Base BERT-Large When using a cased model, make sure to pass -do_lower=False at the time of training. You can download any model of your choice. We have used the BERT-Large-Uncased Model. !wget https: //storage.googleapis.com/bert_models/2018_10_18/uncased_L-24_H-1024_A-16.zip # Unzip the pretrained model
!unzip uncased_L _H _A zip -24 -1024 -16. 4) Download the SQUAD2.0 Dataset For the Question Answering task, we will be using SQuAD2.0 Dataset. (Stanford Question Answering Dataset) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable. SQuAD SQuAD2.0 combines the 100,000+ questions in SQuAD1.1 with over 50,000 new, unanswerable questions written adversarially by crowdworkers to look similar to answerable ones. You can download the dataset from SQUAD site https://rajpurkar.github.io/SQuAD-explorer/ #Download the SQUAD train and dev dataset
!wget https: !wget https: //rajpurkar.github.io/SQuAD-explorer/dataset/train-v2.0.json //rajpurkar.github.io/SQuAD-explorer/dataset/dev-v2.0.json 5) Set up your TPU environment Verify that you are connected to a TPU device You will get know your TPU Address which is used at the time of fine-tuning Perform Google Authentication to access your bucket Upload your credentials to TPU to access your GCS bucket Using code below you can do the above mentioned 4 points: datetime json os pprint random string sys tensorflow tf os.environ, TPU_ADDRESS = + os.environ[ ]
print( , TPU_ADDRESS) google.colab auth
auth.authenticate_user() tf.Session(TPU_ADDRESS) session:
  print( )
  pprint.pprint(session.list_devices()) open( , ) f:
    auth_info = json.load(f)
  tf.contrib.cloud.configure_gcs(session, credentials=auth_info) import import import import import import import import as assert 'COLAB_TPU_ADDR' in 'ERROR: Not connected to a TPU runtime; please see the first cell in this notebook for instructions!' 'grpc://' 'COLAB_TPU_ADDR' 'TPU address is => ' from import with as 'TPU devices:' # Upload credentials to TPU. with '/content/adc.json' 'r' as # Now credentials are set for all future sessions on this TPU. 6) Create an output directory : You will need a GCP (Google Compute Engine) account and a GCS (Google Cloud Storage) bucket to run the colab file. Please follow the Google Cloud for how to create a GCP account and GCS bucket. You have $300 free credit to start with any GCP product, learn more about it at . Prerequisite https://cloud.google.com/tpu/docs/setup-gcp-account You can create your GCS bucket from here . http://console.cloud.google.com/storage As we are using the Cloud TPU, we need to store the pre-trained model and the output directory in the Google Cloud Storage. If you are not storing it on Bucket you may face the following error : ERROR:tensorflow: recorded training_loop: From /job:worker/replica: /task: :
Unsuccessful TensorSliceReader : Failed to get matching files on uncased_L-24_H-1024_A-16/bert_model.ckpt: Unimplemented: File system scheme '[local]' not implemented (file: 'uncased_L-24_H-1024_A-16/bert_model.ckpt')
[[node checkpoint_initializer_14 (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]] Error from 0 0 constructor You will get your fine_tuned model in the Google cloud storage bucket after completion of training. For that, you need to provide your BUCKET name and OUPUT DIRECTORY name. BUCKET = BUCKET, output_dir_name = BUCKET_NAME = .format(BUCKET)
OUTPUT_DIR = .format(BUCKET,output_dir_name)
tf.gfile.MakeDirs(OUTPUT_DIR)
print( .format(OUTPUT_DIR)) 'bertnlpdemo' #@param {type:"string"} assert '*** Must specify an existing GCS bucket name ***' 'bert_output' #@param {type:"string"} 'gs://{}' 'gs://{}/{}' '***** Model output directory: {} *****' 7) Move Pretrained Model to GCS Bucket Need to move Pre-trained Model at GCS (Google Cloud Storage) bucket, as Local File System is not Supported on TPU. If you don’t move your pre-trained model to TPU you may face the error. The command allows you to move data between your local file system and the cloud, move data within the cloud, and move data between cloud storage providers. gsutil mv !gsutil mv /content/bert/uncased_L _H _A $BUCKET_NAME -24 -1024 -16 8) Training Below is the command to run the training. To run the training on TPU you need to make sure about below Hyperparameter, that is tpu must be true and provide the tpu_address that we have found above. -use_tpu=True -tpu_name=YOUR_TPU_ADDRESS !python run_squad.py \
  --vocab_file=$BUCKET_NAME/uncased_L _H _A /vocab.txt \
  --bert_config_file=$BUCKET_NAME/uncased_L _H _A /bert_config.json \
  --init_checkpoint=$BUCKET_NAME/uncased_L _H _A /bert_model.ckpt \
  --do_train= \
  --train_file=train-v2 .json \
  --do_predict= \
  --predict_file=dev-v2 .json \
  --train_batch_size= \
  --learning_rate= \
  --num_train_epochs= \
  --use_tpu= \
  --tpu_name=grpc:// : \
  --max_seq_length= \
  --doc_stride= \
  --version_2_with_negative= \
  --output_dir=$OUTPUT_DIR -24 -1024 -16 -24 -1024 -16 -24 -1024 -16 True .0 True .0 24 3e-5 2.0 True 10.1 .118 .82 8470 384 128 True Create Testing File We are creating input_file.json as a blank JSON file and then add the data in the file in the SQuAD dataset format. is used to create a file - is used to write a file in the colab - touch %%writefile You can pass your own questions and context in the below file. !touch input_file.json %%writefile input_file.json
{ : , : [
        { : , : [
                { : [
                        { : , : , : },
                        { : , : , : },
                        { : , : , : }
                    ], : }
            ]
        }
    ]
} "version" "v2.0" "data" "title" "your_title" "paragraphs" "qas" "question" "Who is current CEO?" "id" "56ddde6b9a695914005b9628" "is_impossible" "" "question" "Who founded google?" "id" "56ddde6b9a695914005b9629" "is_impossible" "" "question" "when did IPO take place?" "id" "56ddde6b9a695914005b962a" "is_impossible" "" "context" "Google was founded in 1998 by Larry Page and Sergey Brin while they were Ph.D. students at Stanford University in California. Together they own about 14 percent of its shares and control 56 percent of the stockholder voting power through supervoting stock. They incorporated Google as a privately held company on September 4, 1998. An initial public offering (IPO) took place on August 19, 2004, and Google moved to its headquarters in Mountain View, California, nicknamed the Googleplex. In August 2015, Google announced plans to reorganize its various interests as a conglomerate called Alphabet Inc. Google is Alphabet's leading subsidiary and will continue to be the umbrella company for Alphabet's Internet interests. Sundar Pichai was appointed CEO of Google, replacing Larry Page who became the CEO of Alphabet." Prediction Below is the command to perform your own custom prediction, that is you can change the input_file.json by providing your paragraph and questions after then execute the below command. !python run_squad.py \
  --vocab_file=$BUCKET_NAME/uncased_L _H _A /vocab.txt \
  --bert_config_file=$BUCKET_NAME/uncased_L _H _A /bert_config.json \
  --init_checkpoint=$OUTPUT_DIR/model.ckpt \
  --do_train= \
  --max_query_length= \
  --do_predict= \
  --predict_file=input_file.json \
  --predict_batch_size= \
  --n_best_size= \
  --max_seq_length= \
  --doc_stride= \
  --output_dir=output/ -24 -1024 -16 -24 -1024 -16 -10859 False 30 True 8 3 384 128 To make it easier for you, we have already created a Colab file which you can copy in your Google Drive and execute the commands. You can access the colab file at: . Question Answering System using BERT + SQuAD on Colab TPU If you have any further questions or doubt then please feel free to post them in comments. We’ll get back to you. Previously published at https://www.pragnakalp.com/nlp-tutorial-setup-question-answering-system-bert-squad-colab-tpu/ References Our case study Question Answering System in Python using BERT NLP BERT based Question and Answering system demo

NLP Tutorial: Creating Question Answering System using BERT + SQuAD on Colab TPU

How to Structure a PyTorch ML Project With Google Colab and TensorBoard

You have a plain old TensorFlow model that’s too computationally expensive to train on your standard-issue work laptop. I get it. I’ve been there too, and if I’m being honest, seeing my laptop crash twice in a row after trying to train a model on it is painful to watch. In this article, I’ll be breaking down the steps on how to train any model on a TPU in the cloud using Google Colab. After this, you’ll never want to touch your clunky CPU ever again, believe me. TLDR: This article shows you how easy it is to train any TensorFlow model on a TPU with very few changes to your code. What’s a TPU? The Tensor Processing Unit (TPU) is an accelerator — custom-made by Google Brain hardware engineers — that specialises in training deep and computationally expensive ML models. Let’s put things into perspective just to give you an idea of how awesome and powerful a TPU is. A standard MacBook Pro Intel CPU can perform a few operations per clock cycle. A standard off-the-shelf GPU can perform tens of thousands of operations per cycle. A state-of-the-art TPU can perform hundreds of thousands of operations per cycle (sometimes up to 128K OPS). To understand the scale, imagine using these devices to print a book. A CPU can print character-by-character. A GPU can print a few words at a time. A TPU? Well, it can print a whole page at a time. That’s some amazing speed and power that we now have at our disposal; shoutout to Google for giving broke high-schoolers (like me) access to high-performance hardware. If you’re interested in the inner workings of a TPU and what makes it so amazing, go check out the Google Cloud blog article where they discuss everything from hardware to software here . Also, the operations per clock cycle for these devices are rough estimates stated by Google engineers themselves. Please don’t call me out to a duel. Running an MNIST model on TPUs For the sake of this tutorial, I’ll be running through a quick and easy MNIST model. Do note that this model can be whatever you want it to be. To better help you visualise what’s going on, I’ve chosen good old MNIST. Let’s first begin by extracting and preprocessing our dataset. This shouldn’t be much of a problem: numpy np tensorflow tf tensorflow.keras.datasets mnist tensorflow.keras.utils to_categorical

(x_train, y_train), (x_test, y_test) = mnist.load_data() x_train = x_train.reshape(x_train.shape[ ], ) x_train = x_train.astype( )
x_test = x_test.astype( )
x_train /= x_test /= y_train = to_categorical(y_train, )
y_test = to_categorical(y_test, ) (x_train.shape, y_train.shape) (x_test.shape, y_test.shape) import as import as from import from import # preprocessing the dataset 0 1 # 784, 1] 'float32' 'float32' 255 255 10 10 print print # >> (60000, 784), (60000, 10) # >> (10000, 784), (10000, 10) I’ve tried to make this article as comprehensive as possible so that you can train your models at unfathomable speeds and feel like you’re on top of the world. There are 4 steps we will be taking to train our model on a TPU: 1. Connect to a TPU 2. Initialise a parallelly-distributed training strategy 3. Build our model under that strategy 4.Train the model and feel like a Superhero Let’s delve right into it! Note: At times, I’ve heard people say that we need to convert our datasets into TFRecords . Bear in mind that it isn’t necessary and that any dataset can be fed into a model for training on a TPU. Connecting to a TPU When I was messing around with TPUs on Colab, connecting to one was the most tedious. It took quite a few hours of searching online and looking through tutorials, but I was finally able to get it done. For that, we first need to connect our Colab notebook running locally to the TPU runtime. To change runtime, simply click the Runtime tab in the navigation bar. A dropdown menu will be displayed from which you can select the Change runtime type option. A popup window will show up where you can select the TPU option from the dropdown selector. Next, we need to see if any TPUs are available for use: os pprint os.environ:
    print( ) :
    tpu_address = + os.environ[ ] ( , tpu_address) tf.Session(tpu_address) session:
      devices = session.list_devices()

    print( )
    pprint.pprint(devices) import import # for pretty printing our device stats if 'COLAB_TPU_ADDR' not in 'ERROR: Not connected to a TPU runtime; please see the first cell in this notebook for instructions!' else 'grpc://' 'COLAB_TPU_ADDR' print 'TPU address is' with as 'TPU devices:' This should give you a list of 8 devices that can be used to train our models on. It should print something out that looks like this: Initialising a Distributed Training Strategy Now that we’ve changed the runtime and have acquired a list of available TPUs, we need to invoke the TPU by creating a distributed training strategy — a wrapper around the model that makes it compatible with multi-core training. To do so, we need to create a that takes in the available device and provisions one for us. TPUClusterResolver resolver = tf.contrib.cluster_resolver.TPUClusterResolver(tpu= + os.environ[ ])
tf.contrib.distribute.initialize_tpu_system(resolver)
strategy = tf.contrib.distribute.TPUStrategy(resolver) 'grpc://' 'COLAB_TPU_ADDR' The gives us access to the TPU such that we can finally build a parallelly-distributed pipeline on it. This is a necessary step because a TPU is a distributed training processor and is not single-core like a traditional CPU. With this strategy method, jumping on board a TPU is very simple! resolver Building our model under the distributed training strategy With a training strategy now at hand, we can proceed to build our model using that strategy as such: strategy.scope(): model = tf.keras.models.Sequential()
	model.add(tf.keras.layers.Dense( , input_size=[ ,], activation= ))
	model.add(tf.keras.layers.Dense( , activation= ))
	model.add(tf.keras.layers.Dense( , activation= ))
	model.add(tf.keras.layers.Dense( , activation= ))
	model.add(tf.keras.layers.Dense( , activation= )) model.compile(
		optimizer=tf.train.RMSPropOptimizer(learning_rate= ),
		loss=tf.keras.losses.sparse_categorical_crossentropy,
		metrics=[ ]
	)

	model.summary() with """
	Note: This model can be whatever you want it to be.
	Here, I'm building a simple fully-connected network using 
	our distributed training strategy. 
	This essentailly takes our model and makes it 
	compatible to train on a TPU.
	""" 512 784 'relu' 256 'relu' 128 'relu' 64 'relu' 10 'softmax' # compiling the model using the RMSProp optimizer # and Sparse Categorical Crossentropy loss 1e-2 'sparse_categorical_accuracy' This looks like a lot of jargon, but all it does is convert a regular model (that is typically run on a CPU) into a TPU-ready model for distributed training. tf.keras Training our model This is the exciting part of the process. We can finally train our model on a Cloud TPU for free. With so much power in our hands, let’s make good use of it and train on MNIST (I know…very anti-climatic). history = model.fit(x_train,  y_train, epochs= , steps_per_epoch= )
model.save_weights( , overwrite= ) 20 50 './mnist_model.h5' True Ensure that the number of instances is perfectly divisible by the parameter so that all the instances are used during training. For example, we have 60000 instances in our training set. 60000 is divisible by 50 so that means all our instances are fed into the model without any leftovers. steps_per_epoch A final round-up With that, training should commence soon. Colab will boot up a TPU and upload the model architecture on it. You should soon see the classic Keras progress bar style layout in the terminal output. Congrats! You’ve successfully just used a TPU. Introspectively, I always thought training on a TPU was a thing only wizards could handle. TPU training was always that one big thing out of my grasp simply because of the hype around it (making it seem very difficult to use). Never in my wildest dreams did I think it’d be as simple as adding 5–6 lines of code to the preexisting model. After reading this article, I want you to know that anyone can train on accelerated hardware regardless of experience. Some observations about TPU training The Neural Machine Translation model I had written (for the TensorFlow Docs) took me less than a minute to train on a TPU. I was astonished because the same model took more than 4 hours to train on a CPU (which probably explains why my laptop crashed and ran out of memory twice). My model hit a very high accuracy — higher than what I achieved training on a CPU. With performance and speed, the Cloud TPU is second to none when it comes to quality training! Note: You can find my code here . Have fun! In a nutshell Google always comes bearing gifts when it launches new Machine Learning toys that we can tinker and play around with. The TPU processor is certainly a boon to the ML community as it plays a major role in the democratisation of AI — it gives everyone a chance to use accelerated hardware regardless of demographic. With TPUs, research and experimentation are hitting all-time highs and people are now engaging in Machine Learning like never before! I hope this article has helped you train your models in a much more efficient and elegant manner. If you have any questions about the use of TPUs or want to chat in general about Tech and ML, feel free drop them down below in the comments or catch me on or . I usually reply within a day. Twitter LinkedIn Until then, I’ll catch you in the next one! Original article by Rishabh Anand A call to action…of sorts Interested in reading about the latest-and-greatest technologies and getting your hands dirty with the recent happenings in Machine Learning, Data Science, and Technology? Do catch my other articles or give me a follow! Your feedback and constant support mean a lot and encourage me to continue writing high-quality content for your learning! Give me a follow if you love reading about technology, especially Machine Learning and Data Science! You can find me . here

Stories, News & Mentions about ECOLAB

Categories

Trending Topics

Stories, News & Mentions about ECOLAB

Ecolab’s Stories on HackerNoon

Ecolab's News & Mentions

Categories

Trending Topics