Async

async

One of my favorite interview questions is “What do such words as async and await tell you?” because it opens up an opportunity to have an interesting discussion with an interviewee… Or it doesn’t because they float on this topic. In my opinion, it is drastically important to understand why we use this technique. I feel like many developers prefer to rely on the “it is the best practice” statement and use asynchronous methods blindly. This article shows the difference between asynchronous and synchronous methods in practice. Tools .NET Web API application (test target) 2 Azure SQL Databases 2 Azure App Service on Windows (hosts the application) Azure App Insights (to gather metrics) locust framework (to simulate user load). Configuration I will run a benchmark in the following way. Two independent locust instances are running on two machines. Locust instances simulate a user that does the following: The user from the locust host 1 hits the synchronous endpoint of the App Service 1, gets the response, and stays idle for 0.5–1 seconds (the exact time delay is random). Repeats till the end of the experiment. The user from the locust host 2 behaves precisely the same, with only one difference — he hits the asynchronous endpoint of App Service 2. Under the hood, each App Service connects to its own database and executes a SELECT query that takes five seconds and returns a few rows of data. See the controller’s code below for references. I’ll use Dapper to make a call to the database. I’d like to draw your attention to the fact that the asynchronous endpoint calls the database asynchronously as well (QueryAsync<T>). It's worth adding that I deploy the same code to both app services. During the test, the number of users grows evenly to the target number (Number of Users). The speed of growth is controlled by a Spawn Rate parameter (number of unique users to join per second) — the higher the number is, the quicker the users are being added. The spawn rate is set to 10 users/s for all experiments. All experiments are limited to 15 minutes. You may find machine configuration details in the Technical Details section of the article. Metrics requests per minute — shows the number of requests that the application actually processed and returned a status code.
thread count — shows the number of threads the app service consumes.
median response time, ms The red lines refer to asynchronous, and the blue lines — to the synchronous endpoint, respectively. That’s it about theory. Let’s start. Experiment #1 number of users: 75 (per service) We can see that both endpoints perform similarly — handling around 750 requests per minute with a median response time of 5200 ms. The most fascinating graph in this experiment is a thread trend. You can see significantly higher numbers for the synchronous endpoint (a blue graph) — more than 100 threads! That’s expected, though, and matches the theory — when a request comes in, and the application makes a call to the database, the thread is blocked because it has to wait for a roundtrip to complete. Therefore, when another request comes in, the application has to produce a new thread to handle it. The red graph — the asynchronous endpoint thread count— proves different behavior. When a request comes in and the application makes a call to the database, the thread returns to a thread pool instead of being blocked. Therefore, when another request comes in, this free thread is reused. Despite incoming requests growing, the application doesn’t require any new threads, so their count remains the same. It’s worth mentioning the 3rd metric — median response time. Both endpoints showed the same result — 5200 ms. So, there is no difference in terms of performance. Now, it is time to pull up the stakes. Experiment #2 number of users: 150 We doubled the load. The asynchronous endpoint handles this task successfully—its request per minute rate floats around 1500. The synchronous brother eventually reached a comparable number of 1410. But if you look at the graph below, you will see that it took 10 minutes! The reason is that the synchronous endpoint reacts to a new user's arrival by creating another thread, but users are being added to the system (just to remind you that the Spawn Rate is 10 users/s) quicker than the web server can adapt. That’s why it queued so many requests at the very beginning. Unsurprisingly, the thread count metric is still around 34 for the asynchronous endpoint, while it increased from 102 to 155 for the synchronous one. The median response time degraded similarly to the request per minute rate — the synchronous response time was much higher at the beginning of the experiment. If I had kept the test for 24 hours, the median numbers would become even. Experiment #3 number of users: 200 The third experiment is intended to prove trends revealed during the second one —we can see further degradation of the synchronous endpoint. Conclusion Using asynchronous instead of synchronous operations doesn’t directly improve performance or user experience. First, it enhances stability and predictability under pressure. In other words, it raises the load threshold so the system can process more before it degrades. Appendix #1. Technical Details Azure App Service: B1, 100 ACU, 1,75 Gb Memory, A-Series compute equivalent.
Azure SQL Database: Standard S4: 200 DTUs, 500 Mb storage.
SQL Connection settings: Max Pool Size=200. Appendix #2. Notes To achieve the cleanest test result, I should have run tests from 2 VMs located in the same network where the target App Services sit. However, I assumed that a network lag would impact both apps in a more or less similar way. Therefore, it cannot jeopardize the main goal — comparing how asynchronous and synchronous methods behave. Appendix #3. Bonus Experiment What did I hack to force the synchronous endpoint to perform almost as asynchronous and plot the graph below (experiment conditions are the same as in the third one — 200 users)? One of my favorite interview questions is “What do such words as async and await tell you?” because it opens up an opportunity to have an interesting discussion with an interviewee… Or it doesn’t because they float on this topic. In my opinion, it is drastically important to understand why we use this technique. async await I feel like many developers prefer to rely on the “it is the best practice” statement and use asynchronous methods blindly. This article shows the difference between asynchronous and synchronous methods in practice. Tools .NET Web API application (test target) .NET Web API application (test target) 2 Azure SQL Databases 2 Azure SQL Databases 2 Azure App Service on Windows (hosts the application) 2 Azure App Service on Windows (hosts the application) Azure App Insights (to gather metrics) Azure App Insights (to gather metrics) locust framework (to simulate user load). locust framework (to simulate user load). locust Configuration I will run a benchmark in the following way. Two independent locust instances are running on two machines. Locust instances simulate a user that does the following: The user from the locust host 1 hits the synchronous endpoint of the App Service 1, gets the response, and stays idle for 0.5–1 seconds (the exact time delay is random). Repeats till the end of the experiment. The user from the locust host 1 hits the synchronous endpoint of the App Service 1, gets the response, and stays idle for 0.5–1 seconds (the exact time delay is random). Repeats till the end of the experiment. synchronous App Service 1, The user from the locust host 2 behaves precisely the same, with only one difference — he hits the asynchronous endpoint of App Service 2. The user from the locust host 2 behaves precisely the same, with only one difference — he hits the asynchronous endpoint of App Service 2. asynchronous App Service 2. Under the hood, each App Service connects to its own database and executes a SELECT query that takes five seconds and returns a few rows of data. See the controller’s code below for references. I’ll use Dapper to make a call to the database. I’d like to draw your attention to the fact that the asynchronous endpoint calls the database asynchronously as well ( QueryAsync<T> ). QueryAsync<T> It's worth adding that I deploy the same code to both app services. During the test, the number of users grows evenly to the target number ( Number of Users ). The speed of growth is controlled by a Spawn Rate parameter (number of unique users to join per second) — the higher the number is, the quicker the users are being added. The spawn rate is set to 10 users/s for all experiments. Number of Users Spawn Rate spawn rate All experiments are limited to 15 minutes. You may find machine configuration details in the Technical Details section of the article. Metrics requests per minute — shows the number of requests that the application actually processed and returned a status code. thread count — shows the number of threads the app service consumes. median response time, ms requests per minute — shows the number of requests that the application actually processed and returned a status code. requests per minute thread count — shows the number of threads the app service consumes. thread count median response time, ms median response time, ms The red lines refer to asynchronous, and the blue lines — to the synchronous endpoint, respectively. That’s it about theory. Let’s start. Experiment #1 Experiment #1 number of users: 75 (per service) number of users : 75 (per service) number of users We can see that both endpoints perform similarly — handling around 750 requests per minute with a median response time of 5200 ms. The most fascinating graph in this experiment is a thread trend. You can see significantly higher numbers for the synchronous endpoint (a blue graph) — more than 100 threads! That’s expected, though, and matches the theory — when a request comes in, and the application makes a call to the database, the thread is blocked because it has to wait for a roundtrip to complete. Therefore, when another request comes in, the application has to produce a new thread to handle it. The red graph — the asynchronous endpoint thread count— proves different behavior. When a request comes in and the application makes a call to the database, the thread returns to a thread pool instead of being blocked. Therefore, when another request comes in, this free thread is reused. Despite incoming requests growing, the application doesn’t require any new threads, so their count remains the same. It’s worth mentioning the 3rd metric — median response time . Both endpoints showed the same result — 5200 ms. So, there is no difference in terms of performance. median response time Now, it is time to pull up the stakes. Experiment #2 number of users: 150 number of users : 150 number of users We doubled the load. The asynchronous endpoint handles this task successfully—its request per minute rate floats around 1500. The synchronous brother eventually reached a comparable number of 1410. But if you look at the graph below, you will see that it took 10 minutes! request per minute The reason is that the synchronous endpoint reacts to a new user's arrival by creating another thread, but users are being added to the system (just to remind you that the Spawn Rate is 10 users/s) quicker than the web server can adapt. That’s why it queued so many requests at the very beginning. Spawn Rate Unsurprisingly, the thread count metric is still around 34 for the asynchronous endpoint, while it increased from 102 to 155 for the synchronous one. The median response time degraded similarly to the request per minute rate — the synchronous response time was much higher at the beginning of the experiment. If I had kept the test for 24 hours, the median numbers would become even. thread count median response time request per minute Experiment #3 number of users: 200 number of users : 200 number of users The third experiment is intended to prove trends revealed during the second one —we can see further degradation of the synchronous endpoint. Conclusion Using asynchronous instead of synchronous operations doesn’t directly improve performance or user experience. First, it enhances stability and predictability under pressure. In other words, it raises the load threshold so the system can process more before it degrades. Appendix #1. Technical Details Azure App Service: B1, 100 ACU, 1,75 Gb Memory, A-Series compute equivalent. Azure SQL Database: Standard S4: 200 DTUs, 500 Mb storage. SQL Connection settings: Max Pool Size=200. Azure App Service: B1, 100 ACU , 1,75 Gb Memory, A-Series compute equivalent. ACU Azure SQL Database: Standard S4: 200 DTUs, 500 Mb storage. SQL Connection settings: Max Pool Size=200. Appendix #2. Notes To achieve the cleanest test result, I should have run tests from 2 VMs located in the same network where the target App Services sit. However, I assumed that a network lag would impact both apps in a more or less similar way. Therefore, it cannot jeopardize the main goal — comparing how asynchronous and synchronous methods behave. Appendix #3. Bonus Experiment Appendix #3. Bonus Experiment What did I hack to force the synchronous endpoint to perform almost as asynchronous and plot the graph below (experiment conditions are the same as in the third one — 200 users)?

Async vs Sync Benchmark (.NET): The Difference Between Asynchronous and Synchronous Methods

مقارنة بين Async وSync (.NET): الفرق بين الطرق المتزامنة وغير المتزامنة

Async vs Sync Benchmark (.NET): Asinxron və Sinxron Metodlar Arasındakı Fərq

Async vs Sync Benchmark (.NET): la diferencia entre métodos asincrónicos y sincrónicos

Async vs Sync Benchmark (.NET): a diferenza entre os métodos asíncronos e síncronos

Async vs Sync Benchmark (.NET): Az aszinkron és a szinkron módszerek közötti különbség

Async vs Sync Benchmark (.NET): la differenza tra metodi asincroni e sincroni

非同期 vs 同期ベンチマーク (.NET): 非同期メソッドと同期メソッドの違い

Async vs Sync Benchmark (.NET)៖ ភាពខុសគ្នារវាងវិធីសាស្ត្រ Asynchronous និង Synchronous Methods

Async vs Sync Benchmark (.NET): Асинхрондуу жана синхрондуу методдордун ортосундагы айырма

Async vs Sync Benchmark (.NET): różnica między metodami asynchronicznymi i synchronicznymi

Async vs Sync Benchmark (.NET): Rozdiel medzi asynchrónnymi a synchrónnymi metódami

Async vs Sync Benchmark (.NET): Musiyano uripakati peAsynchronous uye Synchronous Methods

Designed for developers at the intersection of curiosity and professional growth, this primer aims to illuminate how asynchronous programming can elevate application performance, scalability, and responsiveness. By focusing on the universal principles of non-blocking operations and event loops, we venture beyond specific technologies like AsyncIO, Node.js, or Go. This exploration is crafted for software developers eager to grasp the efficiencies driving modern software without the complexity of technical jargon. Think of how a single thread handling requests is like a single-lane road in a busy city: each request is a car, and when there's only one lane, cars line up, causing delays. But what if our city — our service — could manage traffic more cleverly? That's where the magic of an asynchronous runtime comes into play. It's like adding an intelligent traffic system to our city, guiding cars through multiple lanes and intersections without waiting. This system keeps traffic flowing smoothly, ensuring no car (or task) waits too long, which is precisely how an asynchronous runtime keeps our software running efficiently. Let's consider an example of a service that completes I/O operations synchronously. For clarity, these operations are shown outside the main flow of execution in the diagram: Blocking I/O during your service's startup phase might be acceptable, but avoiding this approach when processing external requests is advisable. The following diagram illustrates how service efficiency can be enhanced through the adoption of non-blocking I/O operations: These examples highlight scenarios where the server's performance gains are maximized. Nevertheless, the advantage of non-blocking operations persists across any frequency of incoming requests. Even in less-than-ideal conditions, performance benefits from integrating dedicated I/O threads into the processing workflow. The total time required to process a request (from the client's initial request to the final response, as depicted by the blue number on the right) will invariably decrease, provided there are sufficient threads to handle all requests. In the worst-case scenario, this duration will not surpass that of synchronous processing methods. We then encounter a practice that might seem counterintuitive to many developers. When I/O operations constitute a significant portion of request processing time, optimizing other code segments may yield slight improvement. The duration to fetch data from a cache might align closely with the time dedicated to business logic and template rendering. Employing caching or an in-process database can reduce data retrieval times compared to other processing activities. To segment code effectively and facilitate callback execution, one can instruct the runtime to proceed to the next event loop cycle. Below is an illustrative example of how this concept is applied: // blocking callbacks
function func1_cb(str, cb) {
 var res = func1(str);
 cb(res);
}

function func2_cb(str, cb) {
 var res = func2(str);
 cb(res);
}

// non-blocking callbacks
function func1_cb(str, cb) {
 var res = func1(str);
 process.nextTick(function () {
   cb(res);
 });
}

function func2_cb(str, cb) {
 var res = func2(str);
 process.nextTick(function () {
   cb(res);
 });
}

// usage example
func1_cb(content, function (str) {
 func2_cb(str, function (result) {
   // work with result
 });
}); By adopting this methodology to split two parts of calculations, the overall processing time for scenarios with nearly concurrent request arrivals remains the same. However, the response to the first request is delayed: This scenario represents the least favorable outcome when employing the “next tick” strategy. As shown in the initial example, the “next tick” method proves benign for infrequent requests. Should requests arrive at a moderate pace, leveraging this technique enhances processing speed by allowing the initiation of new requests and the start of non-blocking operations to be interleaved during execution breaks. This approach effectively reduces both the total processing time and the average time per request: In conclusion, adopting non-blocking I/O is crucial for enhancing application performance and beneficial in environments with both sparse and heavy incoming request volumes. Additionally, effectively sequencing the execution flow — illustrated by concepts similar to the "next tick" technique — significantly improves server efficiency. Embracing these asynchronous programming practices offers clear advantages over traditional, synchronous methods.

Asynchronous Runtimes: A Primer for the Perplexed Developer

Asynchronous Runtimes: A Primer for the Perplexed ডেভেলপার

Asynchrone Laufzeiten: Eine Einführung für den verwirrten Entwickler

Runtimes asíncronos: un primer para los desarrolladores perplejos

Environnements d'exécution asynchrones : un guide pour les développeurs perplexes

Asynchronous Runtimes: एक भ्रमित डेवलपर्स के लिए एक प्राथमिक

非同期ランタイム: 困惑する開発者のための入門書

Asynchronous Runtimes: A Primer for the Perplexed 개발자

Tempos de execução assíncronos: um guia para o desenvolvedor perplexo

Асинхронные среды выполнения: практическое руководство для озадаченных разработчиков

Asynchronous Runtimes: A Primer for the Perplexed Geliştirici

Thời gian chạy không đồng bộ: Một bài học vỡ lòng cho nhà phát triển bối rối

异步运行时：困惑的开发人员的入门指南

Async await in is a feature that allows you to write asynchronous code that is easy to read and maintain. It greatly simplifies how you handle asynchronous operations by providing a structured way to await tasks and their results. With the rate at which computers are scaling performance & cores, and our growing need to interface with many external services, async await in C# can make our developer lives easier. C# In the following sections, I’ll highlight 3 beginner tips for using async await in C#. Each tip will be accompanied by a code example, allowing you to see these concepts in action. These cancellation, and performance optimization, so you’ll be off to the races writing async code. tips will cover important aspects such as error handling, Tip 1: Understand the Basics of async await in C# Async await in C# are two special keywords that are used to handle asynchronous code execution. Asynchronous programming enables us to write C# code that can perform multiple operations concurrently, improving the overall performance and responsiveness of an application. In C#, a task represents an asynchronous operation and it’s used to encapsulate a unit of work that can be executed concurrently with other tasks. When using async/await, you can use tasks to handle and manage asynchronous operations in a structured way. To define an async method in C#, you need to add the async modifier to the method signature. For example: public async Task DoAsyncWork()
{
    // Asynchronous code here
} In the above example, the method is declared as async and returns a . This indicates that the method will perform asynchronous operations and can be awaited. DoAsyncWork Task Within the async method, you can use the await keyword to signal the point where the execution should wait for the completion of an asynchronous operation. For example: public async Task DoAsyncWork()
{
    // Perform asynchronous operations
    await Task.Delay(1000);

    // Continue with other operations
} In the above example, the statement pauses the execution of the method until the operation completes. This allows other code, such as other async tasks, to execute concurrently. By using tasks, you can of asynchronous programming in C#. await Task.Delay(1000) Task.Delay harness the power Tip 2: Handling Multiple Exceptions with async await . When working with async code in C#, handling exceptions can present some challenges due to the asynchronous nature of the code. In certain situations, multiple exceptions can occur when executing async code. It’s important to be able to handle and aggregate these exceptions effectively. To handle multiple exceptions, C# provides the class, which allows us to group and manage multiple exceptions as a single entity. Exception handling is an important aspect of writing any software application AggregateException When multiple exceptions occur in an async operation, the class collects these exceptions and allows us to access and handle them in a centralized manner. This can be particularly useful for scenarios where we need to process all exceptions generated during an async operation. To handle exceptions using the class, we can wrap the code that may throw exceptions in a try-catch block. Within the catch block, we can access the property of the to access each individual exception that occurred. AggregateException AggregateException InnerExceptions AggregateException Here’s an example of how to use the class to handle and aggregate exceptions in async code: AggregateException try
{
    await SomeAsyncOperation();
}
catch (AggregateException ex)
{
    foreach (var innerException in ex.InnerExceptions)
    {
        // Handle each individual exception
        Console.WriteLine($"Exception: {innerException.Message");
    }
} In the example above, represents an async operation that may throw one or more exceptions. By wrapping the statement in a try-catch block and catching an , we are able to access and handle each individual exception that occurred within the async operation. By properly handling and aggregating exceptions in async code using the class, we can ensure that our code can gracefully handle any unexpected errors that may occur during async operations. SomeAsyncOperation await AggregateException AggregateException Tip 3: Avoiding Deadlocks with async await Deadlocks can occur when two or more tasks are waiting for each other to complete indefinitely, resulting in a state where none of the tasks can proceed. Stop Blocking Your Async Calls! One of the most common causes of deadlocks in async methods includes using the or methods inside an async method, which blocks the main thread and can lead to deadlocks. The best advice for avoiding deadlocks with async code: Task.Result Task.Wait “Don’t block on async code” Stephen Cleary He points out in his article that this tip (stop blocking) or the tip that immediately follows this (configuring the awaiter) are both able to help prevent deadlocks… but relying ONLY on the next pattern is more of a risk. That is, of these two suggestions, making your C# code async await all of the way through is the safer bet. That means instead of having something like: public class MyController(
  IWeatherService _weatherService) :
  ApiController
{
  public string GetWeatherData()
  {
    var weatherData = _weatherService.GetWeatherData(...).Result;
    return Results.OK(weatherData);
  }
} Instead, we’d make the top-level async as well: public class MyController(
  IWeatherService _weatherService) :
  ApiController
{
  public async Task<string> GetWeatherData()
  {
    var weatherData = await _weatherService.GetWeatherData(...);
    return Results.OK(weatherData);
  }
} As soon as we introduce a or call, we are effectively blocking — and that is one of the key situations that can lead to deadlocks as the application gets more complex. .Result .Wait() Using ConfigureAwait(false) with Async Await in C# To prevent deadlocks, the method can be used with the parameter. This instructs the operator to not capture the current synchronization context, allowing the continuation to be executed on any available thread from the thread pool — not just the thread that started the call! ConfigureAwait false await With , we can explicitly state that we do not require the original synchronization context to be captured, preventing the deadlock issue. Keep in mind that using should be done carefully and only when it is absolutely necessary, as it may introduce potential issues if the code relies on the captured synchronization context for specific requirements. For example, if you’re running UI logic and you come back to execution not on the UI thread, you may be in for some surprises! ConfigureAwait(false) ConfigureAwait(false) Let’s take a look at an example to illustrate the usage of to prevent deadlocks: ConfigureAwait(false) public async Task<string> GetDataAsync()
{
    // NOTE: assume we enter this method on some given thread

    // use a false parameter on ConfigureAwait
    await SomeAsyncMethod().ConfigureAwait(false);

    // NOTE: we could be on a different thread than when we started this method!

    return ProcessData();
} In the above example, is awaited using . This allows the continuation of the code to execute on any available thread, ensuring that the synchronization context is not captured and preventing potential deadlocks. SomeAsyncMethod ConfigureAwait(false) And while Stephen Cleary does say that this will help prevent deadlocks, he also had this to add: Using to avoid deadlocks is a dangerous practice. You would have to use for in the transitive closure of all methods called by the blocking code, . Using to avoid deadlock is at best ). ConfigureAwait(false) ConfigureAwait(false) every await including all third- and second-party code ConfigureAwait(false) just a hack Stephen Cleary Wrapping up Async Await in C# In this article, I’ve discussed 3 beginner tips for using async await in C# to write concurrent code. Async await in C# can be a lot to absorb right in the beginning, so it will take some time and practice for it to feel more natural! We got to see the basics of the keyword usage for async await — With this, you can now write asynchronous code! From there, we looked at how multiple exceptions can arise in async await code paths and how to navigate that. And we concluded with some ideas around preventing deadlocks by looking at 2 possibilities we can leverage. Async await in C# is something you’re going to want to learn and practice! If you found this useful and you’re looking for more learning opportunities, consider and check out my ! subscribing to my free weekly software engineering newsletter free videos on YouTube Also published here.

3 Beginner Tips You Need to Know for async await in C#

Handling asynchronous operations in Javascript is a fundamental part of creating responsive and interactive web applications. allows tasks like API calls, file reading, or timers to run in the background, ensuring the application remains responsive. Enter JavaScript Promises, a powerful abstraction for managing these asynchronous operations. Asynchronous programming The Basics of Promises A in JavaScript represents the eventual completion (or failure) of an asynchronous operation and its resulting value. A promise can be in one of three states: Promise : The initial state where the operation has not been completed yet. Pending : The operation was completed successfully. Fulfilled : The operation failed. Rejected Structure of a Promise The basic syntax for creating a promise is: const myPromise = new Promise((resolve, reject) => {
  // Asynchronous operation here
  if (/* operation successful */) {
    resolve('Success');
  } else {
    reject('Error');
  }
}); Creating and Using Promises Creating a promise involves passing an executor function to the Promise constructor, which contains the asynchronous operation and dictates whether the promise should be resolved or rejected. Using Promises Once a promise is created, it can be used with , , and methods to handle the fulfilled or rejected state. then() .catch() .finally() myPromise
  .then((value) => {
    // Handle success
    console.log(value);
  })
  .catch((error) => {
    // Handle error
    console.error(error);
  })
  .finally(() => {
    // Execute cleanup or final operations
    console.log('Completed');
  }); allows you to chain multiple promises, creating a sequence of asynchronous operations that execute one after the other. .then() Error Handling in Promises Proper error handling is crucial in asynchronous programming to ensure the reliability and robustness of web applications. Promises provide a clean and straightforward way to catch and handle errors. Using .catch() The .catch() method is used to handle any errors that occur in the promise chain. fetch('https://api.example.com/data')
  .then(response => response.json())
  .catch(error => console.error('Failed to fetch data:', error)); Error propagation in promises ensures that if an error is not caught by a in the chain, it will bubble up to the next . .catch() .catch() Advanced Promise Features Promises in JavaScript offer more than just basic functionality. There are several advanced features designed to handle complex asynchronous patterns efficiently. Promise.all() When you need to run multiple promises in parallel and wait for all of them to complete, is incredibly useful. Promise.all() Promise.all([promise1, promise2, promise3])
  .then((results) => {
    // results is an array of each promise's result
    console.log(results);
  })
  .catch((error) => {
    // If any promise is rejected, catch the error
    console.error("A promise failed to resolve", error);
  }); Promise.race() is similar to , but it resolves or rejects as soon as one of the promises in the iterable resolves or rejects, with the value or reason from that promise. Promise.race() Promise.all() Promise.race([promise1, promise2, promise3])
  .then((value) => {
    // Value of the first resolved promise
    console.log(value);
  })
  .catch((error) => {
    // Error of the first rejected promise
    console.error(error);
  }); Promise.allSettled() This method returns a promise that resolves after all of the given promises have either been resolved or rejected, with an array of objects that each describe the outcome of each promise. Promise.allSettled([promise1, promise2, promise3])
  .then((results) => {
    results.forEach((result) => console.log(result.status));
  }); Promise.any() takes an iterable of Promise objects and, as soon as one of the promises in the iterable fulfills, returns a single promise that resolves with the value from that promise. Promise.any() Promise.any([promise1, promise2, promise3])
  .then((value) => {
    console.log(value);
  })
  .catch((error) => {
    console.error('All promises were rejected');
  }); Real-world Use Cases for Promises Promises are not just theoretical constructs but have practical applications in everyday programming. Here are a couple of examples: Fetching Data from an API One common use case is fetching data from a remote API: fetch('https://api.example.com/data')
  .then(response => response.json())
  .then(data => console.log(data))
  .catch(error => console.error('Fetch error:', error)); Wrapping Callback-based APIs Another practical use is to wrap callback-based functions (like those in Node.js) into promises for a cleaner, more modern API usage: const fs = require('fs').promises;

fs.readFile('path/to/file.txt', 'utf8')
  .then(data => console.log(data))
  .catch(error => console.error('Read file error:', error)); Promises vs. Async/Await While promises are powerful on their own, the async/await syntax introduced in ES2017 provides a more straightforward way to work with asynchronous operations, making your code cleaner and easier to understand. async function fetchData() {
  try {
    const response = await fetch('https://api.example.com/data');
    const data = await response.json();
    console.log(data);
  } catch (error) {
    console.error('Fetch error:', error);
  }
} async/await is syntactic sugar over promises and can be used wherever promises are used. Best Practices and Common Mistakes To make the most out of promises, here are some best practices and common pitfalls to avoid: Just like callback hell, promise chains can become nested and complicated. Use Promise.all() or async/await to keep your code clean and readable. Avoid Promise Hell: Unhandled promise rejections can cause hard-to-find bugs. Always use .catch() or try/catch with async/await. Always Catch Errors: Remember to return promises from your .then() handlers to keep the chain going. Chain Properly: Conclusion JavaScript promises are a robust tool for handling asynchronous operations, offering a more manageable approach to callbacks. By understanding and leveraging promises, you can write cleaner, more efficient JavaScript code. Keep experimenting with promises and async/await to find the patterns that work best for your applications. Also published . here

Mastering Asynchronous JavaScript: An In-Depth Guide to JavaScript Promises and Best Practices

When we discuss async EventHandlers, the first thing that comes to mind for many of us is that it’s the only exception that we seem to allow for . the dreaded async void setup When I had written about this before, I was excited that I was exploring a solution that involved actually allowing the async void to exist (without wanting to pull the rest of my hair out). For me, this was much more about some clever tricks we can use to overcome async EventHandlers than it was to provide solutions for avoiding the problem entirely. With that said though, there was a lot of traction on the article, which I am very thankful for, and some folks expressed opinions that they’d rather solve async EventHandlers a different way. I thought this was a great point, so I wanted to come up with an alternative approach that doesn’t fix async void, but it allows you to a-void it (see what I did there?) entirely while solving some of the challenges with async EventHandlers. In this article, I will present another solution that you can try out in your own code. We’ll address the pros and cons from my perspective with respect to how it can be used so you can decide if it makes sense for your use case. You can also find some interactable code on .NET fiddle right . Otherwise, you can if you’d like to clone it down locally to try it out. over here check the code out on GitHub A Companion Video! to check out the video! Click here The Problem The problem we face with async EventHandlers is that the signature for events that we can subscribe to in C# by default looks something like this: void TheObject_TheEvent(object sender, EventArgs e); And, you’ll notice that by having void out the front of this signature, we’re forced to use void in our own handlers in order to subscribe to the event. This means that if you want your handler to ever run async/await code, you’ll need to await inside your void method… Which introduces the big scary async void pattern that we’re told to avoid like the plague. And why? Because async void breaks the ability for exceptions to bubble up properly and can cause a ton of headaches as a result. addressed this by allowing us to get creative on the invocation side of things but… The previous article We might need support for this on objects we don’t control the invocation of events for (i.e., you are hooking up to a button’s click event in your favorite UI framework) Some people see the usage of the context inside of that solution as a hack (I’m not disagreeing with that either). … Specifically, with event handlers, we have some other more simple tricks we can do to support async EventHandlers! In my opinion, simple is better… so if you read my previous article on async void and your goal was really just to deal with EventHandlers, this should help. Solving Async EventHandlers With Try/Catch Based on the conditions previously stated, the exception handling breaks down over the boundary of the async void. If you have an exception that needs to bubble up crossing this boundary, then you’re going to be in for a fun time. And by fun, I mean if you enjoy debugging why stuff isn’t working and you don’t have a clear indication as to what’s breaking, then you’ll really have a great time. So what’s the easiest way to fix this? Let’s prevent exceptions from being able to cross this boundary in the first place by using a simple tool we have access to: try/catch. objectThatRaisesEvent.TheEvent += async (s, e) =>
{
    // if the try catch surrounds EVERYTHING in the handler, no exception can bubble up
    try
    {
        await SomeTaskYouWantToAwait();
    }
    catch (Exception ex)
    {
        // TODO: put your exception handling stuff here
    }

    // no exception can escape here if the try/catch surrounds the entire handler body
} As noted in the code above, if you place a try/catch block around the ENTIRE body of your event handler, then you can prevent any exceptions from bubbling up across that async void boundary. On the surface, it’s quite simple and doesn’t require anything fancy to implement this. Pros: Extremely simple. No complex mechanisms to understand. No packages required. You don’t need to be the owner of the class that raises the event for this to work. This means that this approach will work for all existing event-raising objects including WinForms and WPF UI components. Cons: You need to remember to do this… everywhere. It’s possible that as your code evolves over time, someone might accidentally write logic outside of the event handler’s try-catch that can throw exceptions With that said, this solution truly is simple, but I think we can do a little bit better. A (Slightly) Fancier Approach to Improving Async EventHandlers One improvement that I think we can make over the initially proposed solution is that we can make it a little bit more explicit that we have an async EventHandler that should be safe from bubbling up exceptions. This approach will also prevent code drift over time from causing problematic code from running outside of the event handler. However, it will not address the fact that you need to remember to add this in manually! Let’s check out the code: static class EventHandlers
{
    public static EventHandler<TArgs> TryAsync<TArgs>(
        Func<object, TArgs, Task> callback,
        Action<Exception> errorHandler)
        where TArgs : EventArgs
		=> TryAsync<TArgs>(
            callback,
            ex =>
            {
                errorHandler.Invoke(ex);
                return Task.CompletedTask;
            });
    
    public static EventHandler<TArgs> TryAsync<TArgs>(
        Func<object, TArgs, Task> callback,
        Func<Exception, Task> errorHandler)
        where TArgs : EventArgs
    {
        return new EventHandler<TArgs>(async (object s, TArgs e) =>
        {
	    try
            {
                await callback.Invoke(s, e);
            }
            catch (Exception ex)
            {
                await errorHandler.Invoke(ex);
            }
        });
    }
} The code above quite literally uses the exact same approach for preventing exceptions from crossing the async void boundary. We simply try to catch around the body of the event handler, but now we’ve bundled it up into an explicitly dedicated method to reuse. Here’s how it would look to apply it: someEventRaisingObject.TheEvent += EventHandlers.TryAsync<EventArgs>(
    async (s, e) =>
    {
        Console.WriteLine("Starting the event handler...");
    	await SomeTaskToAwait();
        Console.WriteLine("Event handler completed.");
    },
    ex => Console.WriteLine($"[TryAsync Error Callback] Our exception handler caught: {ex}")); We can see that we now have a delegate with an async Task signature to work with, and anything we put inside of that we rest assured will have a try/catch around it within the helper method we saw earlier. Here’s a screenshot showing the error handler callback properly capturing the exception: Pros: Still very simple. The wrapper function is *slightly* more complex, but still very basic. No packages required. You don’t need to be the owner of the class that raises the event for this to work. This means that this approach will work for all existing event-raising objects including WinForms and WPF UI components. The intention is more obvious for working with async EventHandlers because of the syntax when hooking up the handler to the event. Code drift that eventually throws more exceptions will still be wrapped inside the try/catch Cons: You still need to remember to hook this thing up! Closing Thoughts On Async EventHandlers While originally I set out to explore , the reader feedback was valid in that the examples focused on async EventHandlers, and there surely must be a more simple way. interesting ways to deal with async void In this article, we explored what I might argue is the most simple way to make your async EventHandlers behave properly, and the refined solution (in my opinion) only has the drawback that you need to remember to use it. A commenter had suggested that one could explore (AoP) to inject this sort of behavior across your application so that you wouldn’t need to go remember to do it. Aspect Oriented Programming There are some compile-time AoP frameworks that exist, but I’ll leave that as an exercise for you as the reader (because it’s also an exercise for me to go follow up on).

A Simple Safety Net For Async EventHandlers

Async ইভেন্টহ্যান্ডলারদের জন্য একটি সহজ নিরাপত্তা নেট

Ein einfaches Sicherheitsnetz für asynchrone EventHandler

Una red de seguridad simple para controladores de eventos asíncronos

Un filet de sécurité simple pour les gestionnaires d'événements asynchrones

Async EventHandlers के लिए एक साधारण सुरक्षा जाल

非同期 EventHandler の単純なセーフティ ネット

비동기 이벤트 핸들러를 위한 간단한 안전망

Uma rede de segurança simples para manipuladores de eventos assíncronos

Eşzamansız EventHandler'lar İçin Basit Bir Güvenlik Ağı

Mạng lưới an toàn đơn giản cho Trình xử lý sự kiện không đồng bộ

异步事件处理程序的简单安全网

What comes to a Python developer’s mind when he needs to write an I/O bound application? Of course . Why? Usually, because it’s a trend. And I’ve seen this case many times. But is this always a good idea? async Let’s consider a case when we’re writing a core I/O bound component but don’t need . async Let’s suppose we have several data sources, where data is represented by JSON messages. We have Kafka as a message broker, so data sources write messages to corresponding topics. And for each topic, we need a message processor to consume a message, find related records in reference tables, and save the message to the table with related foreign keys. It’s a typical case. So what do we do next? We think: “Hm, I need an I/O bound app, so I take ”. We don’t figure out the subject area, we don’t figure out requirements. We just take by default thinking that it’s optimal. But it’s not always optimal. Let’s try to take a better look. messages async async Let’s suppose we have PostgreSQL (and actually, on the real project we had that one), so we’ll need a proper tool for it. What choices do we have? The most popular and reliable are SQLAlchemy and Django ORM. If we need a powerful library, we choose SQLAlchemy. SQLAclehemy supports , but in a beta state. So we get a beta library in a core component in production. it doesn’t seem right. async If we take Django ORM, we get Django, less featured ORM, and support of by adapters. Not impressive either. async In our project, we didn’t have a high load and we didn’t need to process a big amount of messages per second. Moreover, sequential message processing was a strict requirement dictated by the subject area, and in this case, the asynchronous design is a fundamental mistake. So what do we have eventually? Wrong design that could cause serious bugs, “beta library” in production in a core component, and higher code complexity. I don’t think somebody would argue, that code is much simpler than . sync async But there are cases without such restrictions and with a higher load. What then? Is a good choice in these cases? Not always. could reduce the cost of infrastructure due to the better utilization of CPU time but it also increases the complexity of code, which in turn increases the cost of development and maintenance of a program and can cause more bugs. If the load is not too big and the system doesn’t need to be scaled so much, the company might take increasing the number of CPU cores as a solution. And that would be optimal in my opinion async async

When You Shouldn't Use Async in Python

Golang is a concurrent programming language. It has powerful features like and that can handle asynchronous tasks very well. Also, goroutines are not OS threads, and that's why you can spin up as many goroutines as you want without much overhead, it's stack size starts at only. So why ? Async/Await is a nice language feature that provides a simpler interface to asynchronous programming. Goroutines Channels 2KB async/await Project Link: https://github.com/Joker666/AsyncGoDemo How Does it Work? Started with F# and then C#, now in Python and Javascript, async/await is an extremely popular feature of a language. It simplifies the asynchronous method execution structure and, it reads like synchronous code. So much easier to follow for developers. Let's see a simple example in C# how async/await works Task Main(string[] args)
{
    Console.WriteLine( ); done = DoneAsync();
    Console.WriteLine( );
    Console.WriteLine( done);
} Task<int> DoneAsync()
{
    Console.WriteLine( ); Task.Delay( );
    Console.WriteLine( ); ;
} static async "Let's start ..." var "Done is running ..." await static async "Warming up ..." await 3000 "Done ..." return 1 We have the function that would be executed when the program is run. We have which is an async function. We stop the execution of the code with function for 3 seconds. Delay is an async function itself, so we call it with await. Main DoneAsync Delay await only blocks the code execution within the async function In the main function, we do not call with await. But the execution starts for . Only when we await it, we get the result back. The execution flow looks like this DoneAsync DoneAsync Let 's start ...
Warming up ...
Done is running ...
Done ...
1 This looks incredibly simple for asynchronous execution. Let's see how we can do it with Golang using Goroutines and Channels {
	r := ( )
	fmt.Println( ) {
		time.Sleep( * time.Second)
		r <- fmt.Println( )
	}() r
} {
	fmt.Println( )
	val := DoneAsync()
	fmt.Println( )
	fmt.Println(<- val)
} func DoneAsync () chan int make chan int "Warming up ..." go func () 3 1 "Done ..." return func main () "Let's start ..." "Done is running ..." Here, runs asynchronously and returns a channel. It writes a value to the channel once it's done executing the async task. In function, we invoke and keep doing our operations and then we read the value from the returned channel. It is a blocking call that waits till the value is written to the channel and after it gets the value it writes to the console. DoneAsync main DoneAsync Let 's start ...
Warming up ...
Done is running ...
Done ...
1 We see, we achieve the same outcome as the C# program but it doesn't look as elegant as async/await. While this is actually good, we are able to do a lot more granular things with this approach much easily, we can also implement async/await keywords in Golang with a simple struct and interface. Let's try that. Implementing Async/Await The full code is available in the project link. To implement async/await in Golang, we will start with a package directory named . The project structure looks like async .
├── async
│   └── async ├── main └── README.md .go .go In the async file, we write the simplest future interface that can handle async tasks. async Future {
	Await() {}
} future {
	await {}
} {} { f.await(context.Background())
} {}) Future { result {}
	c := ( {}) { (c)
		result = f()
	}() future{
		await: {} { { <-ctx.Done(): ctx.Err() <-c: result
			}
		},
	}
} package import "context" // Future interface has the method signature for await type interface interface type struct func (ctx context.Context) interface func (f future) Await () interface return // Exec executes the async function func Exec (f () func interface var interface make chan struct go func () defer close return func (ctx context.Context) interface select case return case return Not a lot is happening here, we add a interface that has the method signature. Next, we add a struct that holds one value, a function signature of the function. Now struct implements interface's Await method by invoking its own function. Future Await future await futute Future await Next in the function, we execute the passed function asynchronously in goroutine. And we return the function. It waits for the channel to close or context to read from. Based on whichever happens first, it either returns the error or the result which is an interface. Exec await Now armed with this new async package, let's see how we can change our current go code {
	fmt.Println( )
	time.Sleep( * time.Second)
	fmt.Println( ) } {
	fmt.Println( )
	future := async.Exec( {} { DoneAsync()
	})
	fmt.Println( )
	val := future.Await()
	fmt.Println(val)
} func DoneAsync () int "Warming up ..." 3 "Done ..." return 1 func main () "Let's start ..." func () interface return "Done is running ..." At the first glance, it looks much cleaner, we are not explicitly working with goroutine or channels here. Our function has been changed to a completely synchronous nature. In the main function, we use the package's method to handle . Which starts the execution of . The control flow is returned back to function which can execute other pieces of code. Finally, we make blocking call to and read back data. DoneAsync async Exec DoneAsync DoneAsync main Await Now the code looks much simpler and easier to read. We can modify our async package to incorporate a lot of other types of asynchronous tasks in Golang, but we would just stick to simple implementation for now in this tutorial. Conclusion We have gone through what async/await it and implemented a simple version of that in Golang. I would encourage you to look into async/await a lot more and see how it can ease the readability of the codebase much better.

Async/Await in Golang: An Introductory Guide

Async/Await en Golang: una guía introductoria

The text you are about to read describes an imaginary dialog between two programmers on their way to discover the of a library called . orgastic pleasures asyncFn asyncFn provides additional methods to eg. or to introduce “ ” for the promises returned by mock functions. This simplifies async unit testing by allowing tests that read chronologically, . jest.fn sinon.spy late resolve like a story From this, we get a of structuring and writing unit tests. new perspective Foreword Be warned that we’ll also cover topics such as , , , and “ ”. They are not the main topic here, but some words are spared for them nonetheless to justify . TDD pair programming evil pairing negation testing a way of thinking Best of luck, . Dear Reader Chapter 1: The premise and the foundation : My name is Fry, and I find it difficult to unit test async-stuff in JavaScript. Fry : Tell me more. Leela : Yup, say I wanted to implement something to this specification: Fry : As a player, I can play a game called Monster Beatdown : I can damage an encountered monster I encounter a monster I choose to attack it the monster loses hit points : I can try flee a monster I encounter a monster I choose to flee the monster eats me I lose : I can win the game by beating a monster until it is knocked out I encounter a monster I attack it until it has no hit points the monster is knocked out I win Feature Scenario Given And Then Scenario Given And Then And Scenario Given When Then And : The kicker here is that in this game, prompting the player for action is . Fry asynchronous : Mm-hmm, I think I get it. Let’s get our hands dirty and see where it lands us. Leela : Here’s an initial test for good measure: Fry gameWithoutDependencies ;

it( , () => { messagePlayerMock = jest.fn(); game = gameWithoutDependencies({ : messagePlayerMock,
  });

  game.encounterMonster();

  expect(messagePlayerMock).toHaveBeenCalledWith( ,
  );
}); import from './monsterBeatdown' 'when a monster is encountered, informs the player of this' const const messagePlayer 'You encounter a monster with 5 hit-points.' Contents of so far: ./monsterBeatdown.js ({ messagePlayer }) => ({ : {
    messagePlayer( );
  },
}); export default encounterMonster => () `You encounter a monster with 5 hit-points.` : Ok, so far so good. I see you chose to the mock-function for messaging the player as an argument for the function we are testing. Crystal. Full steam ahead. Leela inject : Right on. Next up will be my vision for asking the player if they want to attack, and if not, they lose. Fry Chapter 2: The problems emerge gameWithoutDependencies ;

it( , () => { messagePlayerMock = jest.fn(); game = gameWithoutDependencies({ : messagePlayerMock,
  }); game.encounterMonster();

  expect(messagePlayerMock).toHaveBeenCalledWith( ,
  );
});

it( , () => { askPlayerToHitMock = jest.fn(); game = gameWithoutDependencies({ : askPlayerToHitMock,
  }); game.encounterMonster();

  expect(askPlayerToHitMock).toHaveBeenCalledWith( );
});

it( , () => { askPlayerToHitMock = jest.fn( .resolve( )); messagePlayerMock = jest.fn(); game = gameWithoutDependencies({ : askPlayerToHitMock, : messagePlayerMock,
  }); game.encounterMonster();

  expect(messagePlayerMock).toHaveBeenCalledWith( ,
  );
});

it( , () => { askPlayerToHitMock = jest.fn( .resolve( )); messagePlayerMock = jest.fn(); game = gameWithoutDependencies({ : askPlayerToHitMock, : messagePlayerMock,
  }); game.encounterMonster();

  expect(messagePlayerMock).toHaveBeenCalledWith( );
}); import from './monsterBeatdown' 'when a monster is encountered, informs the player of this' async const const messagePlayer await 'You encounter a monster with 5 hit-points.' 'when a monster is encountered, asks player to attack' async const const askPlayerToHit await 'Do you want to attack it?' 'given a monster is encountered, when player chooses to flee, the monster eats the player' async // When asked to attack, choosing false means to flee. const => () Promise false const const askPlayerToHit messagePlayer await 'You chose to flee the monster, but the monster eats you in disappointment.' 'given a monster is encountered, when player chooses to flee, the game is over' async const => () Promise false const const askPlayerToHit messagePlayer await 'Game over.' Contents of so far: ./monsterBeatdown.js ({ messagePlayer = {}, askPlayerToHit = {} }) => ({ : { messagePlayer( );

    messagePlayer( );

    askPlayerToHit( );

    messagePlayer( ,
    );
  },
}); export default => () => () encounterMonster => () // For real life, the implementation below makes little sense. // This is intentional, and is so to prove a point later. // At this point, it suffices that all tests are green 'Game over.' 'You encounter a monster with 5 hit-points.' 'Do you want to attack it?' 'You chose to flee the monster, but the monster eats you in disappointment.' : Hold the phone. I see two problems here. One is the , and the other one is about the test describing occurrences in , making the test harder to read. Let’s start by fixing the first problem Leela duplication non-chronological order . Chapter 3: The clumsy fix : Uh, I was just about to remove the duplication anyway. , right? Fry Red-green-refactor : Right. Leela gameWithoutDependencies ;

describe( , () => { askPlayerToHitMock; messagePlayerMock; game;

  beforeEach( {
    askPlayerToHitMock = jest.fn();
    messagePlayerMock = jest.fn();

    game = gameWithoutDependencies({ : askPlayerToHitMock, : messagePlayerMock,
    });
  });

  it( , () => { game.encounterMonster();

    expect(messagePlayerMock).toHaveBeenCalledWith( ,
    );
  });

  it( , () => { game.encounterMonster();

    expect(askPlayerToHitMock).toHaveBeenCalledWith( ,
    );
  });

  describe( , () => {
    beforeEach( () => { askPlayerToHitMock.mockResolvedValue( ); game.encounterMonster();
    });

    it( , () => {
      expect(messagePlayerMock).toHaveBeenCalledWith( ,
      );
    });

    it( , () => {
      expect(messagePlayerMock).toHaveBeenCalledWith( );
    });
  });
}); import from './monsterBeatdown' 'given a monster is encountered' let let let => () askPlayerToHit messagePlayer 'informs the player of this' async await 'You encounter a monster with 5 hit-points.' 'asks player to attack' async await 'Do you want to attack it?' 'when player chooses to flee' async // When asked to attack, choosing false means to flee. false await 'player is informed of the grim outcome' 'You chose to flee the monster, but the monster eats you in disappointment.' 'the game is over' 'Game over.' Contents of so far: ./monsterBeatdown.js // no changes, as only tests were refactored. : There. Now the duplication for the test setup has been removed, but only to a degree. However, the other problem of test not being written in chronological order prevents us from removing all the duplication. See how we are forced to repeat because needs to know how to behave before it is called. Worse yet, this makes the “describe” dishonest, as it claims to display how " ", but in reality, this is something that happens only later in the test setup, if ever. All this kind of bums me out, and I’ve felt the pain of this getting out of hand as in real-life requirements and features start to pile up. I know many ways to ease the pain a little bit, but nothing I’ve tried has left me comfortable. Fry game.encounterMonster() askPlayerToHitMock given a monster is encountered : I see. Let’s now introduce as an expansion to the normal to tackle all. Behold. Leela asyncFn jest.fn Chapter 4: The beholding of asyncFn gameWithoutDependencies ; asyncFn ;

describe( , () => { askPlayerToHitMock; messagePlayerMock; game;

  beforeEach( { askPlayerToHitMock = asyncFn();
    messagePlayerMock = jest.fn();

    game = gameWithoutDependencies({ : askPlayerToHitMock, : messagePlayerMock,
    });

    game.encounterMonster();
  });

  it( , () => {
    expect(messagePlayerMock).toHaveBeenCalledWith( ,
    );
  });

  it( , () => { expect(askPlayerToHitMock).toHaveBeenCalledWith( ,
    );
  });

  describe( , () => {
    beforeEach( () => { askPlayerToHitMock.resolve( );
    });

    it( , () => {
      expect(messagePlayerMock).toHaveBeenCalledWith( ,
      );
    });

    it( , () => {
      expect(messagePlayerMock).toHaveBeenCalledWith( );
    });
  });
}); import from './monsterBeatdown' import from '@asyncFn/jest' 'given a monster is encountered' let let let => () // Note: Before we used jest.fn here instead of asyncFn. askPlayerToHit messagePlayer 'informs the player of this' async 'You encounter a monster with 5 hit-points.' 'asks player to attack' // Note how even if askPlayerToHitMock is now a mock made using asyncFn(), it still is a jest.fn(). This means eg. toHaveBeenCalledWith can be used with it. 'Do you want to attack it?' 'when player chooses to flee' async // Note how choosing false here still means fleeing combat, but here we use asyncFn's .resolve() to control the outcome. And then we await for the consequences of that. await false 'player is informed of the grim outcome' 'You chose to flee the monster, but the monster eats you in disappointment.' 'the game is over' 'Game over.' Contents of so far: ./monsterBeatdown.js // no changes, as only tests were refactored. : Now the duplication is gone, and everything takes place in clean, chronological order. It kind of reads like a story, don’t you think? Leela : Mm-hmm. I see it. I have a good feeling about it. But something is bothering me with the production code. I see that the tests are all green, but clearly, the code does not do anything sensical. It just blows through, merely satisfying the tests. Fry : You got me there. We are, in fact doing something called “evil pairing”. If you want to mold the production code your way, you need to order it from by writing a test first. Let me show you how. Leela the universe very specific Chapter 5: Evil is good and negative is positive gameWithoutDependencies ; asyncFn ;

describe( , () => { askPlayerToHitMock; messagePlayerMock; game;

  beforeEach( {
    askPlayerToHitMock = asyncFn();
    messagePlayerMock = jest.fn();

    game = gameWithoutDependencies({ : askPlayerToHitMock, : messagePlayerMock,
    });

    game.encounterMonster();
  });

  it( , () => {
    expect(messagePlayerMock).toHaveBeenCalledWith( ,
    );
  });

  it( , () => {
    expect(askPlayerToHitMock).toHaveBeenCalledWith( ,
    );
  }); it( , () => {
    expect(messagePlayerMock).not.toHaveBeenCalledWith( ,
    );
  }); it( , () => {
    expect(messagePlayerMock).not.toHaveBeenCalledWith( );
  });

  describe( , () => {
    beforeEach( () => { askPlayerToHitMock.resolve( );
    });

    it( , () => {
      expect(messagePlayerMock).toHaveBeenCalledWith( ,
      );
    });

    it( , () => {
      expect(messagePlayerMock).toHaveBeenCalledWith( );
    });
  });

  describe( , () => {
    beforeEach( () => { askPlayerToHitMock.resolve( );
    }); it( , () => {
      expect(messagePlayerMock).not.toHaveBeenCalledWith( ,
      );
    });

    it( , () => {
      expect(messagePlayerMock).toHaveBeenCalledWith( );
    });
  });
}); import from './monsterBeatdown' import from '@asyncFn/jest' 'given a monster is encountered' let let let => () askPlayerToHit messagePlayer 'informs the player of this' async 'You encounter a monster with 5 hit-points.' 'asks player to attack' 'Do you want to attack it?' // This is the "negation test" referred later in narrative. 'when player has not chosen anything yet, the player is not eaten' 'You chose to flee the monster, but the monster eats you in disappointment.' // This is another negation test. 'when player has not chosen anything yet, the game is not over' 'Game over.' 'when player chooses to flee' async await false 'player is informed of the grim outcome' 'You chose to flee the monster, but the monster eats you in disappointment.' 'the game is over' 'Game over.' 'when player chooses to attack' async await true // Another negation test 'player does not lose' 'You chose to flee the monster, but the monster eats you in disappointment.' 'the game is over' 'Game over.' Contents of so far: ./monsterBeatdown.js ({ messagePlayer, askPlayerToHit }) => ({ : () => {
    messagePlayer( ); playerChoseToAttack = askPlayerToHit( ,
    ); (!playerChoseToAttack) {
      messagePlayer( ,
      );
    }

    messagePlayer( );
  },
}); export default encounterMonster async 'You encounter a monster with 5 hit-points.' const await 'Do you want to attack it?' if 'You chose to flee the monster, but the monster eats you in disappointment.' 'Game over.' : There. I’ve reduced the “evilness” of the code by adding something we call “negation tests”. By writing tests that describe what is not supposed to happen we forced the production code to make a little bit more sense. And if you are wondering where the name “evil pairing” comes from, it comes from playing unit testing ping-pong in such a way that the production code is always written in the most evil way possible, ie. the code is non-sensical for real-life, yet it still satisfies the tests. The only way out of this is to force sense in production through unit tests. Sometimes you can tell coders are evil pairing from the . In general, this “ ” helps produce code that is very robust for the sake of refactoring, and also allows programmers to hone their a little bit more. But I digress. However important this may be, it is slightly off-course. What is relevant is that supports evil pairing as a line of thinking. Let’s motor on. Leela evil laughter evil pairing -mentality TDD-mojo asyncFn Chapter 6: The multiplicity challenge and the enlightenment : For sure. Can we see how the game develops some steps further? Particularly, I’ve been suffering from testing of functions that are called , yet return promises. Fry multiple times : I feel you. My guess is an asyncFn-mock will get called multiple times soon enough, when we start attacking the monster multiple times. Let’s see how that pans out. Leela gameWithoutDependencies ; asyncFn ;

describe( , () => { askPlayerToHitMock; messagePlayerMock; game;

  beforeEach( {
    askPlayerToHitMock = asyncFn();
    messagePlayerMock = jest.fn();

    game = gameWithoutDependencies({ : askPlayerToHitMock, : messagePlayerMock,
    });

    game.encounterMonster();
  });

  it( , () => {
    expect(messagePlayerMock).toHaveBeenCalledWith( ,
    );
  });

  it( , () => {
    expect(askPlayerToHitMock).toHaveBeenCalledWith( ,
    );
  });

  it( , () => {
    expect(messagePlayerMock).not.toHaveBeenCalledWith( );
  });

  describe( , () => {
    beforeEach( () => { askPlayerToHitMock.resolve( );
    });

    it( , () => {
      expect(messagePlayerMock).toHaveBeenCalledWith( ,
      );
    });

    it( , () => {
      expect(messagePlayerMock).not.toHaveBeenCalledWith( ,
      );
    });

    it( , () => {
      expect(messagePlayerMock).toHaveBeenCalledWith( );
    });
  });

  it( , () => { askPlayerToHitMock.resolve( );

    expect(messagePlayerMock).toHaveBeenCalledWith( ,
    );
  });

  describe( , () => {
    beforeEach( () => { askPlayerToHitMock.resolve( ); askPlayerToHitMock.resolve( ); askPlayerToHitMock.resolve( ); askPlayerToHitMock.resolve( ); askPlayerToHitMock.resolve( );
    });

    it( , () => {
      expect(messagePlayerMock).not.toHaveBeenCalledWith( ,
      );
    });

    it( , () => {
      expect(messagePlayerMock).toHaveBeenCalledWith( ,
      );
    });

    it( , () => {
      expect(messagePlayerMock).toHaveBeenCalledWith( );
    });

    it( , () => {
      expect(messagePlayerMock).not.toHaveBeenCalledWith( ,
      );
    });
  });

  it( , () => { askPlayerToHitMock.resolve( ); askPlayerToHitMock.resolve( ); askPlayerToHitMock.resolve( ); askPlayerToHitMock.resolve( );

    expect(messagePlayerMock).not.toHaveBeenCalledWith( );
  });
}); import from './monsterBeatdown' import from '@asyncFn/jest' 'given a monster is encountered' let let let => () askPlayerToHit messagePlayer 'informs the player of this' async 'You encounter a monster with 5 hit-points.' 'asks player to attack' 'Do you want to attack it?' 'when player has not chosen anything yet, the game is not lost' 'Game over.' 'when player chooses to flee' async await false 'player is informed of the grim outcome' 'You chose to flee the monster, but the monster eats you in disappointment.' 'the game is not won' 'You knock out the monster. You are a winner.' 'the game is over' 'Game over.' 'when player hits the monster, the monster loses a hit-point' async await true 'You hit the monster. It now has 4 hit-points remaining.' 'when player hits the monster enough times to knock it out' async // Here the monster is attacked multiple times, and therefore, askPlayerToHitMock is called multiple times, and then the test resolves the mock multiple times. // Using asyncFn, the logic remains the same: things that happen are written and observed in chronological order. await true await true await true await true await true 'does not show out-of-place message about no hit-points' 'You hit the monster. It now has 0 hit-points remaining.' 'player is congratulated' 'You knock out the monster. You are a winner.' 'game is won' 'Game over.' 'the game is not lost' 'You chose to flee the monster, but the monster eats you in disappointment.' 'when player hits the monster until it only has 1 hit-point remaining, the game is not over yet' async // Hit the monster only 4 times instead of 5. await true await true await true await true 'Game over.' The final form of : ./monsterBeatdown.js ({ askPlayerToHit, messagePlayer }) => ({ : () => { monsterHitPoints = ;

    messagePlayer( ,
    ); ( askPlayerToHit( )) {
      monsterHitPoints--; (monsterHitPoints === ) { ;
      }

      messagePlayer( ,
      );
    } (monsterHitPoints === ) {
      messagePlayer( );
    } {
      messagePlayer( ,
      );
    }

    messagePlayer( );
  },
}); export default encounterMonster async let 5 `You encounter a monster with hit-points.` ${monsterHitPoints} while await 'Do you want to attack it?' if 0 break `You hit the monster. It now has hit-points remaining.` ${monsterHitPoints} if 0 'You knock out the monster. You are a winner.' else 'You chose to flee the monster, but the monster eats you in disappointment.' 'Game over.' : There. Now the monster gets hit multiple times, all while things still happen in a clear order. Our work is done here. Leela : Color me enlightened. This has changed my view of the world as a programmer and a human being. I shall make sacrifices in your honor and get an asyncFn-tattoo. My grandchildren will know your name. Fry Summary Having witnessed , we’ve seen how the of permitted troublesome tests, with awkward readability, to be refactored orderly as a story, with significantly better readability. the adventure late resolve asyncFn is something that we, as a team, have used for multiple years now. It has categorically changed the way we approach unit testing of asynchronous code in JavaScript. Which we so adore. asyncFn See more about asyncFn for or . jest sinon Who are we? asyncFn is lovingly crafted by at from . Your Pals Team: Igniter Houston Inc. Consulting We are a software development team of friends, with proven tradition in professional excellence. We specialize in holistic without sacrificing . rapid deployments quality Come say hi at , us, or check out the . We just might be available for hiring ;) Gitter email team itself Previously published at https://medium.com/houston-io/how-to-unit-test-asynchronous-code-for-javascript-in-2020-41c124be2552

Testing Asynchronous JS Code: 2020 Edition

This article was originally posted on my blog. A few weeks ago, I was working on a Python script to extract books' metadata for a . After a couple of hours, I realized that I needed to make thousands of requests to the Google Books API to get the data. So I thought there had to be a way of speeding up the process. content-based recommender As I enjoy learning, especially when it's also a chance of procrastinating on my goals, I decided to build a project using . Afterward, feeling guilty for the time wasted, I decided to write this tutorial with what I learned in the process. asyncio This article aims to provide the basics of how to use for making asynchronous requests to an API. I focus mostly on the actual code and skip most of the theory (besides the short introduction below). However, if you are looking for a more in-depth introduction to , check the recommendations in the references[1]. asyncio asyncio in 30 Seconds or Less asyncio is a Python library that allows you to execute some tasks in a [2] manner. It is commonly used in web-servers and database connections. It is also useful for speeding up IO-bound tasks, like services that require making many requests or do lots of waiting for external APIs. asyncio seemingly concurrent The essence of is that it allows the program to continue executing other instructions while waiting for specific processes to finish (e.g., a request to an API). In this tutorial, we will see how to use for accelerating a program that makes multiple requests to an API. asyncio asyncio Sequential vs. Asynchronous So let's get down to business. To get the most out of this tutorial, try running the code yourself. You can copy-and-paste the code in a Jupyter Notebook and run it without modifications. Just remember to first install the required libraries ( and .) requests aiohttp We'll build a sequential and an asynchronous version of a small program and compare their results and structure. Both programs will do the same: Read a list of ISBNs (international identifier of books) Request the books' metadata to the Google Books API Parse the results from the requests Print the results to the screen The algorithm would look something like the diagram below. We'll now compare two possible approaches for building this algorithm. First, , which executes the requests sequentially. Then, , which uses to run requests asynchronously. Option A Option B asyncio Option A: Sequential Algorithm A sequential version of that algorithm could look as follows: os requests requests.exceptions HTTPError


GOOGLE_BOOKS_URL = LIST_ISBN = [ , , , , , , , , , ,
] volume_info = item.get( , {})
    title = volume_info.get( , )
    subtitle = volume_info.get( , )
    description = volume_info.get( , )
    published_date = volume_info.get( , ) (
        title,
        subtitle,
        description,
        published_date,
    ) url = GOOGLE_BOOKS_URL + isbn
    response = :
        response = session.get(url)
        response.raise_for_status()
        print( ) HTTPError http_err:
        print( ) Exception err:
        print( )
    response_json = response.json()
    items = response_json.get( , [{}])[ ] items requests.Session() session: isbn LIST_ISBN: :
            response = get_book_details_seq(isbn, session)
            parsed_response = extract_fields_from_response(response)
            print( ) Exception err:
            print( ) import import from import "https://www.googleapis.com/books/v1/volumes?q=isbn:" '9780002005883' '9780002238304' '9780002261982' '9780006163831' '9780006178736' '9780006280897' '9780006280934' '9780006353287' '9780006380832' '9780006470229' : def extract_fields_from_response (item) """Extract fields from API's response""" "volumeInfo" "title" None "subtitle" None "description" None "publishedDate" None return : def get_book_details_seq (isbn, session) """Get book details using Google Books API (sequentially)""" None try f"Response status ( ): " {url} {response.status_code} except as f"HTTP error occurred: " {http_err} except as f"An error ocurred: " {err} "items" 0 return with as for in try f"Response: " {json.dumps(parsed_response, indent= )} 2 except as f"Exception occured: " {err} pass Now, let's breakdown the code to understand what's going on. As usual, we start by importing the required libraries. Then, we define two variables: for specifying the URL of the Google's API we'll use for the requests. GOOGLE_BOOKS_URL which is a sample list of ISBNs for testing the program. LIST_ISBN , For illustration purposes, this how a request to the Google Books API looks like: https://www.googleapis.com/books/v1/volumes?q=isbn:9780002005883 Next, we define the function. This function takes as input the response from the API and extracts the fields we're interested in. extract_fields_from_response The parsing process in is based on the response's structure from the Google Books API, which looks as follows: extract_fields_from_response { : , : , : [
  { : , : , : , : , : { : , : ,
    ... "kind" "books#volumes" "totalItems" 1 "items" "kind" "books#volume" "id" "3Mx4QgAACAAJ" "etag" "FWJF/JY16xg" "selfLink" "https://www.googleapis.com/books/v1/volumes/3Mx4QgAACAAJ" "volumeInfo" "title" "Mapping the Big Picture" "subtitle" "Integrating Curriculum and Assessment, K-12" Finally, we go into the most relevant parts of the program: how we make requests to the Google Books API. url = GOOGLE_BOOKS_URL + isbn
    response = :
        response = session.get(url)
        response.raise_for_status()
        print( ) HTTPError http_err:
        print( ) Exception err:
        print( )
    response_json = response.json()
    items = response_json.get( , [{}])[ ] items requests.Session() session: isbn LIST_ISBN: :
            response = get_book_details_seq(isbn, session)
            parsed_response = extract_fields_from_response(response)
            print( ) Exception err:
            print( ) : def get_book_details_seq (isbn, session) """Get book details using Google Books API (sequentially)""" None try f"Response status ( ): " {url} {response.status_code} except as f"HTTP error occurred: " {http_err} except as f"An error ocurred: " {err} "items" 0 return with as for in try f"Response: " {json.dumps(parsed_response, indent= )} 2 except as f"Exception occured: " {err} pass There are two major pieces here: , which is the function that executes the requests. It takes as input an ISBN and a session object[4] and returns the response from the API as a JSON structure. It also handles possible errors, like providing a wrong URL or going over your daily quota of requests. get_book_details_seq The code block under , is where the actual execution of requests happens. It iterates through the list of ISBNs, getting the book details, parsing them, and finally printing them to the screen. with requests.Session() as session For me, . If you only need to do this a couple of times, you will not find much benefit from using . However, if instead of 10 requests, you need to do 10,000, having some concurrency in your program pays out. In the next section, we'll see how to make this algorithm faster using . executing this process takes ranges from 4 to 6 seconds asyncio asyncio Option B: Asynchronous Algorithm An asynchronous version of the same algorithm may look something as follows: aiohttp asyncio os aiohttp ClientSession


GOOGLE_BOOKS_URL = LIST_ISBN = [ , , , , , , , , , ,
] item = response.get( , [{}])[ ]
    volume_info = item.get( , {})
    title = volume_info.get( , )
    subtitle = volume_info.get( , )
    description = volume_info.get( , )
    published_date = volume_info.get( , ) (
        title,
        subtitle,
        description,
        published_date,
    ) url = GOOGLE_BOOKS_URL + isbn :
        response = session.request(method= , url=url)
        response.raise_for_status()
        print( ) HTTPError http_err:
        print( ) Exception err:
        print( )
    response_json = response.json() response_json :
        response = get_book_details_async(isbn, session)
        parsed_response = extract_fields_from_response(response)
        print( ) Exception err:
        print( ) ClientSession() session: asyncio.gather(*[run_program(isbn, session) isbn LIST_ISBN]) import import import from import "https://www.googleapis.com/books/v1/volumes?q=isbn:" '9780002005883' '9780002238304' '9780002261982' '9780006163831' '9780006178736' '9780006280897' '9780006280934' '9780006353287' '9780006380832' '9780006470229' : def extract_fields_from_response (response) """Extract fields from API's response""" "items" 0 "volumeInfo" "title" None "subtitle" None "description" None "publishedDate" None return async : def get_book_details_async (isbn, session) """Get book details using Google Books API (asynchronously)""" try await 'GET' f"Response status ( ): " {url} {response.status} except as f"HTTP error occurred: " {http_err} except as f"An error ocurred: " {err} await return async : def run_program (isbn, session) """Wrapper for running program in an asynchronous manner""" try await f"Response: " {json.dumps(parsed_response, indent= )} 2 except as f"Exception occured: " {err} pass async with as await for in First, check the function. An keyword prepends it. This keyword tells Python that your function is a . Then, in the function's body, there are two await keywords. These tell that to suspend execution and give back control to the event loop, while the operation it is awaiting finishes. get_book_details_async async coroutine coroutine A is a type of generator function in Python that, instead of producing values, consumes values[4]. The interesting thing about it is that its execution pauses while waiting for new data being sent to it. In our case, this allows the execution of other parts of the program to continue in a manner. coroutine seemingly concurrent In this case, the execution of is suspended while the request is being performed: get_book_details_async . await session.request(method='GET', url=url) It is suspended again, while the request response is being parsed into a JSON structure: . await response.json() Next, we have the coroutine. This one is simply a wrapper around the pipeline of getting a response from the API, parsing it, and printing the results in the screen. It the execution of the coroutine. run_program awaits get_book_details_async Finally, we have the code block under . Using the syntax, we tell the program to schedule all the tasks based on the list of coroutines we provided. This is what allows us to execute tasks concurrently. async with ClientSession() as session: asyncio.gather For me, running this process takes around 800-1000 milliseconds. Results Comparing both versions, we see that than the sequential version. If we increase the number of requests, you'll likely get an even higher speedup. Besides, , which makes using asyncio an excellent option for the kind of task we reviewed in the tutorial. the asynchronous one is around 4 to 7.5 times faster the version using asyncio is not much more complicated than the sequential version Additional recommendations Here are some tips I gathered while working with asyncio: keeps changing all the time, so be wary of old Stack Overflow answers. Many of them are not up to date with the current best practices asyncio APIs will not allow you to run unlimited concurrent requests. To overcome that, take a look at . It will enable you to limit the concurrency of your application. asyncio's Semaphore Not all programs can be speedup with asyncio. Research the type of issue you are facing before doing any substantial modification of code. Other alternatives might work for you (e.g., ) threading, multiprocessing I made a complete version of the program we went through in this tutorial for getting the metadata of almost 7 thousand books. Here's a link to it: . Google Books Crawler Notes and References [1] Real Python has a two of amazing articles introducing : and asyncio Async IO in Python Speed Up Your Python Program With Concurrency [2] It is not strictly concurrent execution. But in practical terms, it looks like it is. [3] S. Buczyński, (2017) What Is the use case of coroutines and asyncio in Python 3.6? [4] The session object is a functionality from the requests library that allows you to persist certain parameters across sessions. This usually results in requests with lower latency. Read more . here [5] D. Beasly, (2009) A Curious Course on Couroutines and Concurrency

Fast & Asynchronous: Accelerate Your Requests Using Python's asyncio

Recently at my workplace our IT team finally upgraded our distributed Python versions to 3.5.0. While this is a huge upgrade from 2.6, this still came with some growing pains. Unfortunately, Python 3.5.0 doesn’t meet some of the minimum requirements of some popular libraries, including . aiohttp With these restrictions I still needed to write a script that could pull hundreds of .csv files from our APIs and manipulate the data. Python by itself isn’t event-driven and natively asynchronous (like NodeJS), but the same effect can still be achieved. This article will help detail what I learned while also showing the benefits of asynchronous operations. If you have a higher version of Python available (3.5.2+), I highly recommend using aiohttp instead. It’s an incredibly robust library and a great solution for this kind of problem. There are many tutorials online detailing how best to use the library. Disclaimer: Assumptions This article makes the following assumptions: You already have familiarity with Python and most of its syntaxYou already have familiarity with basic web requestsYou have a lose concept of asynchronous operations If you’re just looking for the solution, scroll down to the bottom and the full code is posted. Enjoy! Setup Before getting started, ensure that you have installed on your machine. The easiest way to install is by typing the following command into your terminal: requests $ python -m pip install requests Alternatively, if you don’t have administrative permissions you can install the library with this command: $ python -m pip install requests --user The wrong approach: synchronous requests To demonstrate the benefits of our parallel approach, let’s first look at approaching the problem in a synchronous manner. I’ll also give an overview of what’s going on in the code. Ultimately, we want to do able to perform a GET request to the URL containing the .csv file and measure the time it takes to read the text inside. We’ll be downloading multiple .csv files of varying sizes from /, which provides plenty of data for our example. https://people.sc.fsu.edu/~jburkardt/data/csv As a disclaimer, we’ll be using the Session object from the requests library to perform our GET request. First, we’ll need a function that executes the web request:c base_url = session.get(base_url + csv) response:
        data = response.text response.status_code != :
            print( .format(url)) data : def fetch (session, csv) "https://people.sc.fsu.edu/~jburkardt/data/csv/" with as if 200 "FAILURE::{0}" # Return .csv data for future consumption return View code gist here This function takes in a object and the name of the .csv file desired, performs the web request, then returns the text inside the response. Session Next, we need a function that can efficiently loop a list of our desired files and measure the time it takes to perform the request: timeit default_timer() csvs_to_fetch = [ , , , , , , , , , , , ] requests.Session() session:
        print( .format( , )) total_start_time = default_timer() csv csvs_to_fetch:
            fetch(session, csv)
            elapsed = default_timer() - total_start_time
            time_completed_at = .format(elapsed)
            print( .format(csv, time_completed_at)) from import : def get_data_synchronous () "ford_escort.csv" "cities.csv" "hw_25000.csv" "mlb_teams_2012.csv" "nile.csv" "homes.csv" "hooke.csv" "lead_shot.csv" "news_decline.csv" "snakes_count_10000.csv" "trees.csv" "zillow.csv" with as "{0:<30} {1:>20}" "File" "Completed at" # Set any session parameters here before calling `fetch` # For instance, if you needed to set Headers or Authentication # this can be done before starting the loop for in "{:5.2f}s" "{0:<30} {1:>20}" View code gist here This function creates our object and then loops through each .csv file in the list. Once the operation is completed, the measured time is calculated and displayed in an easy-to-read format. Session csvs_to_fetch fetch Finally, our function will be simple (for now) and call our function: main get_data_synchronous()

main() : def main () # Simple for now View code gist here Once we put it all together, here is what the code looks like for our synchronous example: requests timeit default_timer base_url = session.get(base_url + csv) response:
        data = response.text response.status_code != :
            print( .format(url)) data csvs_to_fetch = [ , , , , , , , , , , , ] requests.Session() session:
        print( .format( , )) total_start_time = default_timer() csv csvs_to_fetch:
            fetch(session, csv)
            elapsed = default_timer() - total_start_time
            time_completed_at = .format(elapsed)
            print( .format(csv, time_completed_at)) get_data_synchronous()

main() import from import : def fetch (session, csv) "https://people.sc.fsu.edu/~jburkardt/data/csv/" with as if 200 "FAILURE::{0}" # Return .csv data for future consumption return : def get_data_synchronous () "ford_escort.csv" "cities.csv" "hw_25000.csv" "mlb_teams_2012.csv" "nile.csv" "homes.csv" "hooke.csv" "lead_shot.csv" "news_decline.csv" "snakes_count_10000.csv" "trees.csv" "zillow.csv" with as "{0:<30} {1:>20}" "File" "Completed at" # Set any session parameters here before calling `fetch` # For instance, if you needed to set Headers or Authentication # this can be done before starting the loop for in "{:5.2f}s" "{0:<30} {1:>20}" : def main () # Simple for now View code gist here Let’s take a look at the results when we run this script: Synchronous example. Notice how each operation doesn’t start until the last one is completed Thankfully, we can vastly improve this performance with Python 3’s built-in library! asyncio The right approach: performing multiple requests at once asynchronously In order to get this to work, we’ll have to rework some of our existing functions. Beginning with : fetch requests timeit default_timer START_TIME = default_timer() base_url = session.get(base_url + csv) response:
        data = response.text response.status_code != :
            print( .format(url)) elapsed = default_timer() - START_TIME
        time_completed_at = .format(elapsed)
        print( .format(csv, time_completed_at)) data import from import # We'll need access to this variable later : def fetch (session, csv) "https://people.sc.fsu.edu/~jburkardt/data/csv/" with as if 200 "FAILURE::{0}" # Now we will print how long it took to complete the operation from the # `fetch` function itself "{:5.2f}s" "{0:<30} {1:>20}" return View code gist here Next, we need to make our function asynchronous: get_data asyncio timeit default_timer concurrent.futures ThreadPoolExecutor csvs_to_fetch = [ , , , , , , , , , , , ]
    print( .format( , )) ThreadPoolExecutor(max_workers= ) executor: requests.Session() session: loop = asyncio.get_event_loop() START_TIME = default_timer() tasks = [
                loop.run_in_executor(
                    executor,
                    fetch,
                    *(session, csv) ) csv csvs_to_fetch
            ] response asyncio.gather(*tasks): import from import from import async : def get_data_asynchronous () "ford_escort.csv" "cities.csv" "hw_25000.csv" "mlb_teams_2012.csv" "nile.csv" "homes.csv" "hooke.csv" "lead_shot.csv" "news_decline.csv" "snakes_count_10000.csv" "trees.csv" "zillow.csv" "{0:<30} {1:>20}" "File" "Completed at" # max_workers is set to 10 simply for this example, Note: # you'll have to tweak with this number for your own projects # as you see fit with 10 as with as # Set any session parameters here before calling `fetch` # Initialize the event loop # Set the START_TIME for the `fetch` function # Use list comprehension to create a list of # tasks to complete. The executor will run the `fetch` # function for each csv in the csvs_to_fetch list # Allows us to pass in multiple arguments to `fetch` for in # Initializes the tasks to run and awaits their results for in await pass View code gist here This code will now create multiple threads for each .csv file and execute the function for each that needs to be downloaded. fetch Finally, our function needs a small tweak to properly initialize our async function: main loop = asyncio.get_event_loop()
    future = asyncio.ensure_future(get_data_asynchronous())
    loop.run_until_complete(future)

main() : def main () View code gist here Now, let’s run the new code and see the results: Asynchronous example. Notice how the files are not being obtained in order. With this small change, all 12 of these .csv files were able to be downloaded in 3.43s vs 10.84s. That is a nearly 70% decrease in the time it took to download! The Asynchronous Code requests asyncio concurrent.futures ThreadPoolExecutor timeit default_timer

START_TIME = default_timer() base_url = session.get(base_url + csv) response:
        data = response.text response.status_code != :
            print( .format(url))

        elapsed = default_timer() - START_TIME
        time_completed_at = .format(elapsed)
        print( .format(csv, time_completed_at)) data csvs_to_fetch = [ , , , , , , , , , , , ]
    print( .format( , )) ThreadPoolExecutor(max_workers= ) executor: requests.Session() session: loop = asyncio.get_event_loop()
            START_TIME = default_timer()
            tasks = [
                loop.run_in_executor(
                    executor,
                    fetch,
                    *(session, csv) ) csv csvs_to_fetch
            ] response asyncio.gather(*tasks): loop = asyncio.get_event_loop()
    future = asyncio.ensure_future(get_data_asynchronous())
    loop.run_until_complete(future)

main() import import from import from import : def fetch (session, csv) "https://people.sc.fsu.edu/~jburkardt/data/csv/" with as if 200 "FAILURE::{0}" "{:5.2f}s" "{0:<30} {1:>20}" return async : def get_data_asynchronous () "ford_escort.csv" "cities.csv" "hw_25000.csv" "mlb_teams_2012.csv" "nile.csv" "homes.csv" "hooke.csv" "lead_shot.csv" "news_decline.csv" "snakes_count_10000.csv" "trees.csv" "zillow.csv" "{0:<30} {1:>20}" "File" "Completed at" with 10 as with as # Set any session parameters here before calling `fetch` # Allows us to pass in multiple arguments to `fetch` for in for in await pass : def main () View code gist here I hope you enjoyed this article and can use these skills for any projects that require an older Python version (or maybe without as many dependencies). Although Python may not have a straightforward to an async / await pattern, it isn’t difficult at all to achieve fantastic results. Enjoy!

How to run asynchronous web requests in parallel with Python 3.5 (without aiohttp)

Stories, News & Mentions about ASYNC

Categories

Trending Topics

Stories, News & Mentions about ASYNC

Async’s Stories on HackerNoon

Categories

Trending Topics