Everything You Need To Know About
Async & Meteor

Meteor runs on top of Node.js. This means I/O activities such as reading a file or sending a request over the network won’t block the whole program. Instead, we provide callbacks that will be executed in the Event Loop later when those activities finish. Ok, I agree, that may not make a lot of sense. How about some cartoons!

Let’s say our task is to read an encrypted file, then decrypt it and get the secret content:

 

Here is what a generic, garden-variety Event Loop looks like:

meteor_async_a

The Event Loop is just a queue of functions waiting to be executed. Every time we call a function, it is put onto the Event Loop

 

When we execute getSecretData to decrypt and print out the secret, the function readFile get called and appear on the Event Loop:

meteor_async_b

That readFile guy doesn’t care about what happens later at all, he just tells the OS to send the file and then go away!

 

Some moment later, the readFile operation is completed. A guy with the name ‘callback’ will jump into the Event Loop:

meteor_async_c

Later, when the file is received, our hero appears and finishes off the job

 

That’s quite nice and get the job done. But what if our task is more sophisticated and require many level of async operations? We might end up with something like this:

callback-hell

so-many-callbacks

The problem of async control flow is it makes the code more difficult to read and maintain. It would be nicer if we can have getSecretData return the secret content and print it out synchronously, like this:

This code does not work, because getSecretData will not wait for the readFile operation to finish. It just goes on and return decryptedSecret as an undefined value. To solve this problem, we need a new hero. Here comes Fiber!

meteor_async_f

Meet Fiber, the big guy who can carry many functions inside him

 

A Fiber is a special container function. He can be put into the Event Loop like other normal functions. But Fiber has a special power: He can halt at any point in the middle of his execution, get out of the Event Loop to take a rest, then come back at any time, all at his will (or in fact, the developer’s will). When a Fiber halts, control will be passed to the next function in the Event Loop (which may be a normal function or yet another Fiber).

You properly already see the advantage here: If our Fiber contains a function that performs a time-consuming I/O activity, he can just get out of the Event Loop and wait for the result. In the mean time, we can go on and run the next functions waiting in the queue. Life is short and time is precious! When the I/O activity finishes, our Fiber can jump back in again and resume what he was doing the last time.

Here is our code, Fiber-powered:

Alright, that may not make a lot of sense. Here are your cartoons:

meteor_async_d

When Fiber encounters a yield, he knows it’s time to take a rest!

meteor_async_e

Calling run() will signal the resuming of execution for this Fiber. Whatever passed to run() will become the value returned by yield()

 

I hear you saying: ‘Ok, that’s looks good. But the yield and run stuffs still sound weird to me’.
I got you. We’ll see something even nicer than a Fiber. It’s a Future!

You can see Future as an abstraction layer on top of Fiber. This gives us the power of Fiber with a nicer API. Like a well-groomed Fiber.

Wait! In the examples above, we have freely modified our getSecretData function. What if you come across an async function that you can’t modify (like functions from external APIs)? No worry, instead of modifying it, we can wrap it up!

Hmm, looks like we’ll need to remember to call wait() every time. What a hassle!
Fortunately, it’s even simpler if we use Meteor.wrapAsync:

I never know


Actually there’s more to async than meets the eye. Other useful things that are worth mentioning:

– – –

Future.wrap and Meteor.wrapAsync are very selective

They only do business with native, pure async functions. That is, functions that expect a callback with error and result as arguments. Also, they only work on the server-side (since yielding is not possible on the client – there’re no Fibers living there).

– – –

Meteor.wrapAsync will turn your innocent function into Two-Face !!!

two-face

Fortunately, two-faced functions are not as destructive as Harvey Dent. In fact, they’re pretty useful: They can be called synchronously (like what we were doing above) or asynchronously (with a callback passed to them).

On server-side, methods such as HTTP.call and collection.insert/update/remove are all already pre-wrapped this way. Take HTTP.call for example: If you call it without a callback, the method will block until the response is received. If called with a callback, HTTP.call returns immediately, and will later excute the provided callback when network response has arrived.

On client-side, since blocking/yielding is not possible, we always have to provide a callback to these methods.

– – –

Fiber hiccups

By default, method calls from a client are run inside a single Fiber – they’re run one at a time. This Fiber gets access to a set of environment variables that are specific to the currently connected client (e.g. Meteor.userId()). This may result in two common problems:

1) On server-side, calling methods like HTTP.call synchronously will block other subsequent method calls from the current client. This may not be a good thing. If subsequent methods are independent from the current running method, we can save time by using this.unblock(), which will allow other method calls to be run in a new Fiber:

 

2) “Meteor code must always run within a Fiber”

Looks familiar? This error often occurs when you try to call a third-party API with async callback. You’re not allowed to do this, since the callback function would be executed outside Fiber, without access to necessary environment variables. One way to solve this is wrapping the callback function with Meteor.bindEnvironment, which will return a Fiber-wrapped and environment-packed version of the function. The second way is using Meteor.wrapAsync like what we were doing above (actually wrapAsync already called bindEnvironment internally for us!).


 

I hope you’ve learned something useful about async and Meteor in this article. Happy coding!

 

P.S. For anyone wanting to learn more about this topic, here are two interesting classes from EventedMind:

Meteor Fibers and Dynamics

The JavaScript Runtime

25 Comments Everything You Need To Know About
Async & Meteor

  1. aaron

    fantastic! this post just saved me hours of time!

    This one line: “They can be called synchronously (like what we were doing above) or asynchronously (with a callback passed to them).”

    Is what clear up the confusion for me!

    Thanks again!

    Reply
  2. Pingback: Everything You Need To Know About Async & Meteor | Dinesh Ram Kali.

  3. Pingback: Best Learning Resources for Meteor.js | 懒得折腾

  4. Pingback: Meteor-Learning | 懒得折腾

  5. dddbseyhseuuhb

    są pierwsze wyimaginować sumę stwórz matrymonialnych, wespół z poufnymi, skoro mają że awansowanie się na nie
    da im pewną przewaga. Istnieją niewyobrażalnie perfidne
    natomiast skoro faktycznie toż potrafię zachwycić – bezwyjątkowe.
    W 4 casusach na 5 swobodnie porwane użyciem podkreślaj do skoncentrowanego posagu.

    Reply
  6. Pingback: Meteor.js异步全解 | Meteor中国

  7. MaikelH

    phucnguyen.info has potential, you can make your blog go viral easily using one tricky method. Just search in google:
    Sulingi’s Method To Go Viral

    Reply
  8. Pingback: List of resources to learn to build amazing apps with JavaScript – Meteor |

  9. Pingback: Synchronous call using wrapAsync - HTML CODE

  10. Pingback: How to learn meteor | About Resarch

    1. Jobeth

      HiI have connected my Mac to a Linux box directly using a cable (no home router involved).In Mac, I am setting a static IP address to 10.10.11.13 with subnet mask 255.255.255.0 and in Linux 10.10.11.16 with same mask. My Mac has two NICs (2nd one is connected to router and WWW). How can I add a route in Mac so all the traffic intended for the Linux box goes through first NIC? I am interested in waking the Linux machine up using something like:$ wakeonlan -i 10.10.11.16 00BA::AB:XX:XXIs this routing possible?

      Reply

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">