Infinite Scrolling In Firebase

This article will demonstrate you how to implement infinite scrolling in JavaScript and firebase without having to modify your existing firebase data structure.

I tried to make this article as general and applicable to any framework / library of your choice as possible, so hopefully it gives you a good sense of the steps that you need to go through in order to implement it in your own projects.

Note: If you don’t want to follow along and are anxious to see the code you can skip to the bottom of the article.

Outline

Before we begin, let’s define the expected behaviour:

• Initial fetch: retrieve the 5 newest items
• Subsequent fetches: retrieve the next 5 newest items

Let’s also define the test data that we’ll pretend to be fetching from firebase — using fake data will make it easier to visualise what’s going on with the queries:

// our firebase databaseitems: {firstInsertedItem: { … }, // oldestSecondInsertedItem: { … },ThirdInsertedItem: { … },FourthInsertedItem: { … },FifthInsertedItem: { … },SixthInsertedItem: { … },SeventhInsertedItem: { … },EighthInsertedItem: { … },NinethInsertedItem: { … },TenthInsertedItem: { … }, // newest}

// in case you were wondering about the order that I put the items // in, that's just to make it easier to follow along// and it would make no difference to us if firebase actually// stored those items in a different order internally

Firebase push keys

Let’s begin by introducing the magic behind firebase push keys and how they will make all of this possible.

It turns out that firebase push keys aren’t just some random sequence of characters. In fact all firebase push keys consist of a combination of timestamp and random data encoded in a modified base64 alphabet to preserve their chronological order — i.e. their insertion order.

This is a powerful feature as it will allow us to make use of firebase’s sorting queries.

Now that we defined this important characteristic we can get on with the tutorial.

First step

In order for all of our fetches, after the very first one, to return what we expect them to return we will need to keep some sort of a reference to the oldest previously fetched key.

So let’s define a variable called:

let referenceToOldestKey = ‘’;

Initial fetch

Next let’s create a query for fetching our first 5 newest inserted items:

firebase.database().ref(‘items’).orderByKey().limitToLast(5).once(‘value’).then((snapshot) => { … } ).catch((error) => { … } );

This query does the following:

- orderByKey sorts the items by their keys in chronological order (oldest to newest)- limitToLast selects 5 items from the end (starting with the 5th)

The returned object will look like this:

{SixthItemFromTheEnd: { … },SeventhItemFromTheEnd: { … },EighthInsertedItem: { … },NinethInsertedItem: { … },TenthInsertedItem: { … },}

Next we need to reverse the order of this object so that the latest item is on top rather than on the bottom.

We have several options here, depending on how you like to manage your applications state:

• Reverse the object
• Transform the object into an array of objects and reverse the array
• Keep the object as is but make a shallow copy and reverse on the spot when displaying to the user

We’ll go with the second option, due to to it being the most popular way of storing this sort of data.

So let’s add the following to .then of our fetch function:

let arrayOfKeys = Object.keys(snapshot.val()).sort().reverse();

let results = arrayOfKeys.map((key) => snapshot.val()[key]);

Note that we’re sorting the keys before reversing their order — this is done to ensure the right order, since JavaScript does not guarantee object key order.

Next, we need to initialise our reference with the oldest out of the 5 keys so that when the user scrolls to the bottom of the page our next fetch function knows where to carry on from:

referenceToOldestKey = arrayOfKeys[arrayOfKeys.length-1];

(The oldest key is in last position because remember that limitToLast returns items in chronological order and we reversed it.)

And, now that we’re finished with the first fetch:

// You can do what you want to do with the data, i.e.// append to page or dispatch({ … }) if using redux

Next fetch

Ok, our user has just hit the bottom of the page let’s fetch the next 5 newest items for them by creating the following query:

firebase.database().ref(‘items’).orderByKey().endAt(referenceToOldestKey).limitToLast(6).once(‘value’).then((snapshot) => { … } ).catch((error) => { … } );

This query does the following:

• orderByKey sorts the items by their keys in chronological order (oldest to newest)
• endAt selects all items from the 1st one ever added up to our reference variable (inclusive)
• limitToLast selects 6 items from the end (starting with the 6th)

The returned object will look like this:

{firstInsertedItem: { … },SecondInsertedItem: { … },ThirdInsertedItem: { … },FourthInsertedItem: { … },FifthInsertedItem: { … },SixthInsertedItem: { … }, // our reference is included!}

• But why are we limiting to the 6 latest items rather than 5, like we did in the first fetch?

Because endAt is inclusive which means that our reference key gets included in the returned object. And so if we limit to 5 we would end up with only 4 new items as 1 would be a duplicate, therefore we need to request for 6 and then handle the removal of the 6th one on the client side.

Ok, now that we see how the query works let’s reverse the returned object and then remove the duplicate.

let arrayOfKeys = Object.keys(snapshot.val()).sort().reverse().slice(1);

let results = arrayOfKeys.map((key) => snapshot.val()[key]);

Why slice(1)? Because after reverse() our duplicate moved from last position into the first one. How do we know it was in last position to begin with? Because remember that the keys were returned in chronological order.

Lastly, we need to update our reference with the oldest key from the current fetch:

referenceToOldestKey = arrayOfKeys[arrayOfKeys.length-1];

And, now that we’re finished with the second fetch:

// You can do what you want to do with the data, i.e.// append to page or dispatch({ … }) if using redux

And this concludes the tutorial, thank you for reading and hopefully you learned something new. Look below to see the full code sample.

The complete code sample

Note: code duplication inside .then of both queries is intentional to make it easier to read, no need to look all over the place to find the relevant code, as would be required if the code was moved out into a separate function.

let referenceToOldestKey = ‘’;

if (!referenceToOldestKey) { // if initial fetch

firebase.database().ref(‘items’).orderByKey().limitToLast(5).once(‘value’).then((snapshot) => {

  // changing to reverse chronological order (latest first)  
  let arrayOfKeys = Object.keys(snapshot.val())  
     .sort()  
     .reverse();

  // transforming to array  
  let results = arrayOfKeys  
     .map((key) => snapshot.val()\[key\]);

  // storing reference  
  referenceToOldestKey = arrayOfKeys\[arrayOfKeys.length-1\];  

  // Do what you want to do with the data, i.e.  
  // append to page or dispatch({ … }) if using redux

}).catch((error) => { … } );

} else {

firebase.database().ref(‘items’).orderByKey().endAt(oldestKeyReference).limitToLast(6).once(‘value’).then((snapshot) => {

 // changing to reverse chronological order (latest first)  
 // & removing duplicate  
 let arrayOfKeys = Object.keys(snapshot.val())  
     .sort()  
     .reverse()  
     .slice(1);

  // transforming to array  
  let results = arrayOfKeys  
     .map((key) => snapshot.val()\[key\]);

  // updating reference  
  referenceToOldestKey = arrayOfKeys\[arrayOfKeys.length-1\];

  // Do what you want to do with the data, i.e.  
  // append to page or dispatch({ … }) if using redux

}).catch((error) => { … } );

}

Hopefully the article wasn’t complete waste of your time and you actually picked something up from it :).

By the way, if you’re a Twitter person you can reach me there at linasmnew, (I’m new there 😅 😄)