Firebase cursor implementaion

Fetching data in firebase is very easy and fast but fetching large amount of data at once is a bad idea. In this case, we have to use pagination. Unfortunately, Firebase doesn't have built in pagination but we can achieve by using its predefined methods like startAt(), endAt(), limitToFirst() and limitToLast() methods.In this blog, let's achieve pagination using those predefined methods.

For better understanding let's implement pagination for the following data.

"users": {
    1: "",
    2: "",
    3: "",
    4: "",
    5: "",
    6: "", // cursor
    7: "",
    8: "",
    9: "",
    10: "",
    11: "",
    12: "",
    13: "",
    14: "",
    15: "",
    16: "",
    17: "",
    18: "",
    19: "",
    20: ""
}

The idea is if we want to fetch 5 records per page then we should fetch (5 + 1) records, the sixth record acts as a cursor for fetching next set of records. If fetched records count is less than per page record count (here it is 5) then there are no records to fetch further and no cursor.

The code looks like this :

firebase.database().ref("/users")
              .orderByKey().startAt(<cursor>)
              .limitToFirst(<perPage> + 1)
              .once("value").then((data) => {
                       // handle data and save last record id as cursor.
              })

As we know firebase stores and retrieves data in ascending order, the above code fetches the data in that order because we are using startAt() and limitToFirst() methods. If we want in descending order we need to use endAt() and limitToLast() methods.

Then the code looks like this :

firebase.database().ref("/users")
              .orderByKey().endAt(<cursor>)
              .limitToLast( + 1)
              .once("value").then((data) =&gt; {
                       // handle data and save cursor.
              });

I had written a class which handles all the things which we discussed above.
pagination.ts

//Typescript version
class Pagination {
    public dataRef: string;
    public perPage: number;
    public order: string;
    public hasRecords: boolean = true;

    private _pageRecordCount: number = 0;
    private _cursor: string = "";

    constructor(dataRef: string, perPage?: number, order?: string){
        this.dataRef = dataRef;
        this.perPage = perPage || 10;
        this.order = order || 'dsc';
    }

    private _handleData(data: any){
        let currentRecords = [];
        this.hasRecords = false;
        this._pageRecordCount = 0;
        this._cursor = "";
        if(data){
            let entities = data;
            let entityKeys = Object.keys(entities);

            if(entityKeys.length > this.perPage){
                (this.order == 'asc') ? (this._cursor = entityKeys.pop()) : (this._cursor = entityKeys.shift());
                this.hasRecords = true;
            }
            
            for(let entity of entityKeys){
                this._pageRecordCount++;
                if(this._pageRecordCount <= this.perPage){
                    let tempData = entities[entity];
                    tempData["id"] = entity;
                    (this.order == 'asc') ? (currentRecords.push(tempData)) : (currentRecords.unshift(tempData));
                }
            }
        }
        return new Promise((resolve, reject) => {
            resolve(currentRecords);
        });
    }

    public fetchRecords(){
        let databaseRef = firebase.database().ref(this.dataRef);
        let orderByRef = databaseRef.orderByKey();
        let filterRef;
        let cursorRef;

        if(this._cursor){
            cursorRef = (this.order == 'asc') ? orderByRef.startAt(this._cursor) : orderByRef.endAt(this._cursor);
        }else{
            cursorRef = orderByRef;
        }

        if(this.order == 'asc'){
            filterRef = cursorRef.limitToFirst(this.perPage + 1);
        }else {
            filterRef = cursorRef.limitToLast(this.perPage + 1);
        }

        return filterRef.once("value").then((data) => {
            return this._handleData(data.val());
        });
        
    }
}

Now you can use the above pagination class in the following manner.

// initialize object
// new Pagination(<ref>, <perPage>, <order type 'asc' or 'dsc'>)
let pagination = new Pagination("/users", 5, 'asc'); 
//for fetching records
pagination.fetchRecords().then((data) =&gt; {
 //data
})

//hasRecords indicates that there are probably more results
pagination.hasRecords 

Demo URL:
https://jayaram-blog-examples.firebaseapp.com/pagination/

You can find the whole code in the following URLs:
Typescript:
https://gist.github.com/kurapatijayaram/32f4dc8e49948b07a9caba1a6b4449c7

Javascript:
https://gist.github.com/kurapatijayaram/aa135e83388439f7a16a77d1509e9d6f

AngularJS(2.x/4.x) component:
https://gist.github.com/kurapatijayaram/0a2e4ded7806c190d6f77f9e739dca46

Async pattern - Data fetching from firebase database using Angular 2

Real-time features like notifications, fetching the details is a breeze with Firebase database and angular 2's async pipe. Firebase is pretty cool to store your data. It's ideal for blogs, notifications. If you are like me, you fan out the keys to various nodes(not store the entire denormalized structure). In that case, it's important to loop through all keys and get data at each element level. In this blog, we explore a technique to fetch the data using Angular 2's async pipe from Firebase database.

First, let's create an angular module for app initialization and then pipes for fetching the data.

Before starting install firebase package using npm.

npm install firebase --save

First, let's create a module for initialization.

firebase.module.ts :

//firebase.module.ts
import { NgModule } from "@angular/core";
import * as firebase from "firebase";

export interface IFirebaseConfig{
    "apiKey": string,
    "authDomain": string,
    "databaseURL": string,
    "storageBucket": string,
    "messagingSenderId": string
}

@NgModule()
export class FirebaseConfigModule{
    static init(config: IFirebaseConfig){
        firebase.initializeApp(config);

        return {
            ngModule: FirebaseConfigModule
        }
    }
}

Now import the above module into application root module like this:

import { NgModule } from '@angular/core';
import { BrowserModule } from '@angular/platform-browser';
import { AppComponent } from './app.component';
import { FirebaseConfigModule } from './firebase.module';

@NgModule({
    imports: [
        BrowserModule,
        FirebaseConfigModule.init(
            {
                "apiKey": "",
                "authDomain": "",
                "databaseURL": "",
                "storageBucket": "",
                "messagingSenderId": ""
            }
        )
    ],
    declarations: [ AppComponent ],
    bootstrap: [ AppComponent ]
})
export class AppModule { }

Now you can use the firebase features anywhere in the application just by importing firebase from firebase package in the following manner.

...

@Component({
    selector: "users",
    template: `
        User name: {{userName}}
    `
})
export class UsersComponent {
    public userName: string;

    constructor(){
        firebase.database().ref("users/-KiAAhNT6LZMHGS9c-4g/name").once("value", (data) => {
            this.userName = data.val();
        });
    }

    ...
}

Cool, now everything works fine.

But how it would be if we had a pipe which accepts data location path and fetches data for us? instead of writing in the component each time when we want to fetch the data, It sounds great right? let's write a pipe to achieve this.

Add the following pipe function into firebase.module.ts:

@Pipe({name: "firePipeValue"})
export class FirebasePipe implements PipeTransform {
    transform(ref: string, listenType: "on" | "once"="on"): Observable<any>{
        return Observable.create((observer: Observer<any>) => {
            function onValue(data){
                observer.next(data.val());
            }

            if(listenType == "on"){
                firebase.database().ref(ref).on("value", onValue);
            }else{
                firebase.database().ref(ref).once("value", onValue);
            }

            return function(){
                firebase.database().ref(ref).off("value", onValue);
            }
        });
    }
}

And change the FirebaseConfigModule class to the following manner.

@NgModule({
    exports: [FirebasePipe],
    declarations: [FirebasePipe]
})
export class FirebaseConfigModule{
    static init(config: IFirebaseConfig){
        firebase.initializeApp(config);

        return {
            ngModule: FirebaseConfigModule
        }
    }
}

Now you can use the pipe in the view like this:

{{'clients/-KiAAhNT6LZMHGS9c-4g/name' | firePipeValue:'on' | async}}
{{'clients/-Ki9saJqb03SACfx89_X' | firePipeValue:'off' | async}}

In this pipe we need to pass one argument "on" or "off". "on" is for listening to the data changes and if you don't want changes to reflect? pass the argument "off".

Note: Don't use this pipe for fetching lists of data. For fetching lists I have written another pipe, I will share that gist.

You can find the above code in the following gist URL:
https://gist.github.com/kurapatijayaram/e2f97d0966bf5baac07a5aa7a0c3e77d