What is Tensorflow.js
There are three main ways to harness the capabilities of tensorflow.js –
1. Define and Train Deep Learning Models
Tensorflow.js sports high-level API layers similar to the Keras API which makes it easier to define and train deep learning models inside the web browser or in node.js server.
2. Use Pre-Trained Models
You can use pre-trained models and inference them for prediction using Tensorflow.js. You can either convert your own pre-trained python made models into tensorflow.js models or use out-of-the-box state-of-art pre-trained models like VGG16, ResNet, DenseNet, MobileNet, etc.
3. Transfer learning
Although there are pre-trained models available to be used in Tenosrflow.js you may still like to create a model for your own custom needs. Usually, training state-of-art deep learning models requires a large no of parameters and computing power. But you can apply transfer learning in Tensorflow.js that shortcuts this process by reusing the existing state-of-art pre-trained models by removing its final or pre-final layers for training with your custom data.
Why Use Tensorflow.js?
- Tensorflow.js reduces inference and training latency due to the locality of data and ML model in the browser.
- Ability to run models when the client is offline, due to the fact that the data is hosted on the web browser and not on a remote server
- Provides privacy of data since data never leaves the browser. This is helpful in cases when working with healthcare, financially, or with preferential data.
- If you are using tensorflow.js for browser it can help to reduce server cost.
- It simplifies the development and deployment stack providing a zero-install user experience(No installations are required to build ML models)
- An inherently interconnected environment opens direct access to various sources of machine-learning data and resources coupled with visualization and interactivity
Disadvantages of Tensorflow.js
In spite of many advantages that we discussed above, tensorflow.js also has some disadvantages as follows –
- Training larger models could be time-consuming with tensorflow.js in the browser.
- Pre-trained of more than 30Mb are generally too bulky to be loaded in the browser and it can slow down the inference.
How to use Tensorflow.js
It is extremely easy to use Tensorflow.js in the browser, just add the following to your HTML file and it will load the library from the CDN whenever users open the page on the browser.
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/[email protected]/dist/tf.min.js"></script>
You can also install it using NPM (node package manager) –
npm install @tensorflow/tfjs
npm install @tensorflow/tfjs-node
Tensorflow.js Pre-trained models
Let us take a look at some interesting pre-trained models that can be used out of the box in Tensorflow.js –
1. MobileNet model
Trained on the vast ImageNet database, MobileNet is optimized for small devices for fast load and inference. It can classify images into 1000 object categories, such as keyboard, mouse, pencil, and many animals. And is easily applicable to embedded devices too.
2. Pose Estimation
The BodyPix and DeepLab models are used for real-time human body parts and semantic segmentation.
4. Face Detection
It has BlazeFace model for rapid face detection that is optimized of the browser. It also provides face landmark detection model which can easily approximate the surface geometry of one face and plot landmarks.
5. Text Based Models
Tensorflow.js universal sentence encoder model sporting a 512-dimensional word embedding can be used for sentiment and similarity detection models. And a real-world applicable toxicity-detector model which can be used directly in apps to detect toxic texts.
6. Audio Models
Speech-Commands model which is trained on speech command data set can classify 1-second audio clips into commands.
7. General Models
It also provides the implementation of KNN algorithm statistical pattern recognition that can be useful for transfer learning on pre-trained models.
Tensorflow.js real-world Examples
face-api.js is a Tensorflow.js and node.js based high-level API for face recognition which is highly popular and easy to use. It also provides additional interesting functionalities like face-landmark detection, face expression detection, and age and gender estimation
ii) Real-Time Person Removal