Introduction
If you are a Javascript developer and don’t know much about Python yet you want to explore deep learning then Tensorflow.js is the right framework for you. In this article, we will explain what is tensorflow.js and what are its various features and offerings. And we will also do a comparison between Javascript vs Python for machine learning to help you understand what you get and what you miss out on if you use Javascript based framework like Tensorflow.js
What is Tensorflow.js
Tensorflow.js, launched by Google in 2018, is the most popular and open-source machine learning library in Javascript that lets you create, train, deploy and infer machine learning models directly in the browser or node.js server.
Traditionally state of art deep learning models are created, trained, and deployed on the server-side usually with Python-based libraries. On the other hand, modern Web browsers are equipped with arguably the most comprehensive technologies and APIs of Javascripts for processing textual, image, audio, and video data at the client side.
Tensorflow.js tries to leverage this Javascript framework not only for in-browser machine learning capabilities but also supports node.js for full-stack developers.
It supports both inference and training within web browsers and Node.js as it leverages GPU acceleration (WebGL in browsers and CUDA kernels in Node.js). One can define neural network model architectures in javaScript, serialize and deserialize models, convert to and from Python deep-learning frameworks with built-in support for data pipelines and an API for visualization.
Tensorflow.js Features
There are three main ways to harness the capabilities of tensorflow.js –
1. Define and Train Deep Learning Models
Tensorflow.js sports high-level API layers similar to the Keras API which makes it easier to define and train deep learning models inside the web browser or in node.js server.
2. Use Pre-Trained Models
You can use pre-trained models and inference them for prediction using Tensorflow.js. You can either convert your own pre-trained python made models into tensorflow.js models or use out-of-the-box state-of-art pre-trained models like VGG16, ResNet, DenseNet, MobileNet, etc.
3. Transfer learning
Although there are pre-trained models available to be used in Tenosrflow.js you may still like to create a model for your own custom needs. Usually, training state-of-art deep learning models requires a large no of parameters and computing power. But you can apply transfer learning in Tensorflow.js that shortcuts this process by reusing the existing state-of-art pre-trained models by removing its final or pre-final layers for training with your custom data.
Javascript vs Python for Machine Learning
Both Python and Javascript first appeared in the scene in the 1990s but it is only in recent years that Python saw a surge in popularity due to its ease of use for ML frameworks.
But Javascript had been the most popular and widespread used language among developers for the last two decades and it still is even now. A library like Tensorflow.js appeals to those developers who already know Javascript and are interested to explore machine learning using their beloved Javascript.
Let us list down the comparison between javascript vs python for machine learning.
- JavaScript is the most popular open-source programming language by many measures, so there is an abundance of JavaScript talent and enthusiasm. But Python has seen an exponential rise in popularity in the last 5 years in the machine learning space.
- Javascript has a vibrant ecosystem and wide applications on both client and server sides. While node.js allows applications to run on the server-side without the resource constraints of the browser. Additionally, the V8 engine makes JavaScript code run fast.
- Python has an already established base for machine learning and provides numerous libraries for the same while being highly readable and being used by big companies like Facebook, google, amazon, Netflix, and apple. However, Javascript has limited ML frameworks, Tensorflow.js being the most popular.
- There is a big growing community for Python machine learning and its learning resources can be found easily all over the internet. But when it comes to Javascript libraries like Tensorflow.js there are not many resources available and the only reliable sources are its documentation.
Why Use Tensorflow.js?
- Tensorflow.js reduces inference and training latency due to the locality of data and ML model in the browser.
- Ability to run models when the client is offline, due to the fact that the data is hosted on the web browser and not on a remote server
- Provides privacy of data since data never leaves the browser. This is helpful in cases when working with healthcare, financially, or with preferential data.
- If you are using tensorflow.js for browser it can help to reduce server cost.
- It simplifies the development and deployment stack providing a zero-install user experience(No installations are required to build ML models)
- An inherently interconnected environment opens direct access to various sources of machine-learning data and resources coupled with visualization and interactivity
Disadvantages of Tensorflow.js
In spite of many advantages that we discussed above, tensorflow.js also has some disadvantages as follows –
- Training larger models could be time-consuming with tensorflow.js in the browser.
- Pre-trained of more than 30Mb are generally too bulky to be loaded in the browser and it can slow down the inference.
- As we already highlighted, Javascript as a language is not really popular for machine learning, and so there are not many resources available for tensorflow.js apart from its documentation.
How to use Tensorflow.js
It is extremely easy to use Tensorflow.js in the browser, just add the following to your HTML file and it will load the library from the CDN whenever users open the page on the browser.
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/[email protected]/dist/tf.min.js"></script>You can also install it using NPM (node package manager) –
Browser setup:
npm install @tensorflow/tfjsNode.js setup:
npm install @tensorflow/tfjs-nodeTensorflow.js Pre-trained models
Let us take a look at some interesting pre-trained models that can be used out of the box in Tensorflow.js –
1. MobileNet model
Trained on the vast ImageNet database, MobileNet is optimized for small devices for fast load and inference. It can classify images into 1000 object categories, such as keyboard, mouse, pencil, and many animals. And is easily applicable to embedded devices too.
2. Pose Estimation
The HandPose (used for hand pose detection) and the PoseNet models (used for human pose estimation) are available in python as well as the javascript version of tensorflow.js.
3. Segmentation
The BodyPix and DeepLab models are used for real-time human body parts and semantic segmentation.
4. Face Detection
It has BlazeFace model for rapid face detection that is optimized of the browser. It also provides face landmark detection model which can easily approximate the surface geometry of one face and plot landmarks.
5. Text Based Models
Tensorflow.js universal sentence encoder model sporting a 512-dimensional word embedding can be used for sentiment and similarity detection models. And a real-world applicable toxicity-detector model which can be used directly in apps to detect toxic texts.
6. Audio Models
Speech-Commands model which is trained on speech command data set can classify 1-second audio clips into commands.
7. General Models
It also provides the implementation of KNN algorithm statistical pattern recognition that can be useful for transfer learning on pre-trained models.
Tensorflow.js real-world Examples
i) Face-API.js
face-api.js is a Tensorflow.js and node.js based high-level API for face recognition which is highly popular and easy to use. It also provides additional interesting functionalities like face-landmark detection, face expression detection, and age and gender estimation
ii) Real-Time Person Removal
Everyone hates photobombing but did you know you can remove unwanted people or objects using a tensorflow.js backed javascript app that runs right in the browser. You can try it out yourself on Codepen or Glitch and check its GitHub repository.
iii) Handtrack.js
handtrack.js is a javascript library for real-time hand detection harnessing the power of neural networks and (you guessed it!) tensorflow.js
iv) Other
There is a multitude of things that can be achieved by combining deep learning and javascript. For example, A gaming website that helps improve drawing or a self-driving car simulator that runs reinforcement learning models in the browser or Google’s Project Magenta uses TensorFlow.js to run RNNs and other kinds of deep neural networks to generate musical and instrument sounds in the browser.
