Plugin architecture in JavaScript and Node.js with Plug and Play
By David WORMS
Aug 28, 2020
- Categories
- Front End
- Node.js
- Tags
- Asynchronous
- DevOps
- Programming
- Agile
- JavaScript
- Open source
- Release and features [more][less]
Never miss our publications about Open Source, big data and distributed systems, low frequency of one email every two months.
Plug and Play helps library and application authors to introduce a plugin architecture into their code. It simplifies complex code execution with well-defined interception points, also called hooks, and provides the library users the ability to extends, fix, and alter the execution flow of a code without changing it.
This library was just released and published on NPM under the MIT license. It has been available and open source for a while as part of other libraries. The first implementation was created for Node Parameters, an advanced CLI argument library. It was later improved for Nikita, a deployment automation tool before finally being isolated into the Plug and Play package.
A plugin architecture achieves multiple purposes. When hooks are carefully selected and set up, application and library users can extend and adjust the library to fit their needs. But not just users, the original authors also benefit from plugins when the code grow into complexity.
Complex code can be decomposed into multiple components. It eases development, debugging, and testing. For example, the core of Nikita, the deployment automation tool for Node.js, used to be based on a 600 lines module which was only working because it was backed up by more than 300 tests. New features were complex, if not impossible, to introduce. Subtle and known bugs remain. In its latest refactor, the same core module is about 140 lines and only 100 lines once the Plug and Play wrap-up code is removed. A clear understanding of the code is now possible, even by its original author. Old nasty bugs were removed (new ones might have been introduced :). A bunch of new functionalities appeared. More importantly, the developer experience went from a nightmare to a lot of pleasures when new ideas come up.
This is a tutorial to get you up and running with the main features of Plug and Play. It is not that complicated and we will walk slowly through the following usages.
- Accessing and modifying hook arguments
- Modifying the handler behavior
- Inserting code after the handler execution
- Ordering hooks and requiring dependencies
- Passing the results through the chain of handlers
- Asynchronous hook handlers
- Nested/hierarchical architecture
1. A simple application without any plugins
The sample/1/app.js module is a basic web server code. It consists of the core library and will be made extensible by plugins.
const http = require('http')
module.exports = (config = {}) => {
if(!config.mesage){
config.message = 'hello'
}
const server = http.createServer((req, res) => {
res.end(config.message)
})
return {
start: () => {
server.listen(config.port)
},
stop: () => {
server.close()
}
}
}The module consists of an initial function which takes a configuration object config and returns two functions which are start and stop.
The sample/1/index.js module is a cli interface which prompts the user for the start and stop commands.
const readline = require('readline')
const app = require('./app')()
const rl = readline.createInterface({
input: process.stdin,
output: process.stdout
});
rl.prompt();
rl.on('line', (line) => {
switch (line.trim()) {
case 'start':
app.start()
break;
case 'stop':
app.stop()
break;
default:
process.stdout.write('Only `start` and `stop` are supported\n')
break;
}
rl.prompt();
})To execute this sample, clone the tutorial repository. From the project root directory, run node tutorial/1. At the shell prompt, enter start and then stop. It will start and stop the web server on some random port chosen by Node.js. You could modify the code to print the value returned by server.address().port or use the netcat Linux command to find out which one.
2. Accessing and modifying hook arguments
Let’s enhance our application with a new plugin architecture. Our first plugin will modify the configuration object to default the listening port to a value of 3000. Before registering new plugins, we shall update our original library, the app.js file.
In the sample/2/app.js module, we import the Plug and Play module and initialize it. The created instance named plugins is returned and exposed such that users can register new hooks by calling the plugins.register function.
We then define a first hook named server:start. Its role is to provide an opportunity for users to intercept the start command. The handler function provides the default implementation. The first argument of the handler function is defined inside args. We pass the configuration through the config property.
Users now have the ability to modify, enrich, alter the handler function as well as its arguments. In our case, we have access to the configuration to set the default config.port value.
const http = require('http')
// Import the Plug and Play moduleconst plugandplay = require('plug-and-play')
module.exports = (config = {}) => {
// Initialize our plugin architecture const plugins = plugandplay() const server = http.createServer((req, res) => {
res.end(config.message)
})
return {
// Return and expose the plugin instance plugins: plugins, start: () => {
// Defined the `start` hook plugins.call({ name: 'server:start', args: { config: config }, handler: ({config}) => { server.listen(config.port) } }) },
stop: () => {
server.close()
}
}
}A new plugin is created by calling register. To be effective, a plugin must intercept at least one hook and plug its logic.
The hook name is associated with a user function. It could also be an object with the handler property associated with the function. When only one argument is defined, it will be executed before the original hook handler. We will see later how to gain access to the original handler to alter its execution or to inject logic after its execution by declaring a second argument.
When a new hook is registered, the user function receives the same arguments as defined inside args. In our case, we have access to the config property. The sample/2/index.js module registers and implements our first hook to modify the configuration.
const readline = require('readline')
const myapp = require('./app')()
myapp.plugins.register({ hooks: { 'server:start': ({config}) => { // Set the default port value if( !config.port ){ config.port = 3000 } } }})
const rl = readline.createInterface({
input: process.stdin,
output: process.stdout
});
rl.prompt();
rl.on('line', (line) => {
switch (line.trim()) {
case 'start':
myapp.start()
break;
case 'stop':
myapp.stop()
break;
default:
process.stdout.write('Only `start` and `stop` are supported\n')
break;
}
rl.prompt();
})3. Modifying the handler behavior
Attempting to enter start twice will result with a not so friendly error which looks like:
> start
> start
readline.js:1150
throw err;
^
Error [ERR_SERVER_ALREADY_LISTEN]: Listen method has been called more than once without closing.
at Server.listen (net.js:1399:11)
at Object.start (/home/david/plug-and-play/sample/app.js:10:14)
at Interface.<anonymous> (/home/david/plug-and-play/sample/index.js:14:11)
at Interface.emit (events.js:315:20)
at Interface._onLine (readline.js:337:10)
at Interface._line (readline.js:666:8)
at Interface._ttyWrite (readline.js:1006:14)
at ReadStream.onkeypress (readline.js:213:10)
at ReadStream.emit (events.js:315:20)
at emitKeys (internal/readline/utils.js:335:14) {
code: 'ERR_SERVER_ALREADY_LISTEN'
}The web server is already listening on port 3000 and a new server can not be started. It would be more interesting to avoid starting the HTTP server twice by not calling its server.listen function if already started. For this, we will expose the server instance argument in the sample/3/app.js module:
const http = require('http')
const plugandplay = require('plug-and-play')
module.exports = (config = {}) => {
const plugins = plugandplay()
const server = http.createServer((req, res) => {
res.end(config.message)
})
return {
plugins: plugins,
start: () => {
plugins.call({ name: 'server:start', // Expose the server argument args: { config: config, server: server }, // Grab the server argument handler: ({config, server}) => { server.listen(config.port) } }) },
stop: () => {
server.close()
}
}
}Our user hooks can now access the server.listening property to know if the server is up and running. Depending on its value, we will switch off the execution of the original hook handler.
In our handler, we declare a second argument, the previously called handler which is also the original handler. Hooks can choose to overwrite or bypass handlers by providing an alternative implementation or null. In our case, we return null when the server is already listening. An alternative and similar approach would have been to return an empty function function(){}.
const readline = require('readline')
const myapp = require('./app')()
myapp.plugins.register({
hooks: {
// Declare the `handler` second argument 'server:start': ({config, server}, handler) => { if( !config.port ){ config.port = 3000 } // Return null when the server is already listening return server.listening ? null : handler } }
})
const rl = readline.createInterface({
input: process.stdin,
output: process.stdout
});
rl.prompt();
rl.on('line', (line) => {
switch (line.trim()) {
case 'start':
myapp.start()
break;
case 'stop':
myapp.stop()
break;
default:
process.stdout.write('Only `start` and `stop` are supported\n')
break;
}
rl.prompt();
})4. Inserting code after the handler execution
It would be a nice addition to print a message once the server has started.
A new plugin will be created just for this purpose. It reports information about the server lifecycle to the users. It is registered just after the one previously created.
The hook is plugged into the server:start interception point. It returns a new handler function, thus replacing the original one. It first calls the original handler, prints a message, and returns whatever the original handler was returning. In such a case, our new handler behaves just like the old one, it is just printing a message once the server has started.
const readline = require('readline')
const myapp = require('./app')()
myapp.plugins.register({
hooks: {
'server:start': ({config, server}, handler) => {
if( !config.port ){
config.port = 3000
}
return server.listening ? null : handler
}
}
})
myapp.plugins.register({ hooks: { 'server:start': { handler: (args, handler) => { // Return a new handler function return () => { // Call the original handler const info = handler.call(null, args) // Print a message process.stdout.write('Server is started\n') // Return whatever the original handler was returning return info } } } }})
const rl = readline.createInterface({
input: process.stdin,
output: process.stdout
});
rl.prompt();
rl.on('line', (line) => {
switch (line.trim()) {
case 'start':
myapp.start()
break;
case 'stop':
myapp.stop()
break;
default:
process.stdout.write('Only `start` and `stop` are supported\n')
break;
}
rl.prompt();
})5. Ordering hooks and requiring dependencies
Running twice the start command will result in an error:
> start
> Server is started
start
> (node:1358) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'call' of null
at Object.<anonymous> (/home/david/plug-and-play/sample/4/index.js:21:34)
at Object.hook (/home/david/plug-and-play/lib/index.js:200:24)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
(node:1358) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:1358) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.In the second call, the handler is null instead of being a function, thus throwing the above error. This is expected since our first plugin return null if the server is already started.
One solution is to check the handler value and print a message like “the server is already running” when it equals to null. To illustrate hooks ordering, we choose an alternative approach. We will instead ensure that the second hook is always executed before the first hook.
Let’s give a name to our plugins. The first one is named plugin:enhancer and the second is named plugin:reporter. The server:start hook is no longer a handler function. Instead, it is an object containing the handler as well as a before property pointing to the first plugin called plugin:enhancer. This way, the server:start hook of plugin:reporter will always be executed before the one of plugin:enhancer and the handler argument will never be null.
Note, the before and after properties define the execution order between hooks for multiple plugins. They don’t require the plugin to exist. Dependencies are by default optional. To define a required dependency, you must list its name in the plugin required property in the form of a string or an array of strings representing the plugin names.
const readline = require('readline')
const myapp = require('./app')()
myapp.plugins.register({ name: 'plugin:enhancer', hooks: { 'server:start': ({config, server}, handler) => { if( !config.port ){ config.port = 3000 } return server.listening ? null : handler } }})
myapp.plugins.register({ name: 'plugin:reporter', required: 'plugin:enhancer', hooks: { 'server:start': { before: 'plugin:enhancer', handler: (args, handler) => { return () => { const info = handler.call(null, args) process.stdout.write('Server is started\n') return info } } } }})
const rl = readline.createInterface({
input: process.stdin,
output: process.stdout
});
rl.prompt();
rl.on('line', (line) => {
switch (line.trim()) {
case 'start':
myapp.start()
break;
case 'stop':
myapp.stop()
break;
default:
process.stdout.write('Only `start` and `stop` are supported\n')
break;
}
rl.prompt();
})6. Passing the results through the chain of handlers
When plugins register new hooks, several handlers starting with the original handler are called one after the others. The sequence order is based on the plugin registration as well as the hooks before and after properties.
When implementing hook handlers, library authors get the result returned by the execution chain. The plugins.call function returns the value of the last executed handler. The library author is free to use and return the value later in the function. In the sample/6/app.js module, the handler returns the port number. The port is used to enrich the info object which is returned to the end-user at the end of the start function.
const http = require('http')
const plugandplay = require('plug-and-play')
module.exports = (config = {}) => {
const plugins = plugandplay()
const server = http.createServer((req, res) => {
res.end(config.message)
})
return {
plugins: plugins,
start: async () => {
const info = {
time: new Date()
}
// Get the result returned by the last called handler info.port = await plugins.call({ name: 'server:start',
args: {
config: config,
server: server
},
// Handler returning the port number handler: ({config, server}) => { server.listen(config.port) return server.address().port } })
return info
},
stop: () => {
server.close()
}
}
}In the sample/6/index.js module, the info object is used to print the port number.
const readline = require('readline')
const myapp = require('./app')()
myapp.plugins.register({
name: 'plugin:enhancer',
hooks: {
'server:start': ({config, server}, handler) => {
if( !config.port ){
config.port = 3000
}
return server.listening ? null : handler
}
}
})
myapp.plugins.register({
name: 'plugin:reporter',
required: 'plugin:enhancer',
hooks: {
'server:start': {
before: 'plugin:enhancer',
handler: (args, handler) => {
return () => {
const info = handler.call(null, args)
process.stdout.write('Server is started\n')
return info
}
}
}
}
})
const rl = readline.createInterface({
input: process.stdin,
output: process.stdout
});
rl.prompt();
rl.on('line', async (line) => {
switch (line.trim()) {
case 'start':
// Get the info object and print the port number const info = await myapp.start() process.stdout.write(`Port is ${info.port}\n`) break;
case 'stop':
myapp.stop()
break;
default:
process.stdout.write('Only `start` and `stop` are supported\n')
break;
}
rl.prompt();
})Note, values are always returned as promises. Any other type of value is wrapped inside a promise.
7. Asynchronous hook handlers
There is a little problem with the code above. The message informs the user that the server has started but it is not exactly true. The starting function is an asynchronous function. At the time the message is printed, the server isn’t yet started and ready to handle requests.
An asynchronous handler must return and resolve a promise.
The sample/7/app.js module fixes this. Its handler now returns a promise. The start function itself must also wait for the promise to resolve before returning. Note, the setTimeout function is only present in the code to simulate a slow startup time.
const http = require('http')
const plugandplay = require('plug-and-play')
module.exports = (config = {}) => {
const plugins = plugandplay()
const server = http.createServer((req, res) => {
res.end(config.message)
})
return {
plugins: plugins,
// Return a promise start: async () => { const info = { time: new Date() } // Wait for the promise to resolve info.port = await plugins.call({ name: 'server:start',
args: {
config: config,
server: server
},
handler: ({config, server}) => {
// Return a Promise return new Promise( (accept, reject) => { server.listen({port: config.port}, (err) => { // Simulate a perceptible delay setTimeout( () => { err ? reject(err) : accept(server.address().port) }, 1000) }) }) }
})
return info
},
stop: () => {
server.close()
}
}
}Plugin authors must also honor the handler asynchronous nature and handle their returned value as a promise like in the sample/7/index.js module.
const readline = require('readline')
const myapp = require('./app')()
myapp.plugins.register({
name: 'plugin:enhancer',
hooks: {
'server:start': ({config, server}, handler) => {
if( !config.port ){
config.port = 3000
}
return server.listening ? null : handler
}
}
})
myapp.plugins.register({
name: 'plugin:reporter',
required: 'plugin:enhancer',
hooks: {
'server:start': {
before: 'plugin:enhancer',
handler: (args, handler) => {
// Return a promise return async () => { // Obtain a promise const info = await handler.call(null, args) process.stdout.write('Server is started\n')
return info
}
}
}
}
})
const rl = readline.createInterface({
input: process.stdin,
output: process.stdout
});
rl.prompt();
rl.on('line', async (line) => {
switch (line.trim()) {
case 'start':
const info = await myapp.start()
process.stdout.write(`Port is ${info.port}\n`)
break;
case 'stop':
myapp.stop()
break;
default:
process.stdout.write('Only `start` and `stop` are supported\n')
break;
}
rl.prompt();
})8. Nested/hierarchical architecture
Plugin instances can be nested, which means that an interception point can call registered hooks in the current instance as well as in parent instances.
The Nikita project leverages this feature deep inside its architecture. Nikita is an Infrastructure as Code (IaC) tool used to automate deployments. It is made of nested actions. A parent action may define the procedure to install a component, say the installation of MariaDB. This parent action is composed of multiple child actions, such as installing the YUM or APT packages, maintaining the configuration files, and uploading the SSL certificates. Again, each of those actions consists of multiple child actions. The configuration action writes a file but also check for its permissions and ownerships. The package action executes shell commands to install the package depending on the current Linux distribution. At any level, Nikita provides the user the opportunity to register plugins. Once registered, the plugin is available and effective for every child action. For example, the debug plugin print log information to stdout. When registered and activated, it will be effective for an action and its children.
Another common usage of the nested architecture is to offer to the users the possibility to register plugins at an instance level as well as at a global level. Registering a plugin at a global level makes it available to every instance.
The sample/8/app.js implements the latest pattern. A new plugins instance is exported globally by the main Plug and Play module. This global instance is passed to every created local Plug and Play instance. Parent instances are exposed to their children at initialization with the parent property.
const http = require('http')
const plugandplay = require('plug-and-play')
module.exports = (config = {}) => {
const plugins = plugandplay({ // Pass the global plugins instance parent: module.exports.plugins }) const server = http.createServer((req, res) => {
res.end(config.message)
})
return {
plugins: plugins,
start: async () => {
const info = {
time: new Date()
}
info.port = await plugins.call({
name: 'server:start',
args: {
config: config,
server: server
},
handler: ({config, server}) => {
return new Promise( (accept, reject) => {
server.listen({port: config.port}, (err) => {
setTimeout( () => {
err ? reject(err) : accept(server.address().port)
}, 1000)
})
})
}
})
return info
},
stop: () => {
server.close()
}
}
}
// Export global pluginsmodule.exports.plugins = plugandplay()The sample/8/index.js module illustrates global and local registration. The plugin:enhancer is registered globally while the plugin:reporter is registered locally.
const readline = require('readline')
const app = require('./app')// Global registrationapp.plugins.register({ name: 'plugin:enhancer',
hooks: {
'server:start': ({config, server}, handler) => {
if( !config.port ){
config.port = 3000
}
return server.listening ? null : handler
}
}
})
// Local registrationconst myapp = app()myapp.plugins.register({ name: 'plugin:reporter',
required: 'plugin:enhancer',
hooks: {
'server:start': {
before: 'plugin:enhancer',
handler: (args, handler) => {
return async () => {
const info = await handler.call(null, args)
process.stdout.write('Server is started\n')
return info
}
}
}
}
})
const rl = readline.createInterface({
input: process.stdin,
output: process.stdout
});
rl.prompt();
rl.on('line', async (line) => {
switch (line.trim()) {
case 'start':
const info = await myapp.start()
process.stdout.write(`Port is ${info.port}\n`)
break;
case 'stop':
myapp.stop()
break;
default:
process.stdout.write('Only `start` and `stop` are supported\n')
break;
}
rl.prompt();
})Conclusion
We have been using Plug and Play and its ancestors for a few years now and it has proved to be helpful. It opened a lot of potential in our libraries and applications including extensibility and composability. We hope it will serve you well and are welcoming new features and fixes.
