Modular Application Architecture - Static assets

This is the sixth post from a series of posts that will describe strategies to build modular and extensible applications. In this post we will see how to deal with static files (images, fonts...) and how to make them available to the core application.

In the previous posts we discussed how to build a plugin architecture, hot to register, configure and run plugins. Plugins are composed of many files, each file has its own propose. The source code (or a compiled version of it in case you are dealing with complied languages) is the plugin-itself, but what about "static files" as images, css, or any other file that we want to make available to the application core or to its user?

We need a way to publish those "assets" (the static files...) to the application and later the application can make them available to the user.

As it should be clear, the "application" is the coordinator and should define how a plugin should publish the assets.

I've seen applications that do not define rules on how to publish assets, and sadly often the result is that the plugin authors start using different strategies leading to a chaotic plugin system where no one knows which is the best way to expose static files.

Publishing assets

There are many strategies on how to publish static assets, each of them with advantages and pitfalls. Some of them require coordination from the main application, some of them require coordination from web server or some of them require manual steps to be performed by system administrators and so on...

Lets suppose for our application a directory structure as follows and the application/public folder to be directly accessible by the webserver:

application/
├── src/ # application code
├── plugins/ 
├── public/ # directory accessible to the web-server
│   ├── index.php
│   ├── app.js
│   └── style.css

Obviously this is just an example, the plugins can be located anywhere we want.

1. Install plugins in public folder

The simples strategy to expose public assets from plugins, is just to put the whole plugins folder into the public directory.

application/
├── src/ # application code
└── public/ # directory accessible to the web-server
    ├── plugins/
    │   ├── plugin-1/
    │   │   ├── src/
    │   │   ├── public/
    │   │   └────── kitten.jpg
    │   └── plugin-n/
    ├── index.php
    ├── app.js
    └── style.css

Advantages:

• simple
• does not require special server configurations
• does not require "application" configurations to expose public assets

Disadvantages:

• Very Insecure! (your source code is exposed)
• no easy way to control permissions

This strategy as example is used by Wordpress. **This solutions is very insecure and not really suggested, ** it is here just for informational proposes.

2. Copy public assets

To solve the security issues from the first example, we can copy just the static files to the public folder instead of placing it there the whole plugin.

The application defines one or more folders in the plugin directory structure to be copied to the main public folder. As example, if yor plugin has a folder named public, its content will be copied to the application/public/[plugin-name] folder and made available to users.

If the plugin system has an installation step, the copy can be created automatically when "installing" it, otherwise has to be done manually or in a "deploy" step.

application/
├── src/ # application code
├── plugins/ # plugins code
│   ├── plugin-1/
│   │   ├── src/
│   │   ├── public/
│   │   └────── kitten.jpg
│   └── plugin-n/ 
└── public/ # directory accessible to the web-server
    ├── plugins/
    │   ├── plugin-1
    │   ├────── kitten.jpg
    │   └── plugin-n*
    ├── index.php
    ├── app.js
    └── style.css

Advantages:

• simple
• does not require special server configurations

Disadvantages: - the application should have a way to copy public assets from a plugin (an install step), otherwise should be done manually - requires a "deploy" or an "install" step in which assets are copied to the public folder - necessary to re-sync the public folder when updating the plugin files - not convenient while developing plugins - no easy way to control permissions

3. Symlinking

To solve some of the disadvantages of the "copy" strategy, instead of copying the public folder,
we can use a "symlink" to link its content to the main public folder.

As example, if yor plugin has a folder named public, its content will be symlinked to the application/public/[plugin-name] folder and made available to users.

If the plugin system has an installation step, the link can be created automatically when "installing" it, otherwise has to be done manually or in a "deploy" step.

application/
├── src/ # application code
├── plugins/
│   ├── plugin-1/
│   │   ├── src/
│   │   ├── public/
│   │   └────── kitten.jpg
│   └── plugin-n/ 
└── public/ # directory accessible to the web-server
    ├── plugins/
    │   ├── plugin-1* # points to application/plugins/public
    │   └── plugin-n*
    ├── index.php
    ├── app.js
    └── style.css

Advantages:

• simple
• does not require special server configurations
• changes in the plugin files are automatically available

Disadvantages:

• the application should have a way to expose public assets from a plugin (an install step), otherwise should be done manually
• no easy way to control permissions
• still requires a "deploy" or an "install" step in which assets are symlinked to the public folder
• (some systems do not support symlinks (really few...))

4. Server configuration

Some applications that are shipped together with a web server, can modify directly the server configurations and expose the files through it; in alternative the web server can be configured manually to expose specific folders/files.

In general, we can resume advantages and disadvantages of exposing plugin assets via server configuration as:

Advantages:

• high level of control
• changes in the plugin files are automatically available

Disadvantages:

• requires server configuration (manual or via more complex automation)
• depends on the server that you are using
• no easy way to control permissions

Thanks to the containerization technologies, modern applications tend to ship together with the application also part of the infrastructure as webserver, db and so on. This mean that part of the plugin files we can have also infrastructure configurations as web server configs. This makes the whole plugin much more complex and often is not worth.

We can go more in detail and see which strategies to expose plugin assets via server configuration we can use. Different strategies have different trade-offs and are able to solve some of the disadvantages said before.

Lets consider some examples using nginx and suppose to have a directory structure as:

application/
├── src/ # application code
├── plugins/ # pligins code
│   ├── plugin-1/
│   │   ├── src/
│   │   ├── public/
│   │   └────── kitten.jpg
│   └── plugin-n/ 
└── public/ # directory accessible to the web-server
    ├── index.php
    ├── app.js
    └── style.css

Explicit

We can explicitly define a list of directories and their exact location on the filesystem and make them exposed by the web server. This approach requires an human intervention as the mapping are manually defined (or complex automation).

server {
    ...

    location /plugins/plugin-1 {
       root /application/plugins/plugin-1/public
    }
    ...
    location /plugins/plugin-n {
       root /application/plugins/plugin-n/public
    }    
}

In addition to the general advantages/disadvantages of the configuration based approach we, for this specific strategy we have:

Advantages:

• high level of control

Disadvantages:

• requires server configuration, mostly manual or via complex automation
• each new plugin require editing the configs

Pattern matching

Instead of mapping manually the paths we ca use some pattern matching. If yor plugin has a folder named public, its content can be mapped to the /plugins/[plugin-name] location and made available to users.

server {
    ...

    location ~ ^/plugins/(?<pluginName>.+) {
       root /application/plugins/$pluginName/public
    } 
}

In addition to the general advantages/disadvantages of the configuration based approach we, for this specific strategy we have:

Advantages:

• can be automated and the manual intervention can be avoided
• does not require a new configuration when adding a ned plugin
• no complex automation or human intervention are needed

5. Serving static files through the application

Most of the strategies said before expose in one way or another the files directly to the webserver. The approach tend to be very resource-efficient but the drawbacks of the approach are that the application is not in control the file anymore.

As example to control access permissions to static files part of a plugin we have to serve them through the application. Doing so, each time a file is requested, the application will be invoked and will act as intermediary. The application can do processing (decide if we are allowed to see the file as example), and eventually serve the file to the user.

This approach is extremely flexible but the biggest drawback are the performances as each request will spawn our application too.

Via application only

The application can offer some features and replace the role of a webserver, serving static files by reading them instead of the webserver.

<?php
// suppose this piece if code is executed when requesting /plugins/plugin-1/my_protected_file.txt

// here we can do here any type of security check and store it into $hasPermissions

if ($hasPermissions) {
    header("Content-Type: text/plain");
    $fp = fopen("/application/plugins/plugin-1/public/my_protected_file.txt", "r");
    fpassthru($fp);
    fclose($fp);
} else {
    header('HTTP/1.0 401 Unauthorized');
    // or header("HTTP/1.0 404 Not Found");
}

The main disadvantage of this approach is performance wise. Serving a static file through a scripting/programming language is an overkill, especially if the file is big.

The application should re-implement all the "file" handling capabilities as partial downloads, mime type inspections, download resume, caching... that are part of most webserver core functionalities.

To sum up:

Advantages:

• high level of control
• files can be stored anywhere (when the application know how to access them, it is enough!)

Disadvantages:

• low performance
• requires re-implementation of some HTTP basics at application level

Via application but assisted by the webserver

To solve the performance issues of the previous strategy, but keeping still the ability to control via application who can download the files, we can use the X-Sendfile feature offered by Apache or the analog X-Accel feature offered by Nginx.

The previous example will become:

<?php
// suppose this piece if code is executed when requesting /plugins/plugin-1/my_protected_file.txt

// here we can do here any type of security check and store it into $hasPermissions

if ($hasPermissions) {
    header("Content-Type: text/plain");
    header("X-Sendfile: /application/plugins/plugin-1/public/my_protected_file.txt");
    // or header("X-Accel-Redirect: /application/plugins/plugin-1/public/my_protected_file.txt"); for nginx 
} else {
    header('HTTP/1.0 401 Unauthorized');
    // or header("HTTP/1.0 404 Not Found");
}

This approach combines the ability of giving full control to the application, but delegating to the web server the responsibility of how to read an send the file.

The application decides if the file can be downloaded, and then sends the X-Sendfile containing the path of the file. The webserver will serve the file at the specified location (applying the common HTTP capabilities and optimizations).

To sum up:

Advantages:

• high level of control
• lower performance impact

Disadvantages:

• files must be accessible by the web-server
• creates some dependency on the webserver

Conclusion

In this article we saw some strategies on how to serve static files part of a plugin, each discussed strategy has specific trade-offs and can be useful in some cases.

This is the last article for the series of "Modular Application Architecture" on how to build plugin-based architectures.

Hope you enjoyed this article and the others. If you have some feedback, do not hesitate to leave a comment!