Modular Application Architecture - Intro

This is the first post from a series of posts that will describe strategies to build modular and extensible applications. (inspired by a talk from qafoo in 2011 in Berlin)

When developing a software, one of the most common steps is taking care that the resulting application is extensible and modular.

Let's suppose we have our application or library. If we see it from outside, often it looks as a single thing. Taking a closer look we can identify different and more-or-less independent set of components. Components might still be really interconnected and have dependencies with the rest of the application and their development might require a deep knowledge of the rest of the application.

As the application grows we can continue adding components... but this comes with a price. Components often knows too much of our application and there is a delicate equilibrium of dependencies between them and our application. When not handled carefully, a small change in one component might require changes in many other.

As a rule of thumb, I personally try to follow as much as possible the Acyclic dependencies principle

Another way to allow extensibility but keeping the application "clean" is to introduce modules.

Modules

Modules are "components" developed outside of the application. Modules communicate the the application only via defined entry-points.

Different people use different names for modules as plugins, bundles, add-ons and so on... but the main idea is always the same, independent functionalities developed outside of the application.

Why?

By using modules that are not part of the application, we gain several advantages as:

• Customizations: the application can behave really differently by just enabling/disabling some modules.
• Less dependencies: the modules are more independent from the application itself, till the "entrypoints" are compatible, both modules and the application can evolve independently.
• Third-party extensions: since the modules are not part of the application and the "entrypoints" are well defined, developing modules can be done by third parties.
• Independent development: Since the application and the modules are independent, they can be:
  • developed by external developers
  • released with independent release cycles
  • developed potentially with different technologies
• Smaller application The applications is smaller (many functionalities can be implemented via modules) and smaller application is translated in better maintainability.

Just to make few example to understand why is important to have an application that allows modules:

• Wordpress it the most popular blogging platform nowadays. Part of its success is an incredibly powerful and at the same time incredibly simple plug-in and theme system.
• Symfony is a popular PHP framework. Thanks to its "bundles" system, is possible to get a large variety of ready functionalities with almost no effort.
• PHP, itself offers a great set of extensions and those extensions allowed to integrate the language with a large variety of systems.

Challenges to solve

To make our application modular we have to carefully decide what modules should be able to do, how will communicate with the application and how will expose their functionalities. When implementing a plug-in/modular system we have to consider which part of application behavior a module will be allowed to change and in which measure.

As example, if we are developing a word processor, a module might be able to add just the support for a new file format or just a new icon on the toolbar or on the opposite side, to change completely the whole application.

Depending on this decisions we can architect our module very differently, but almost always we will have to deal with module structure, registration/configuration, resource management and communication with the core.

1. Module structure

A starting point is the module structure as file naming, folder structure, class naming, etc. When the module structure is decided, the remaining aspects are highly influenced.

Often the module structure is the first entrypoint with the application.

My personal suggestion is to not impose any specific folder structure but to have a single "entrypoint" where the module is loaded and using that entrypoint as source of truth and configuration. The meaning of "entrypoint" will be defined in the next paragraph

2. Registration

When the module structure is decided, the next step is to let our application know about it.

There are mainly two ways to do it:

• discovery: our application can try to use predefined set of "rules" to look for new modules and load/register them.
  As example is possible to try to look in specific folder (s) for some particular file (s) or check if a particular class (es) is defined and try to instantiate it.
• configuration: we can explicitly configure the application to load a particular file (s) or a particular class (es).

There is no preferred ways to do it, it really depends on your application. "configuration" based registration might be better if your audience is more technical while the "discovery" approach can offer a more smooth user experience (but might be more difficult to implement).

My personal suggestion is to have again a single "entrypoint" that will register the module into the application core, can be a single file, class, database row, service-name etc...

The application offers one or more "interfaces" to interact with its "core" and they are used via "entrypoints". By "interface" here is intended any possible protocol or convention defined by the application that might be used used by the module to achieve a goal.

Examples:

• composer: the composer way of adding functionalities is by discovery.
  Composer will search for installed packages having type=composer-plugin, will search for a property named class defined in the package's composer.json file and will register the event listeners declared by that class. More info about it are available here
• symfony: the symfony way of adding functionalities is by configuration.
  In the Appkernel class is necessary to define all the "bundles" that will be used by symfony to enrich the framework functionalities. The "bundle" class will be instantiated and is responsible for registering the bundle into the extension points that symfony allows. The bundle class is able to alter almost any part of symfony itself as it has access to the symfony core dependency injection mechanism. More info about it are available here. Recently with flex an automatic discovery and configuration become available.
• laravel: the laravel way of adding functionalities is by discovery.
  Laravel will search for installed packages and will search for a property named extra.laravel defined in the package's composer.json file and will register the service providers declared in there. More info about it are available here

3. Configuration

When the module is registered into the core it might be useful to allow some additional configurations.
Most probably the application is configured somehow to match user-expectations, the same thing should happen for the module.

Is possible to offer different configuration strategies, my personal suggestion is to offer a configuration mechanism very close to the way how the application is configured. If the application is configured via configuration files, environment variables, database entries or a nice control panel, the module should be configured in the same way

A nice feature that modules should have (and application should allow) are "defaults", so virtually a module might need no configuration.

Examples:

• composer: the composer's composer.json file allows you to decide which plugins to load, which commands to run and so on.
• symfony: the symfony allows YAML, XML and PHP as configuration languages, configuration values can to be declared in config.(yml|xml|php) (and in few other places) and bundles can provide default values for it.
• laravel: the laravel uses mainly PHP as configuration language and thank to it is possible to use the power of the PHP syntax when defining configurations.

4. Resources

Some modules might need to expose some content to the user as images, CSS, templates, translations, files or similar. Applications can take different way to make available the content to the user.

When the resource is "application managed", is just about using the "application interface" to provide/override the resource (as example using the application plugin interface).

When resources are not managed by the application but are module-specific, things can get more complicated. Some way to handle it are:

• linking/copying the content to the web directory
  • requires some step to publish the resources
• serve the resources using the applications, as example pipe-it via PHP
  • can be slow or resource intensive
• configure the webserver to serve explicitly the plugin resources
  • makes in vain the "plug & play" idea behind modules by adding additional configurations

• let the user "copy and paste" resources manually

  Each option has drawbacks and advantages. Lately the "linking/copying" strategy is the most popular.

5. Interaction with the application

Now that the module is loaded and configured, everything is ready.
The application can decide all the "interaction" points and go through modules checking if some module is "interested" in interacting with the application core (probably)

There are many strategies to "interact" with the application some of them limit the ability of a module to interact on a specific (user defined) part if the application while other might allow to change potentially the whole application flow.

In the next articles we will go deeper into possible ways to interact with the application.

Conclusion

In this first article I've described some motivations and challenges when building extensible applications. In the next articles we will see some strategies of interaction with the application as hooks, event manager, observer pattern, inheritance and others.

Looking forward for your feedback.