J4 essay – Why Joomla! should clean its file structure
Note: This document is part of a set of documents which constitute an essay about what Joomla! is, or can be in the future. These documents do not constitute the current state of Joomla!'s development, nor are they just rants or brain dumps. They list a set of structured ideas about what Joomla! can be, and provide some ideas on how to get there. Their main purpose is to generate discussion over Joomla!'s community.
This is the second document in the series. The first document explained why (IMO), Joomla! should be broken into two separate parts, a Framework and a Platform. This document will continue the previous discussion, and propose a new file structure to support the same idea.
The first thing I wanted to achieve when I first installed Joomla!, was the decoupling of the administrator from the front-end. Lets face it, a Joomla!'s website, is actually two separate web applications sharing resources. Front-end and back-office. When I was first looking at Joomla!'s code, I found the defines.php file in the include folder, that seamed to let me configure Joomla!'s whole directory structure, and customise all the directory locations. Great! I was convinced. No need to look any further.
Although Joomla!, at first glance, gives you the impression that you can modify its structure, the truth is, if you change the directory's locations, everything breaks. This doesn't happen because of any technical incapacity from the used technologies. It happens because most files are referenced with hard-coded locations.
The main reason I wanted to decouple the administrator, was to segment my web site's traffic using Awstats (Apache's logs). Your back-office traffic logs are the same as your front-end traffic logs. There wasn't any scenario that I thought of, where I wanted to have only one log for both applications.
There is also another good reason for the decoupling, which is the obfuscation of the back-office's location. But when I tried to achieve this, the back-office broke. Some extensions didn't work properly, or just stopped displaying all together. In my opinion, this is a big problem, and by searching a bit on the Internet, I seamed far from being alone.
Structure complexity and redundancy
Another thing I've realised, is the complexity and redundancy of Joomla!'s directory structure.
Extensions are scattered all over the place. Core functionality is all over the place as well. You have two root folders where to put dependencies (media and images)... Some people even use the script.php file with the only purpose of copying files from the media folder (where files are deployed), to the images folder. What's the point, I wonder? They're all dependencies. Why not just use one folder, and set a sub-folder in it as a gallery for the Media Manager?
It's not easy to assume anything about Joomla!'s directory structure, just by looking at it. It is also difficult to know which folders to write-protect, in the eventuality of a Joomla!'s Update.
Uniformity vs Continuity
Seams that there's always a big concern by the Joomla!'s developing community, to provide backwards compatibility between Major Versions. For Joomla!4, and according to Joomla!'s PLT, this backwards compatibility will break, in order to leap further. I personally agree with this.
A complete re-write of Joomla!'s core, does not necessarily mean a change in UX/UI. End-users might not even notice it, because lets face it, users interact with templates, and manipulate the content with extensions provided to them. The core “just” makes everything happen.
Personally, if there were to be a rupture between Joomla!3 and Joomla!4, I think a complete revision of the application's core would be advised. This might not be such a bad thing. It's a fresh start.
Migration vs Upgrading vs Updating vs Porting
It seams to me, there's an institutionalised misconception about these terms in Joomla!'s community, which might lead to a wrong accountability by end-users and developers.
In software development, there's a convention where products come with a version number that can be separated up to 4 sub-groups using a dot notation. This version number is composed by a Major.Minor.Resivion.Build numbers (ex:. v188.8.131.52789).
To keep it simple, an Upgrade is when you increment the Major version number, and usually implies a shift in paradigm/philosophy, or a major rewrite of the application. These usually imply breaking backwards compatibility.
An increment in any other numbers (Minor, Revision or Build) are just Updates, and don't break compatibility (usually). Incrementing the Minor version implies adding and/removing functionality. Increments in the Revision number imply performance and/or security checks, and Build numbers are used in applications which are compiled (not Joomla!'s case).
The difference between Migration and Porting, is in the context. Usually, and keeping it on a basic level, software code (technology) is Ported, and data is Migrated. If you're changing CMS's, then you will need to migrate you data between platforms/environments. If you're planning on rewriting Joomla! in java or .NET (for example), you'll need to Port your code.
Now that I got this out of my system, lets talk about Upgrading from Joomla!3 to Joomla!4. Instead of Joomla! changing its version to v4.x because Marketing requires it, it should change its version because the product requires it. And if Joomla! is going to break compatibility, why not take advantage of it, and try to clean its file structure to a more permanent one.
I think that the directory structure can be improved, by being simplified and contextualised. An Upgrade is the right time to do it. Can't be done on an Update.
In my opinion, and in a minimalist installation, the front-end's directory structure should only be composed by 4 files and 1 folder.
The Administrator would look the same, without the .htaccess and robots.txt files (or not). It could actually have it's own media folder. Media Manager would search media files (in the front-end media folder) using the file system, but would display the thumbnails using the site's main domain in the http request, which would be saved in the config.php file at installation time.
The index.php file, would only have 3 instructions.
<?php define('_JEXEC', 1); // Include directory specification file. include_once 'directories.php'; // Call the Engine's main file, that starts execution. include_once JPATH_CORE . 'index.php';
The last included index.php file, would be the starting point for Joomla!'s Engine.
The directories.php file, would hold all environmental path variables used in both the Engine and the Platform (see previous post), and it would look something like this.
<?php defined('_JEXEC') or die; // Defines Web Application ID (if used). define('JPATH_SITE', 'WebSiteID');// site or admin, in the CMSs case. // Defines all path variables. define('JPATH_ROOT', __DIR__); define('JPATH_BASE', '/path/to/public_html/'); define('JPATH_CORE', '/path/to/core/'); define('JPATH_ENGINE', JPATH_CORE . 'bin'); define('JPATH_LIBS', JPATH_CORE . 'lib'); define('JPATH_APPS', JPATH_CORE . 'usr'); define('JPATH_AUTOLOADER', JPATH_CORE . 'opt/autoloader'); define('JPATH_MEDIA', JPATH_BASE . 'media/'); define('JPATH_THEMES', JPATH_APPS . 'templates/'); define('JPATH_CACHE', JPATH_CORE . 'var/cache/'); define('JPATH_TMP', JPATH_CORE . 'var/tmp/'); define('JPATH_LOG', JPATH_CORE . 'var/log/'); define('JPATH_...', '...'); // Other needed values.
The index.php and the directories.php would remain unchanged throughout all J4's lifespan, even if Joomla!'s core gets Updated. This means they can be write protected in the file system.
The media folder(s), would actually hold all media (images & dependencies) necessary for Joomla!'s extensions and templates. It could also have one separate folder created for each extension (possibly created at the extension's installation).
. .. media media / core media / com_extension1 media / com_extension1 / js media / com_extension1 / css media / com_extension1 / img media / com_extension1 / etc... media / com_extension2 media / com_extensionN ...
The Core's (Engine) file structure
If you read the previous post, you know that I proposed that Joomla! should be a Platform on top of an Engine. This means that, the actual file structure would be the Engine's file structure. This file structure can be more organised, and if it were to be changed, maybe a familiar/popular structure should be adopted.
Nowadays, more and more people are familiarised with a *nix file structure, either through a Mac or through a Linux machine. IMO, a stripped down version of this file structure could fit Joomla!'s necessities very well. Linux file structure may look something like this (depending on distribution):
Most of the listed directories make no sense for Joomla!, but some of them do.
The Engine could have the following folders, inside a core parent folder (which wouldn't have a fixed location):
The bin folder would have all the Engine's files (libraries), and would be automatically included at the Engine's start, even if some classes won't be initialised at load.
The etc folder would contain the application configuration file, the Engine's, the Platform's, and all the extension's configuration files as well (but only in global installations).
The lib folder would have the same purpose as the current libraries (and layouts) folder(s). They would get evoked/imported with the help of an Import static class, just like it currently does.
The opt folder would be used, for example, for add-ons to the Engine (Platform or other big custom made package(s)). It would also have the autoload folder for third party add-ons/packages.
The usr folder would contain all user software applications (extensions and templates). Inside the usr folder, the same root folder configuration could be adopted, but its content would be provided only by extensions. The usr folder could look something like this:
- bin (controllers, models and views)
- etc (configuration files)
- lib (code snippets, form fields, etc...)
- log (extension individual logging)
- sql (extension sql scripts)
- lang (extension translations)
Each usr->bin sub folder would be the name of the extension and/or template, and in case of the extensions, each extension folder would only have the following sub-folders: controllers, models, and views (front-end, back-end, etc...). But more on this in the next topic. usr->etc would hold extensions configuration files (config.xml, access.xml and extension_name.xml). usr->lib would extend the root->lib libraries, and therefore, would make their availability system wide. No need to explain the usr->var folder structure and purpose.
And last, we have the (root) var folder, that contains several (variable content) system folders, such as, cache, log (main), lang (main), sql (main) or tmp folder.
Why a file structure like this? What is its purpose?
Well, for the same reason *nix systems use them, and exactly because *nix system use them.
The reason this file structure is used, is because every root folder in a *nix system has its own function, and therefore, usually has its own set of file access permissions. In Joomla!'s case, file permissions might not apply, but being able to separate each directory function might be an asset. The site builder might wan't to change the extension's directory location (or any other) to another location (hiding it).
The Engine and/or Platform could also (hypothetically) be installed centrally in the host, and each website would only have the remaining local installations (each using separate resources). Therefore, you could share Joomla!'s installation between websites.
Joomla!'s directory structure would become more clean and readable, and therefore more understandable. Public folders (site and admin) would therefore get a lot cleaner and organised.
There could be a decoupling of the Administrator (back-office). For what I have been reading online, I don't seam to be the only one wanting the Administrator in a sub-domain of the main site, opposing to having it in a predefined sub-folder.
Cleaner backup. There's no need to backup all the website's files, apart from the usr, var and media folders.
But the main reason is, you could change all the core files location, and put them in a non public area of the host. Joomla!'s php files usually all start with the same validation instruction, to prevent direct file access to those files, and each directory has a default index.html file, but XML and TXT files do not have that validation, and directories might not have the index.html file. Directory protection can be achieved through other means, but this is not done inside Joomla!. If you own one or more Joomla! websites, and you keep an eye on the server logs, you'll know direct file accesses are real.